Monday, November 7, 2016

The verification game, or gaming verification


At Critical Uncertainties I read a post that I hope is meant in the best of humor, but actually it might be quite serious

Here's the setup:
  • Customer states requirement
  • Customer states requirement verification protocol
  • Project team implements protocol
  • But wait ... protocol is only statistically applicable
And here's what Critical Uncertainties writes:
".... one realises [SIC] that requirements are 'operationally' defined by their associated method of verification. ..... Now if you're in luck ..... you propose adopting a statistical proof (because it's such a tough requirement and/or there's variability in the process, weasel weasel) of compliance based on the median of a sample of tests.
Using the median is important as it's more resistant to outlier values, which is what we want to obfuscate (obviously).
As the method of verification defines the requirement all of a sudden you've taken the customer's deterministic requirement and turned it into a weaker probabilistic one."

This last thing is the key and merits repeating:
"As the method of verification defines the requirement all of a sudden you've taken the customer's deterministic requirement and turned it into a weaker probabilistic one."
OMG! Did you pull one off on the customer, or did you simply introduce the customer to the realism of the verification protocol?


Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
http://www.sqpegconsulting.com
Read my contribution to the Flashblog