The Telephone Game...
Submitted by Antony Marcano on Sun, 10/08/2008 - 11:03.
test driven development
In an earlier post, I mentioned Boehm's distinction between 'verification' and 'validation':
Verification - Am I building the product RIGHT?
Validation - Am I building the RIGHT product?
-Boehm, Software Engineering Economics
As I was writing an article to appear in the September 2008 issue of Better Software Magazine I arrived at a better way of getting my point across...
Test-last approaches to software development are the equivalent of playing the telephone game...
Let's revisit Boehm's differentiation of Verification and Validation. Really, though, what is the right product?
The right product is a product that meets the real requirements. The real requirements are the things that people will actually need to do with the system once they can use it. Traditional specifications (e.g. UML models, use cases, natural-language functional specifications, and the like) attempt to model our anticipation of these real-world requirements in a conceptualizing and generalizing way. I say generalizing because the rules within the specification contain variables rather than specific values — i.e. these rules apply generically for any relevant value used in place of the variables.
Of course, we assume that such generalizing specifications weren’t plucked out of thin air and were derived from concrete examples of anticipated usage in the real-world. I would hope that the business analyst involved in that process derived the rules from examples... In such test-last approaches, however, those examples are often lost or, at best, are second class class citizens to the model.
So on projects that derive tests (or examples of anticipated usage in the real world) from a generalizing specification (or model) that was itself derived from examples of anticipated real-world usage... is at best verification against the abstraction and at worst the software development equivalent of the "Telephone Game".
Acceptance Test Driven Development starts with concrete examples of anticipated real-world usage and automatically evaluates the software against those examples. Short of mind reading and time travel, this is about as close as we can get to validation — as close as we can get to knowing if we are building the right product!