Generalising Specifications vs. Specification by Example - Verification vs. Validation

Specification by example is the concept underlying TDD (in all its forms). In essence, each test is an expression of an example of how a system (in the case of acceptance tests) or component (in the case of unit tests) will be used.

If you want to know why TDD is often described as a form of "specification by example" one of the best analogies is Jason Gorman's Kitchen (I know - strange that he has deviated from golf for a change).

Something I've noticed in the TDD discussions on the web is that the phrase "formal specification" is used to label other more typical techniques of specification. Formal specification has a specific meaning of course, but I'm not sure it is a fair term as I consider the application of "specification by example" in the form of TDD and Acceptance-Test-Driven Development as being very formal.

What I think differentiates "specification by example" from other specification techniques is that the other approaches produce "generalising specifications".

Such generalising specifications are assumed to have evaluated the requirements and produced a generalising specification - maybe use cases, class diagrams (and the like). These specifications abstract the requirements into generalised rules and models that are assumed to fulfil the requirements.

There is a fundamental problem with such generalising specifications...

The formal specification community have constantly had trouble verifying that a design satisfies a specification, particularly in doing this without error prone humans. For Specification By Example, this is easy.

-Martin Folwer

In fact, I think it's deeper than that...

Verification - Am I building the product RIGHT?

Validation - Am I building the RIGHT product?

-Boehm, Software Engineering Economics

In the case of generalising specifications, it's (i) hard to verify the system against the specifications; (ii) hard to validate the specifications against the real requirements; (iii) it is harder still to validate the system against the requirements due to the specifications abstracting the requirements from the system.

Thus - generalising specifications make it extraordinarily difficult to determine whether or not the specification tells you to build the RIGHT product, let alone determining if the system you are building is indeed the RIGHT product.

Specification by example does allow you to "validate" - i.e. answer that very important question - "am I building the RIGHT product?" (regardless of what the product-of-concern is for a given example - the system or just a class). Some may find it useful to specify the system in other ways - hardcore XPers may prefer to think of the code as the only place we need generalise the rules and models that fulfil these examples. Whichever way - validation is clearly one of the strengths of Specification by Example.