To DbE or Not DbE?

I spotted a tweet by Mark Needham referring to a blog post by Brad Wilson coining the phrase "Design by Example".

I like the Design by Example (DbE) name. It's nice. It rolls off the tounge, more so than BDD in my opinion. But it misses out on the "development" (in the literal sense of the word) in the way that phrases ending in "-Driven Development" don't. It does better emphasise the initial purpose of TDD/BDD/DbE - i.e. design. Although design is the initial purpose, the automated tests or 'executable design specification' provide you with the safety to refactor, as Brad himself says.

Thus despite design being the initial purpose, it's not necessarily the main purpose since your ability to refactor is equally important. Fowler explains it nicely here:

These days it's terribly unfashionable to talk about Test Driven Development (TDD) having anything to do with testing, but like Jon I do think that having a comprehensive automated test suite is more valuable than the term 'side-effect' implies. Rather like if someone offered me a million dollars to hike up a mountain. I may say that the main purpose of the hike is the enjoyment of nature, but the side-effect to my wallet is hardly trivial....

DbE or BDD

One of the comments against Brad's post asks how DbE is a better name than 'BDD'.

Although BDD started from a similar place (wanting to avoid the word Test), it is now much more than just synonymous with TDD... it makes the outside-in approach to specification by example explicit. I.e. starting with an 'application behaviour example' (acceptance test) and working inwards progressively driving out just enough design with 'class-level behaviour examples' (unit tests).

[Fact] or fiction?

Brad and co have, as a result of the misconceptions caused by people over-emphasising the word 'test', changed their xUnit [Test] attribute to [Fact]. Brad explains this as it fitting well with their [Theory] attribute:

At the Austin event, we decided to rename [Test] to [Fact]. While some users have lamented this as an arbitrary change, we felt it was right because it removed the focus of the word “test” from the code that you were writing when doing DbE. It also lined up very well with the [Theory] feature we added to xUnit.net Extensions to support David Saff’s data theories.

The essential difference is: a [Fact] is an expression of some condition which is invariant, whereas a [Theory] is an expression of a condition which is only necessarily true for the given set of data. As such, [Theory]s are driven with external data; if you provide invalid data, then the theory will potentially produce invalid results.

I don't think this is quite right. It believe that it makes more sense to use [Example]...

In science, A theory is 'rule' or explanation for a collection of obsevations. Those observations are 'indisputable' facts (see nice explanation on about.com). We may 'test' a theory by predicting input/output examples and the theory is supported as being true all whilst the outcome of those tests are as predicted (i.e. pass). A test result is an indisputable observation (i.e. a 'fact'). Therefore a test failure is no less a 'fact' than a test passing. Calling the 'test method' a [Fact] is misleading in this regard since the 'test' explains what we predict. Essentially the unit-test (in this instance) is a means by which we can apply the example and observe the outcome and ascertain whether the outcome was as predicted (i.e. if our theory is correct).

Setting aside the scientific viewpoint, the (Cambridge) dictionary definition of "fact" supports this viewpoint:

something which is known to have happened or to exist, especially something for which proof exists, or about which there is information

In TDD/BDD/DbE, when we've written a test and it fails - rather than change the theory, we change the context by reflecting the 'theory' or concept in the code. We then re-run the test and observe it passing. Both the failure and the pass-result are facts as they were indisputable observations. Observations aside, each test is also an example. So, if a 'test' is analogous to 'example' and a test-result is analogous to a 'fact' then a suitable name for what we once called a [Test] would instead be [Example].

And when we change something that breaks another 'test' that 'test' as described is no longer a fact... but the failure of the test is indeed a fact. Use the word [Example] and the science metaphor doesn't break down so badly.

Another good reason to not use the word 'Fact' is... In practice, I find that my tests evolve and change as my understanding changes. The word 'fact' is a very hard term. It seems and feels rigid and unchangeable. The term 'example' is a much softer term that feels more open to change... "that was an example based on my understanding at the time, but instead this new example better represents my latest understanding". Now replace "example" in that sentence with "fact" and it doesn't work so well. I think [Fact] goes against the evolving comprehension that occurs during TDD/BDD/DbE.

[Theory] fits, sort of...

The [Theory] concept does, however, work in how they've applied it - sort of. A 'Theory' summarises a collection of examples, or tests that have been observed to support the theory. Again, these examples are are only 'facts' whilst you observe that the predicted outcomes are true. Furthermore, it's only a theory whilst the 'method' has no failures. It is essentially disproven until there are no failures. I personally wouldn't arrive at the point of having a theory until I'd first made several observations with specific examples so a [Theory] is most useful in aggregating several related examples that I've already explored and proven. In essence, I would explore the concept first with examples and then infer the theory from their outcome.

Clarifying the concepts

Example is a specific instance of an input/output combination.

Test is a means of applying that example in a given context allowing for the outcome to be observed, passing if the predicted outcome and the actual outcome are the same.

Fact is an indisputable observation, such as a test-result, whether passed or failed.

Theory is an explanation for a collection of facts.

Thus, if we must avoid the word test (even though scientists seem to have no problem with the word - only us software folk) then [Example] is a much better choice IMHO... especially in light of Brad's phrase "Design by Example".

To DbE or not DbE

As for whether I'm going to start using the phrase "Design by Example"... I am not sure. I think it's becoming a bit of a problem with so many terms flying around. Even BDD had, and still has, a hard time gaining mainstream traction. I think it would take a summit of many industry thought leaders, including people like Kent Beck & Ward Cunningham, to arrive at a sensible conclusion on re-branding the practice and announce it... much like they did when "lightweight methods" was changed to "Agile Methods" in 2001...

Perhaps that's exactly what we need... the "Example-driven Approaches Summit"? Who wants to help me organise that one then?