In the previous post, we implemented the insertion method of a binary heap using Test Driven Development (TDD) and parameterized unit tests (I'll leave the full implementation of the insertion method as an exercise).

In this post, we will take a closer look at the development flow that we used and show how it relates to traditional TDD. For many people, combining TDD and automated test generation makes no sense. I believe this is not true anymore, and this is what this post is about.

Test Driven Development Flow

TDD has a well-defined flow where developers

1. write a unit test,
2. run the test and watch it fail,
3. fix the code,
4. run the test and watch the test pass. Start again (I'll skip refactoring in this discussion)

During this flow, practitioners also refer to the green state when all tests are passing and red state when some tests are failing. The little picture below depicts this little state machine.

A key aspect of this approach is that the design of the API is inferred by writing the 'scenarios', i.e. tests. Therefore, unit tests are a critical building block of the TDD flow.

Note also that xUnit-like test frameworks (pick your favorite framework here) provide the automation tools so that the execution of the test and the investigation of a failure is painless for the programmer.

Test Driven Development Flow With Parameterized Unit Tests

Parameterized unit tests and Pex change the TDD flow while retaining its essence: building the design from tests. Here are the steps, we'll discuss them in detail later on:

1. Write a parameterized unit test,
2. Run Pex and watch some generated unit tests fail,
3. Fix the code,
4. Run Pex again,
   1. Some previously generated unit tests now pass, at least one new failing unit test get generated (go to 3)
   2. All generated tests are passing, start again (go to 1)

The key difference is the shortcut from step 4 (generating unit tests) to 3 (fix the code), without passing through step 1 (write a new test). This is illustrated by the yellow feedback loop in the diagram below:

To the risk of repeating myself, let me emphasize some important points here:

• you still need to write unit tests, it's just that they can have parameters: Pex generates unit tests from parameterized unit tests, by instantiating them. The person who writes the parameterized unit test is you, not the tool!
• it is still about design: using parameterized unit tests is as much about design as closed (i.e. parameterless) unit tests. In fact, one can argue that parameterized unit tests are way closer to a specification that closed unit tests.
• it is test first: in case it was not obvious by now :)

The shortcut from 4 to 3

As mentioned above, the main difference in the flow is that jump from running the tool (step 4) to fixing the code (step 3), without writing new tests (step 1). This happens because a parameterized unit tests captures equivalence classes rather than a single scenario like closed unit tests. As a result,

• you spend more time fixing/implementing the code: the nice thing about the shortcut is that you can spend more time writing the code, rather than writing tests.
• you can leverage automated white-box testing tools: Pex tries to get the maximum coverage out of your parameterized unit test (note remember that getting coverage also means covering the throwing branches in Assert), using an automated white-box code analysis. Now that you have also those manycore CPUs on your motherboard, you can finally make a good use of them :)

To Pex and not to Pex

An important aspect of parameterized unit tests (and tools like Pex) is that you do not have to drop (completely) your existing habits: in many cases, it is easier to write closed unit tests! In fact, you can always start from a closed unit test and refactor it later. We do not expect users to parameterized unit tests write exclusively (another nice read here), but when you do write them, we expect you'll get much more 'out of your buck'.

In future posts, we'll discuss different ways to write parameterized unit tests: from refactoring existing unit tests to using test patterns.

To be continued...

Next time, we'll go back to the heap and look at implementing the 'remove minimum' method...

(Pex is a automated structural testing tool from Microsoft Research. More information at http://research.microsoft.com/pex.)