My (testing practice) motto: equilibrium (Yin Yang)

I'm used to disagree with people since my childhood. If they say that X is black I will oppose with evidence that there are something white as well. If they say X is white I will also oppose with an evidence of black. It doesn’t mean I believe myself that X is either black or white. I believe nothing is either way, nothing but things that we would never start debating. This is how I see most of the controversy topics in testing: scripted and exploratory, automated and manual, etc. Even if I write myself X is white I actually mean that it is also white despite some people say it is only black. Why do I do it? To debate, to learn the reasons of a person I'm opposing. I don't care what he believes but why.

Test Scripting equilibrium

There are people who never buy Exploratory Testing ideas, at least not all of them. There are people who insist on pure Exploratory testing. I’ve been proponent of scripted testing. I believe that just as test automation test design up-front is what we must do whenever we can. And that decision on how detailed tests scripts must be is only derived from how much we will need to reuse them. If we need to do regression tests frequently we need to document tests to the details that even a monkey could do test execution. I was amazed to learn how ET could make testing faster more effective and more efficient as a result. I gave up scripting and missed a lot of bugs. I’ve almost learned to find the equilibrium, the golden mean.

Test automation equilibrium

I’ve seen people saying that automation should be done at API level instead of UI. I’ve seen people talking about type of functionality best suited for automation (like keyword-driven testing is the best because you could increase coverage by simply generating more keys). I don’t think this is so black or white. There is an equilibrium I’ve Presented (sorry it’s in Latvian, but you may try to read the pictures) in 2002 analyzing automation efficiency in term of: how fast/easy it is to automate, how fast/easy it is to run tests manually instead and how big the regression risk is there. That was par of my master degree so analysis was abstract, based on models with certain assumptions that does not always hold true. However it is backed up with my experience that sum of those arguments stays quite balanced despite of at which level you automate or what coverage you target. There is no single silver bullet. It depends on context. Although I see now a lot of issues in my old analysis I still believe in equilibrium regarding all (but trivial) automation decisions.

Quality and Functionality; Yin Yang again

…. seemingly opposing forces are bound together, intertwined, and interdependent in the natural world, giving rise to each other in turn.

With a poor quality functionality is doomed. With poor functionality none will use our product despite the outstanding quality. I disagree with those who says we are ready to ship when all critical bugs are fixed (and there are no more than X medium level bugs). I think we could ship when quiescence is reached.

Yin-yang is an active concept: yin and yang are thought to arise together from an initial quiescence or emptiness, and to continue moving in tandem until quiescence is reached again. For instance, dropping a stone in a calm pool of water will simultaneously raise waves and lower troughs between them, and this alternation of high and low points in the water will radiate outward until the movement dissipates and the pool is calm once more.

I’ve learned how to see the waves in our product as it progress through the new release cycle. The problem is that more stones are dropped as we move forward and I know SCRUM sprint idea is specially designed to shield such stones from balling until the end of the sprint. There are more practices for other/internal risks that may damage the balance.