Software Testing and the Total Depravity of Man
Submitted by John McConda on Mon, 28/01/2008 - 07:20.
I was intrigued by a discussion that went on recently in the software-testing Yahoo message board. Matt Heusser started the conversation, spun off of another thread in the forum about Exploratory vs. Scripted testing (itself always a fun topic for debate :) . The thread started with a question about what kind of testing discovers defects and what kind verifies functionality.
I find very interesting the philosophical angle to this verify vs. break question. It sounds rather simple on the surface, but I think any tester’s answer to the question “Are you verifying the software works or are you trying to break it?” says everything about that tester, the project they are working on, and the work atmosphere as a whole. In short, do you believe the software is basically good, and your job is to “verify” this belief, or are you certain that it was born bad-to-the-bone buggy, and it's your job to find as many defects as you can?
We see these two ways of thinking in philosophies of humankind. The doctrine of Total Depravity comes from Augustine's concept of Original Sin. It says basically that we’re all born bad and need rehabilitating. This conflicts sharply with other philosophies that declare human beings to be born basically good, and it is life events that turn some to evil (see Cooper, the early writings of Rousseau, etc.). Like the break vs. verify question, this seemingly simple dichotomy extends farther than noticed at first glance, influencing virtually every social issue debated today, including criminal law, welfare, and taxes.
Unlike these two philosophies though, I think one’s answer to the break vs. verify question is likely to change, even in the middle of a project. A great example of this often happens during regression testing. Much of this testing is conducted at the end of the project, when there is not much time left to fix any issues that are found. Often, testers who find a high quantity, or any quantity of defects at this stage are chastised for “not finding these sooner”, so what is the motivation to break the software? Very little. In fact, there is high motivation not to find any defects. At this stage we want to believe the software is basically good. Actually, we are defending the testing we did earlier with this final set of tests, proving once and for all that it’s “ready to ship!”
Perhaps then we could look at the act of testing as some sort of rehabilitation. We believe the software is born evil, full of nasty bugs. We as testers come in, point out its “sins”, and the developers “repent” of their evil ways, turning the application into a respectable young program, ready to burst out into the world. We see our final regression tests as the ones we are confident, but desperately hoping our promising program will pass this time, proving it’s ready to be part of society. Then we have “verified” the rehabilitation and our final product is thrust forth upon the world, “cleansed” of all its evils...hopefully....
I find very interesting the philosophical angle to this verify vs. break question. It sounds rather simple on the surface, but I think any tester’s answer to the question “Are you verifying the software works or are you trying to break it?” says everything about that tester, the project they are working on, and the work atmosphere as a whole. In short, do you believe the software is basically good, and your job is to “verify” this belief, or are you certain that it was born bad-to-the-bone buggy, and it's your job to find as many defects as you can?
We see these two ways of thinking in philosophies of humankind. The doctrine of Total Depravity comes from Augustine's concept of Original Sin. It says basically that we’re all born bad and need rehabilitating. This conflicts sharply with other philosophies that declare human beings to be born basically good, and it is life events that turn some to evil (see Cooper, the early writings of Rousseau, etc.). Like the break vs. verify question, this seemingly simple dichotomy extends farther than noticed at first glance, influencing virtually every social issue debated today, including criminal law, welfare, and taxes.
Unlike these two philosophies though, I think one’s answer to the break vs. verify question is likely to change, even in the middle of a project. A great example of this often happens during regression testing. Much of this testing is conducted at the end of the project, when there is not much time left to fix any issues that are found. Often, testers who find a high quantity, or any quantity of defects at this stage are chastised for “not finding these sooner”, so what is the motivation to break the software? Very little. In fact, there is high motivation not to find any defects. At this stage we want to believe the software is basically good. Actually, we are defending the testing we did earlier with this final set of tests, proving once and for all that it’s “ready to ship!”
Perhaps then we could look at the act of testing as some sort of rehabilitation. We believe the software is born evil, full of nasty bugs. We as testers come in, point out its “sins”, and the developers “repent” of their evil ways, turning the application into a respectable young program, ready to burst out into the world. We see our final regression tests as the ones we are confident, but desperately hoping our promising program will pass this time, proving it’s ready to be part of society. Then we have “verified” the rehabilitation and our final product is thrust forth upon the world, “cleansed” of all its evils...hopefully....