Mast the Context-Driven banner
Submitted by Ainars Galvans on Tue, 02/01/2007 - 15:46.
context-driven testing | functional testing | general software testing
Reading, commenting and reading comment for Schools of software testing by Cam Kaner I realized I’m lucky to be in a project where I could apply testing according to the (changing) context rather than according to an existing (static) process. In order to stay lucky I should never change my team and even the company I work for. I don’t feel good about it, although I like my current project for other reasons as well.
I dream it to be an industry standard and my managers to know that testing may take different form(s) just as programming language(s) differ from project to project; and using a legacy language or language not designed for the purpose is a critical risk for the project. I’m afraid however it is impossible given project managers a) want testing to be managed (resource-planned , progress-monitored) without understanding how it works; b)tend to believe in best-practices.
An example: product testing context
This morning colleague of mine sent me a link to an article in IBM WebSphere Developer Technical Journal . Despite the title talking about developer becoming tester I found in the article a lot of product testing context that I’m familiar with. Tester is the one who validate that a new feature fit the purpose not only the design/specification. Developer is not the only person to make errors. Architect, Business Analyst, marketing and sales people – they are all humans with tide schedules. Despite fact that defects are cheaper to fix if found earlier (e.g. better in design rather than development phase), a lot of them are easier (if not “only possible”) to find once the working code is ready. The Agile and especially TDD proponents would say that it is simpler to write and run a test compared to doing reviews of design documents. They seem to be unaware of risks blindly implementing such an approach as the only one.
In classic V-model there are idea of tests (UAT) designed based on the initial customer input. The idea is somehow degraded however. UAT is only associated nowadays with demonstrating customer that their requirements are fulfilled. It is not associated with testing techniques aimed at defect discovery. Every good idea could be degraded if applied without context.
Starting series of mini-experience stories
I plan to post early this year and continue later short testing stories. I will divide each story in two parts: one part will describe a particular feature and a testing task within a given content; in the second: how I solved the situation and defects discovered. I will post them with a delay so that you may consider the solution and check see if you would find the bug(s). Help me to check for more bugs. I encourage you to use it in training and even hiring as you need/want.
With my stories I hope to demonstrate (stress) that text-book ideas such as testing techniques, testing at different levels, design reviews, etc. requires analyze of a context and certain creativity in real life if you really want to discover defects. If you don’t, you could find excuses like “there are infinity test and limited time, that’s why we never discovered that” or “it was never explicitly described that the feature should work as customer actually expected”, but I prefer do the job instead of proving why I didn’t.
List of stories
I will update this list, as more stories will be available.
- designing repeatable tests
- equivalence classes
- design review
- defect reporting
I dream it to be an industry standard and my managers to know that testing may take different form(s) just as programming language(s) differ from project to project; and using a legacy language or language not designed for the purpose is a critical risk for the project. I’m afraid however it is impossible given project managers a) want testing to be managed (resource-planned , progress-monitored) without understanding how it works; b)tend to believe in best-practices.
An example: product testing context
This morning colleague of mine sent me a link to an article in IBM WebSphere Developer Technical Journal . Despite the title talking about developer becoming tester I found in the article a lot of product testing context that I’m familiar with. Tester is the one who validate that a new feature fit the purpose not only the design/specification. Developer is not the only person to make errors. Architect, Business Analyst, marketing and sales people – they are all humans with tide schedules. Despite fact that defects are cheaper to fix if found earlier (e.g. better in design rather than development phase), a lot of them are easier (if not “only possible”) to find once the working code is ready. The Agile and especially TDD proponents would say that it is simpler to write and run a test compared to doing reviews of design documents. They seem to be unaware of risks blindly implementing such an approach as the only one.
In classic V-model there are idea of tests (UAT) designed based on the initial customer input. The idea is somehow degraded however. UAT is only associated nowadays with demonstrating customer that their requirements are fulfilled. It is not associated with testing techniques aimed at defect discovery. Every good idea could be degraded if applied without context.
Starting series of mini-experience stories
I plan to post early this year and continue later short testing stories. I will divide each story in two parts: one part will describe a particular feature and a testing task within a given content; in the second: how I solved the situation and defects discovered. I will post them with a delay so that you may consider the solution and check see if you would find the bug(s). Help me to check for more bugs. I encourage you to use it in training and even hiring as you need/want.
With my stories I hope to demonstrate (stress) that text-book ideas such as testing techniques, testing at different levels, design reviews, etc. requires analyze of a context and certain creativity in real life if you really want to discover defects. If you don’t, you could find excuses like “there are infinity test and limited time, that’s why we never discovered that” or “it was never explicitly described that the feature should work as customer actually expected”, but I prefer do the job instead of proving why I didn’t.
List of stories
I will update this list, as more stories will be available.
- designing repeatable tests
- equivalence classes
- design review
- defect reporting