Exploratory testing
Chance-based testing
Submitted by Ainars Galvans on Mon, 12/01/2009 - 16:59. context-driven testing | exploratory testing…If chance and risk are synonyms, then risk-based testing should not be too far from chance-based testing, should it? No I’m serious. Or do you think following quotes are from people who knows too little about test techniques?
“How can we maximize the chance of finding such a problem in the limited time we have to test?”
“I was lucky enough in my random selection of order for retesting the fixes that the system was in a state for me to get some simple extra tests in for the previous feature.”Casinos makes money on roulette because they knows statistics. Testing is not as random as dice roll, but we could make a few conclusions if we analyze statistics. One of the conclusions that I make is following: it makes a lot of sense to vary tests (minefield analogy) when we do it first few times. But as we do regression again and again the effect is lost.
Do you write test cases to kill time?
Submitted by Ainars Galvans on Mon, 08/12/2008 - 10:52. exploratory testingI wouldn’t believe myself until saw this really happening. Testers keep writing detailed test cases up-front with the primary goal to justify the time they spend early on the project. And right they are (are they?). At least they believe so. Managers see evidence that testers have red and understood all the requirements, covered them by test cases. Testers don’t have to think how to prove time spent on reviewing requirements even if they don’t find any issues there. And if developers delay a milestone we have more time to polish the test cases, do one more review or them, write more details, etc. Test lead doesn’t have to worry that on the next project testers will be added much later on the project schedule. Everything is fine but quality.
Could a novice tester do Exploratory Testing?
Submitted by Ainars Galvans on Tue, 04/11/2008 - 18:01. exploratory testingI think that the question is wrong itself. The question is what a novice „should”. I do believe that for most of the novices this is not the best way to learn testing art/skills, but it depends on a person. However! To learn testing by following scripts (I mean detailed instructions for manual testing) written by others is even worse, much worse!
ET thoughts: The Seeker (CKA) heuristic
Submitted by Erik Petersen on Sun, 20/04/2008 - 07:53. exploratory testing | heuristicsSome people are more into mnemonics than others. I can recall walking along with James Lyndsay one day. This in itself was unusual because we are normally on opposite sides of the earth. We were discussing how there are many great mnemonics for test ideas, but neither of us was able to recall the items, just the mnemonics! I think this strongly influences our exploratory testing approaches. Alan Richardson has blogged about this here . Because it is general, It doesn’t really need a mnemonic, but I will give it an acronym. Seeker, CKA, Challenge Key Assumptions. While any tester can do this, the experienced tester will be more skilled at identifying key assumptions, and be more effective at using it. If you have less experience, or can remember mnemonics (!), maybe try the standard ones! You’ll find some links by searching for “test ideas” at my testingspot.net site
Discovery vs. Confirmation
Submitted by Antony Marcano on Sat, 29/03/2008 - 07:44. exploratory testing | perspectives | test driven developmentWhen I hear debates about scripted vs. exploratory testing... or even debates such as "automate all tests" vs. "you can't automate all tests"... I don't think I'm hearing the real debate.
I think the debate that I'm hearing is "testing is about confirmation" vs. "testing is about discovery".
This distinction in the underlying intent of a given approach seems to not be emphasised in these discussions.
In simple terms, I think of the intent behind testing as having two facets:
- Confirmation - Does the system do what we anticipated it should do?
- Discovery - Does the system do (or not do) anything we did not anticipate it would (or should do)?
Ad-hoc vs. Exploratory Testing
Submitted by Antony Marcano on Wed, 12/03/2008 - 13:47. exploratory testingIn various discussions I've been having lately I've tried to help people distinguish between ad-hoc testing (or as some people say - having a play) and exploratory testing...
Ad-hoc testing in comparison to exploratory testing is like the difference between someone who only runs when they are late for their train and a professional athlete that competes in 100 metre sprints.
The person that runs for the bus simply runs instinctively - yet the professional athlete, with the sports-science consultant, studies ways to optimise their running, trains and practices to improve both their power and technique (such as with the Pose method).
The minute you even read about exploratory testing, you are stepping away from ad-hoc (having a play) testing. The mere fact that you appreciate that it isn't only instinctive but realise that there is a method to what may at a glance seem easy to do is the day you are no longer just "having a play".
Ad-hoc testing in comparison to exploratory testing is like the difference between someone who only runs when they are late for their train and a professional athlete that competes in 100 metre sprints.
The person that runs for the bus simply runs instinctively - yet the professional athlete, with the sports-science consultant, studies ways to optimise their running, trains and practices to improve both their power and technique (such as with the Pose method).
The minute you even read about exploratory testing, you are stepping away from ad-hoc (having a play) testing. The mere fact that you appreciate that it isn't only instinctive but realise that there is a method to what may at a glance seem easy to do is the day you are no longer just "having a play".
Do you keep notes about “the first below the line” tests?
Submitted by Ainars Galvans on Tue, 12/02/2008 - 16:25. exploratory testingWe can’t test everything. So it would be logical to divide tests into must and may (if time permit)” test. To have some tests reserved... Nevertheless, in continuous integration projects I have “it’s now or never” rule as testing new features. I may postpone whole feature testing, but specific test ideas – never. I’m not sure if I’m right – that’s why I’m writing this.
How do I know when I'm done... (again)
Submitted by Antony Marcano on Wed, 06/02/2008 - 18:08. exploratory testing | extreme programming (XP)In the context of an XP-style Story/Acceptance Test Driven Development process, that uses exploratory testing to generation intra-iteration feedback... The question “how do we know we are done” is often asked and I'm not sure I always give or get the best answer. One of the problems is that there is so much context... so, here's one way you might know (based on my better experiences)...
That sweet check-in...
That sweet check-in...