Do you keep notes about “the first below the line” tests?
Submitted by Ainars Galvans on Tue, 12/02/2008 - 16:25.
exploratory testing
We can’t test everything. So it would be logical to divide tests into must and may (if time permit)” test. To have some tests reserved... Nevertheless, in continuous integration projects I have “it’s now or never” rule as testing new features. I may postpone whole feature testing, but specific test ideas – never. I’m not sure if I’m right – that’s why I’m writing this.
Principle: the first below the line
We run test training classes for students on a regular basis, but not everyone is allowed to participate. There is a process likely to hiring process to get into the small (10-12 persons) group. During an interview we decide if this person is worth investing our time in training him. For some students it is clear enough, but there are a few of those that I say – they are the first below the line. It means that if we can’t get 10 suitable persons this one will get in. Typically those are the tricky people – one that we can’t figure out if we they will grow into a tester. It is straight-forward we risk investing our time into them, though we risk missing good testers if we don’t.
I’m using the principle in Performance testing
I sometimes use the same principle in performance testing. There are certain tests that my stakeholders want me to run but there are much more tests that I could come up with – those that could potentially outline a critical performance issue, but I’m not sure if they will. So I may spend time testing them and find nothing, nothing worth to report. Testing them I risk to waste my time; not testing – I risk to miss important information (and I believe that testing has it’s main goal to provide important information).
So I tend to write down those ideas and return back to them later – if I have enough time. Or I may chat with developers and other stakeholders to get some risk assessment from them – developers may assess probability, managers – impact. And risk is impact multiplied by impact.
My Exploratory Testing hint: it’s now or never…
I just realized that when I do scripted testing (and I do script regression tests sometimes) I keep writing down those test ideas to try if time permit (or even divide test steps into must and may). On the other hand – when I do exploratory testing I seldom write down ideas to come back later to test. Especially when testing new functionality. Main reason is this: my stakeholders hate to get information late (late-found-bugs). If I can’t find a bug today, tomorrow will be too late. With a few exceptions of course, but the rule of thumb is this – if I will only be able to find this bug tomorrow – it means it can’t be critical, because customer will not run into this bug soon enough. More over I believe this rule keeps me focused on testing as much as possible today. Besides, I know - there will be more features to test tomorrow …
What I’ve just wrote maybe context-specific: in continues integration projects too much late-found-bugs putting developers at stress and letting even more defects get into code as a result. That’s what I’ve seen a lot. That’s where ET helped me – because with scripted tests many defects and incomplete functional
Principle: the first below the line
We run test training classes for students on a regular basis, but not everyone is allowed to participate. There is a process likely to hiring process to get into the small (10-12 persons) group. During an interview we decide if this person is worth investing our time in training him. For some students it is clear enough, but there are a few of those that I say – they are the first below the line. It means that if we can’t get 10 suitable persons this one will get in. Typically those are the tricky people – one that we can’t figure out if we they will grow into a tester. It is straight-forward we risk investing our time into them, though we risk missing good testers if we don’t.
I’m using the principle in Performance testing
I sometimes use the same principle in performance testing. There are certain tests that my stakeholders want me to run but there are much more tests that I could come up with – those that could potentially outline a critical performance issue, but I’m not sure if they will. So I may spend time testing them and find nothing, nothing worth to report. Testing them I risk to waste my time; not testing – I risk to miss important information (and I believe that testing has it’s main goal to provide important information).
So I tend to write down those ideas and return back to them later – if I have enough time. Or I may chat with developers and other stakeholders to get some risk assessment from them – developers may assess probability, managers – impact. And risk is impact multiplied by impact.
My Exploratory Testing hint: it’s now or never…
I just realized that when I do scripted testing (and I do script regression tests sometimes) I keep writing down those test ideas to try if time permit (or even divide test steps into must and may). On the other hand – when I do exploratory testing I seldom write down ideas to come back later to test. Especially when testing new functionality. Main reason is this: my stakeholders hate to get information late (late-found-bugs). If I can’t find a bug today, tomorrow will be too late. With a few exceptions of course, but the rule of thumb is this – if I will only be able to find this bug tomorrow – it means it can’t be critical, because customer will not run into this bug soon enough. More over I believe this rule keeps me focused on testing as much as possible today. Besides, I know - there will be more features to test tomorrow …
What I’ve just wrote maybe context-specific: in continues integration projects too much late-found-bugs putting developers at stress and letting even more defects get into code as a result. That’s what I’ve seen a lot. That’s where ET helped me – because with scripted tests many defects and incomplete functional