Blindness and biases
Submitted by Mike Goempel on Wed, 19/07/2006 - 15:10.
I recently presented a "how to sell software testing" to our internal sales staff. For the salespeople that have been selling testing for awhile, this was mostly review. For those with less experience selling testing it was an eye opening, educational experience.
At the conclusion of my presentation, I visually demonstrated how project managers and stakeholders that only allow testing teams to validate requirements from manual test scripts are doing their projects a disservice.
I asked one question of my audience:
How many times do the people wearing white shirts catch the ball?
Please take a few minutes and view the video yourself Video
OK, what is your answer?
Here are the answers I received:
Salesperson #1 - 15
Salesperson #2 - 24
Salesperson #3 - 15
Salesperson #4 - 14
Salesperson #5 - 16
Salesperson #6 - 15
I first saw this video during STMR10 when Cem Kaner was demonstrating "inattentional blindness". My answer that day was 15 (I believe most participants answered 13-17). I have shown this video to many stakeholders, project managers, testers, and developers over the past year. I don't recall anyone answering higher than 18.
I had to ask salesperson #2 how they came to 24 as an answer. "I counted how many times each person caught the ball including how many times the person bounced the ball to themselves." was her answer.
So I replayed it and counted again. This time I counted 21 times. I viewed it again to validate my new answer.
I now realize I have been assuming that "caught" meant how many times one person passed the ball to another person, not how many time the ball is catch. That is not the question I have been asking.
The lesson I learned during my presentation is to ask more questions about even this simplest of tests. I made a basic assumption about this test (as a lot of others have) only to find that I would have written a false bug report.
By the way, what else did your notice about the video? Watch it again (without counting how many times the people catch the ball) and see if you missed something.
At the conclusion of my presentation, I visually demonstrated how project managers and stakeholders that only allow testing teams to validate requirements from manual test scripts are doing their projects a disservice.
I asked one question of my audience:
How many times do the people wearing white shirts catch the ball?
Please take a few minutes and view the video yourself Video
OK, what is your answer?
Here are the answers I received:
Salesperson #1 - 15
Salesperson #2 - 24
Salesperson #3 - 15
Salesperson #4 - 14
Salesperson #5 - 16
Salesperson #6 - 15
I first saw this video during STMR10 when Cem Kaner was demonstrating "inattentional blindness". My answer that day was 15 (I believe most participants answered 13-17). I have shown this video to many stakeholders, project managers, testers, and developers over the past year. I don't recall anyone answering higher than 18.
I had to ask salesperson #2 how they came to 24 as an answer. "I counted how many times each person caught the ball including how many times the person bounced the ball to themselves." was her answer.
So I replayed it and counted again. This time I counted 21 times. I viewed it again to validate my new answer.
I now realize I have been assuming that "caught" meant how many times one person passed the ball to another person, not how many time the ball is catch. That is not the question I have been asking.
The lesson I learned during my presentation is to ask more questions about even this simplest of tests. I made a basic assumption about this test (as a lot of others have) only to find that I would have written a false bug report.
By the way, what else did your notice about the video? Watch it again (without counting how many times the people catch the ball) and see if you missed something.