Software testing is improved by good bug reporting

I recently completed (successfully, I might add) the second of the Association for Software Testing's all online, free to members Black Box Software Testing course. Each of these courses is four weeks in length. I've been involved with this program since years before it became a program, and I am an instructor for the first course in the series, called Foundations. For this course, called Bug Advocacy, I was a student.

I've been thinking about the way agilistas handle bugs recently. Several years ago, I was the editor of an internal IT newsletter for a large Australian financial organisation. Every month, I'd include a critical thinking puzzle, and I select a correct entry to win 2 movie tickets. I was able to give these out to my Australian readers, but I used to get some entries from our Indian IT shop as well. I arranged to have them win 2 movie tickets as well, if they were chosen as the winner. I thought this was a comparable prize, then I discovered that movie tickets are very cheap in India. In Australia, the prize would pay for a weeks public transport, but in India it would be only a day or two.

Ever heard “You report too much defects” from a developer or project manager? Before blaming them for being not committal for quality, try understanding that number of defects is not the only (and even not the right) quality measure. With this post I will try to share my experience on assuming responsibility for not-reporting certain defects – performing triage, thus helping to achieve better quality (due to both logical and psychological reasons).

Dear colleagues,

We are debating what should go into a list in our defect tracking system for specifying "Defect Cause", for example:
- Programming Error
- Missing Requirements
- Test Case Error
Etc.

Since every team has a little or a lot to add, the list is getting a little too complicated to handle and agree upon.

Do you know of a popular, famous, industry-best-practice, or simply common list of defect causes that we can look at for reference?

Questions are frequently asked regarding evaluating defect priority, severity and differentiating between the two values. I would like to explain my vision, based on best practices in our company and supported by IEEE standard, and some recent publications by recognized experts in testing area.
Issue
I chose IEEE standard, because it address issue of evaluating different properties of the defect. For example CMM talks only about severity and type. IEEE (IEEE Std 1044-1993) consider a lot of different values for an anomaly. It requires “The impact of an anomaly shall be considered at each step of the anomaly process”. However I will only talk about initial evaluation by tester. Further IEEE state that “Identifying the Severity of an anomaly is a mandatory category as is identifying the Project schedule and Project cost impacts of any possible solutions for the anomaly.” The standard only suggest priority to be evaluated: “Additional categories that may be useful are Customer value and Priority.”

Pop quiz: A tester finds and logs a bug. Some days (weeks (months)) later a developer picks up the bug and runs through it. Lo and behold, the bug does not occur. What should the developer do with this bug?

Most common answer: Resolve the bug as "No Repro" and send it back to the tester. Bzzt! Wrong! An electric shock to that part of your body you care most about. The bug jolly well does repro or I wouldn't have logged it. Use the build I was using when I found the bug and I guarantee it will reproduce.

Joel writes some stuff in Hebrew and asks the readers to help him translate them.

In the translation post, (which is a good read) joel posts about what he feels the process should be like when he reports a bug in the software:

[textile]Bret Pettichord's blog entry "A bug is anything that bugs someone who matters":http://www.io.com/~wazmo/blog/archives/2004_09.html#000222 was of topical interest to me... I often lose time trying to explain to developers that when I log a test-failure in the 'incident tracking' system, there could be many causes. My test, test-code, environment and even the software under test could contain the fault or bug.

What I love about Bret's post is the phrase "A bug is anything that bugs someone who matters"... that is often all we need developers and project managers to understand...

Bombarding them with definitions with subtle variations is more than anyone else needs to know... it won't change how I conceptualise bugs in my mind...