Why do we do review?
Submitted by Ainars Galvans on Fri, 14/12/2007 - 11:47.
general software testing
We frequently do review to improve either the description of and idea or the idea itself. We also do review to make decisions about the ideas. We do review to validate, to control, to collect an evidence, etc. It’s amazing how different activities are all called review. I’m also wondering why some people ignore alternatives: discussion and trial.
This blog is only prelude to the next one where I plan to specially address subject: test case review. A year old issue coming from older blog why do we write test cases .
intro: Why WHY?
My blogs frequently starts with why? Do you know why? I believe we have enough of so called best practices describing how to do means to do it best. We have enough procedures, guidelines and standards (IEEE, SWEBOOK) telling what to do. We have enough certification programmes teaching those what and how. I think that what we lack in this industry at the moments is realization of why among practitioners.
Types of review - examples
What is a review? In a software development life-cycle there are a lot of quite different activities all called review. They have different reasons, different process, etc. but all called review … Let me mention only a few of them:
1) Requirements document review by testers to learn if they are not ambiguous and testable.
2) Requirements document review by developers to identify features that are had to implement and perhaps change requirements so that it is easier to implement.
3) Code review to identify defect and as bad coding style
4) Defect review by developer to make sure he understood how to repeat the problem
5) Feature request review by management to suggest if this idea (feature) is good enough and need to be added to a software
6) “Employee performance review” – discussion between manager and subordinate to discuss employees’ achievements and further growth options.
Types of review (generic description)
I see a review this way: a person (or a group of people) have examined, analyzed, studied or done other activities upon some source of information (for example: talked to customers to extract requirements, or executed application to learn something about it’s quality). The mental activity produces some sort of result (for review purpose it is better to have it in written). For example requirements document, defect report, test case, code, etc. Now we may want to perform following activities to ensure and improve quality of a result:
1) Repeat information gathering process: i.e. view or see again
2) Repeat the mental activity, i.e. examine or study again
3) Inspect or audit process of mental activity or result documentation. i.e. take a retrospective view of
4) To examine or study the results without deep knowledge of source information i.e. go over (the results) deliberately
Note that I’ve stolen the second part of each item above from dictionary.
There are more meanings. For example “book review” which is yet another study to provide feedback for an audience. There is no goal to improve the book, but only to give a feedback. However in this blog I limit meaning of review to activity aiming to improve quality in any way.
Review specifics in IT
IT industry don’t like to waste time for repeated activities. Quite naturally type 1 described above is not frequently used as it is 100% repeated effort.
Keeping the same principle in mind, I’ve only seen type 2 used in one of the following ways: either a) information source is asked to review the result or b) in cases when paperwork (documenting the results of analysis) takes more than the analysis itself, peer or manager may repeat the study and compare his results with the documented one.
Type 3 – inspections and audits are well known to those who pretend to comply with standards or hold some process certificates (ISO, CMM, etc.). My experience with those have been following: creating documentation that gives no (direct) additional value to our process/customers. However, it may appear valuable once we want to protect ourselves from customers: i.e. if they sue your company, or as competitive advantage (unfortunately).
Nevertheless type 4 is most used in IT. I believe it comes from waterfall-type (including V-model, iterative and other) where different people exchange different artefacts: specifications, design and architecture docs, code, tests, etc. It is most critical that those who get the written artefacts (unambiguously) understand the ideas described in the artefacts. If an audience is single person then the best way is face-to-face discussion or a debate. Review takes place when audience is large enough.
A few more types of review in IT (hybrid review)
So the most typical review process in IT is one aimed to improve the description and optionally (occasionally) to improve the idea itself. I will investigate it in next paragraph but let me first tell you that there are more hybrid review types. Activities that are called “review”:
Hybrid type A: Decision making; committees. The main difference is that person who describe the idea less authority than audience. The audience will decide if the idea is good enough.
Hybrid Type B: Controlling (subordinates). Subordinate describe his actions in so much details that supervisor could review if subordinates is taking appropriate actions. There are two reasons why I call this type hybrid: first – we do not review ideas, but actions; second – this is one-to-one communication better done face-to-face and informally.
Review the description of an idea
In waterfall type life-cycle, where different people analyze requirements, plan architecture, do a design, coding and testing - transition of ideas from one person to another is the most critical aspect. Any ambiguity or misunderstanding will lead to implementation of a false idea. And repeat implementation once anyone realizes that idea is incorrectly transmitted. Whole V&V (validation and verification) term refer to the process of eliminating wrong transmission of ideas at all the stages of waterfall process. Agile proponents may argue that face-to-face communications are faster and better, still there are certain benefits for documenting ideas, as even author may forget the idea the next day, not to mention what will happen when new people are added.
Let me analyze this type of review more detailed. Depending priorities of the goals (for audience to understand the idea, to improve the description of an idea, to improve the idea itself) review process may take different types. For example process known as walkthrough is primary designed for audience to understand the idea. Note that additional gaol in all cases may be for author realize that there are some details of that idea he has not yet clear understanding of. Or details that he/she believes to be self-evident or trivial but is not at a closer look.
Alternative: review the results
I have two years old sun. We play lego. He can’t construct anything reasonable yet, but I he has some ideas what he would like me to build. He don’t speak yet too good to tell me. So what we do – I build something and he tells if and what he likes about it and what he doesn’t. For example, he shows me that doorway is too low for his puppet to fit in. While this is lego I could quickly rebuild the stuff. I don’t think we will ever draw the picture of the house before an attempt to build it. Because reviewing drawing is not as effective and efficient as reviewing the actual building. When building houses they can’t afford it because rebuilding is not as cheap as with lego.
Rebuilding is rather cheap in software development, but not too cheap. That’s why we will all agree that defects are cheaper to remove in requirements than coding.
Endnote
Review is a powerful tool used a lot in software industry, quite frequently in a wrong way. There are a lot of books describing how to implement a good review process. On the other hand many books and publications agree that the process alone don’t guarantee good review. It requires skills. I believe that a prerequisite to develop skills are realization of goals. But they keep documenting only a process. Even tester’s certificate that I hold requires a person to know the types of reviews... They describe different process in terms of roles and activities. I expect that they pretend to have implemented different process for different goals. I believe they are good at describing what they describe. But I can’t see if they address mentoring and skill development. So I hope I’ve filled this gap and informally described types of review from the goals’ prospective.
This blog is only prelude to the next one where I plan to specially address subject: test case review. A year old issue coming from older blog why do we write test cases .
intro: Why WHY?
My blogs frequently starts with why? Do you know why? I believe we have enough of so called best practices describing how to do means to do it best. We have enough procedures, guidelines and standards (IEEE, SWEBOOK) telling what to do. We have enough certification programmes teaching those what and how. I think that what we lack in this industry at the moments is realization of why among practitioners.
Types of review - examples
What is a review? In a software development life-cycle there are a lot of quite different activities all called review. They have different reasons, different process, etc. but all called review … Let me mention only a few of them:
1) Requirements document review by testers to learn if they are not ambiguous and testable.
2) Requirements document review by developers to identify features that are had to implement and perhaps change requirements so that it is easier to implement.
3) Code review to identify defect and as bad coding style
4) Defect review by developer to make sure he understood how to repeat the problem
5) Feature request review by management to suggest if this idea (feature) is good enough and need to be added to a software
6) “Employee performance review” – discussion between manager and subordinate to discuss employees’ achievements and further growth options.
Types of review (generic description)
I see a review this way: a person (or a group of people) have examined, analyzed, studied or done other activities upon some source of information (for example: talked to customers to extract requirements, or executed application to learn something about it’s quality). The mental activity produces some sort of result (for review purpose it is better to have it in written). For example requirements document, defect report, test case, code, etc. Now we may want to perform following activities to ensure and improve quality of a result:
1) Repeat information gathering process: i.e. view or see again
2) Repeat the mental activity, i.e. examine or study again
3) Inspect or audit process of mental activity or result documentation. i.e. take a retrospective view of
4) To examine or study the results without deep knowledge of source information i.e. go over (the results) deliberately
Note that I’ve stolen the second part of each item above from dictionary.
There are more meanings. For example “book review” which is yet another study to provide feedback for an audience. There is no goal to improve the book, but only to give a feedback. However in this blog I limit meaning of review to activity aiming to improve quality in any way.
Review specifics in IT
IT industry don’t like to waste time for repeated activities. Quite naturally type 1 described above is not frequently used as it is 100% repeated effort.
Keeping the same principle in mind, I’ve only seen type 2 used in one of the following ways: either a) information source is asked to review the result or b) in cases when paperwork (documenting the results of analysis) takes more than the analysis itself, peer or manager may repeat the study and compare his results with the documented one.
Type 3 – inspections and audits are well known to those who pretend to comply with standards or hold some process certificates (ISO, CMM, etc.). My experience with those have been following: creating documentation that gives no (direct) additional value to our process/customers. However, it may appear valuable once we want to protect ourselves from customers: i.e. if they sue your company, or as competitive advantage (unfortunately).
Nevertheless type 4 is most used in IT. I believe it comes from waterfall-type (including V-model, iterative and other) where different people exchange different artefacts: specifications, design and architecture docs, code, tests, etc. It is most critical that those who get the written artefacts (unambiguously) understand the ideas described in the artefacts. If an audience is single person then the best way is face-to-face discussion or a debate. Review takes place when audience is large enough.
A few more types of review in IT (hybrid review)
So the most typical review process in IT is one aimed to improve the description and optionally (occasionally) to improve the idea itself. I will investigate it in next paragraph but let me first tell you that there are more hybrid review types. Activities that are called “review”:
Hybrid type A: Decision making; committees. The main difference is that person who describe the idea less authority than audience. The audience will decide if the idea is good enough.
Hybrid Type B: Controlling (subordinates). Subordinate describe his actions in so much details that supervisor could review if subordinates is taking appropriate actions. There are two reasons why I call this type hybrid: first – we do not review ideas, but actions; second – this is one-to-one communication better done face-to-face and informally.
Review the description of an idea
In waterfall type life-cycle, where different people analyze requirements, plan architecture, do a design, coding and testing - transition of ideas from one person to another is the most critical aspect. Any ambiguity or misunderstanding will lead to implementation of a false idea. And repeat implementation once anyone realizes that idea is incorrectly transmitted. Whole V&V (validation and verification) term refer to the process of eliminating wrong transmission of ideas at all the stages of waterfall process. Agile proponents may argue that face-to-face communications are faster and better, still there are certain benefits for documenting ideas, as even author may forget the idea the next day, not to mention what will happen when new people are added.
Let me analyze this type of review more detailed. Depending priorities of the goals (for audience to understand the idea, to improve the description of an idea, to improve the idea itself) review process may take different types. For example process known as walkthrough is primary designed for audience to understand the idea. Note that additional gaol in all cases may be for author realize that there are some details of that idea he has not yet clear understanding of. Or details that he/she believes to be self-evident or trivial but is not at a closer look.
Alternative: review the results
I have two years old sun. We play lego. He can’t construct anything reasonable yet, but I he has some ideas what he would like me to build. He don’t speak yet too good to tell me. So what we do – I build something and he tells if and what he likes about it and what he doesn’t. For example, he shows me that doorway is too low for his puppet to fit in. While this is lego I could quickly rebuild the stuff. I don’t think we will ever draw the picture of the house before an attempt to build it. Because reviewing drawing is not as effective and efficient as reviewing the actual building. When building houses they can’t afford it because rebuilding is not as cheap as with lego.
Rebuilding is rather cheap in software development, but not too cheap. That’s why we will all agree that defects are cheaper to remove in requirements than coding.
Endnote
Review is a powerful tool used a lot in software industry, quite frequently in a wrong way. There are a lot of books describing how to implement a good review process. On the other hand many books and publications agree that the process alone don’t guarantee good review. It requires skills. I believe that a prerequisite to develop skills are realization of goals. But they keep documenting only a process. Even tester’s certificate that I hold requires a person to know the types of reviews... They describe different process in terms of roles and activities. I expect that they pretend to have implemented different process for different goals. I believe they are good at describing what they describe. But I can’t see if they address mentoring and skill development. So I hope I’ve filled this gap and informally described types of review from the goals’ prospective.