Tester doing design review = test design. (Experience)
Submitted by Ainars Galvans on Wed, 10/01/2007 - 07:58.
design & development patterns | general software testing
I continue blog my mini-experiences . While I see almost none interested in answering my questions before myself I will provide solution I’ve applied to the problem in the initial post already. This time I want to show something about design review. If you read the design and simply try to understand what is wrong there you will most probably fail to find problems. This will be the review the developers could do. As a tester you should try to imagine how would you test it. Design tests.
The problem description
I was testing the following object (it was a little bit more complex, but for this post it does not makes any difference). The requirements are: to have a component that takes files from file system and move them into database (as blobs or something). There will be a triggers in DB that will process the file. Error handling should move bad files (validated by DB trigger) to ErrorFolder and do not commit into DB files that are impossible to delete due to corruption or anything.
The design is described as algorithm:
Infinite Loop
__Pick the first file available (in the configured path)
__If no file found sleep a second (configurable amount)
__Else
____Try{
______Start DB transaction.
______Put file content into DB.
______Delete original file from file system
______Commit transaction
______}
____On Exception Rollback transaction and move file to error folder
__End if no file found
End of infinite loop
My review
It looks OK from the first look. There are one (to my mind) quite obvious issue. delete original file happens before commit. What if a commit is unsuccessful afterwards – the file is gone, isn’t it? Still everything else looks OK. However there are more problems besides this one and a good tester should be able to recognize all the problems without waiting for the first one to be fixed.
To realize more issues with this design I suggest to test each line of it. Consider how do we test for exception to happen on each step (line). How could exception happen during Starting DB transaction? If a DB down, correct? This will end up moving all the files despite how correct they are into error node! This is wrong, isn’t it? The failure of starting DB transaction has nothing to do with the file. So if that action fails we need to retry it several times and completely stop the server if this is impossible, right?
As we go further the next problem appears only on the last action in the design On Exception Rollback transaction and move file to error folder
. How about error during trying to move file to error folder? We could do that for example by having a file with such a name already in error folder? What will happen – either of files will be lost or the importing the same file will start over again and loop endless.
The problem description
I was testing the following object (it was a little bit more complex, but for this post it does not makes any difference). The requirements are: to have a component that takes files from file system and move them into database (as blobs or something). There will be a triggers in DB that will process the file. Error handling should move bad files (validated by DB trigger) to ErrorFolder and do not commit into DB files that are impossible to delete due to corruption or anything.
The design is described as algorithm:
Infinite Loop
__Pick the first file available (in the configured path)
__If no file found sleep a second (configurable amount)
__Else
____Try{
______Start DB transaction.
______Put file content into DB.
______Delete original file from file system
______Commit transaction
______}
____On Exception Rollback transaction and move file to error folder
__End if no file found
End of infinite loop
My review
It looks OK from the first look. There are one (to my mind) quite obvious issue. delete original file happens before commit. What if a commit is unsuccessful afterwards – the file is gone, isn’t it? Still everything else looks OK. However there are more problems besides this one and a good tester should be able to recognize all the problems without waiting for the first one to be fixed.
To realize more issues with this design I suggest to test each line of it. Consider how do we test for exception to happen on each step (line). How could exception happen during Starting DB transaction? If a DB down, correct? This will end up moving all the files despite how correct they are into error node! This is wrong, isn’t it? The failure of starting DB transaction has nothing to do with the file. So if that action fails we need to retry it several times and completely stop the server if this is impossible, right?
As we go further the next problem appears only on the last action in the design On Exception Rollback transaction and move file to error folder
. How about error during trying to move file to error folder? We could do that for example by having a file with such a name already in error folder? What will happen – either of files will be lost or the importing the same file will start over again and loop endless.