Multi-edit heuristic
Submitted by Ainars Galvans on Wed, 07/11/2007 - 08:32.
heuristics
When you test client-server application does not forget to test cases when two users work with the same data simultaneously: one update business item while other deletes it, or one add a new child to a parent moved by another person, or both does rename to the same item...
How to test
Preferably you will need two computers for that. You may apply pair-testing and simply try to work with the same data simultaneously. However I would suggest synchronizing activities done in both clients. If it is web-application fill in forms in both browsers, but press submit simultaneously or with a very little delay. If you work alone you could do it quite well if you have two hands (so if one arm is broken, you can’t do it :) – you could navigate to submit button using tab and then press space simultaneously (it is harder to do that with two mice)
If you test two different activities (e.g. deleting and updating) make sure you tested both cases – when deleing is prior to updating and vice versa.
Well know issue, still missed by developers
It is well know that multi-editing is a big concern for client-server applications: databases have commit/rollback mechanism, source control systems – merging functionality, Lotus Notes – “replication or save conflict” items. The same way some mechanism should be implemented in your application and you have to test it.
Requirements seldom mention this - only in case when working with shared data is explicitly defined by functional feature. An example is any queuing feature, i.e. task management systems may have queue of unassigned tasks and each user could get one task assigned to him the top of the queue.
Nevertheless, it is supposed implicitly that client-server is “multi-edit safe”. You would never write in requirements for client-server application that multiple users should be able to login to system simultaneously, this is self-evident, would you? And if it is not written, everyone tend to forget about it – both developer and tester.
Use Cases
There are so much use cases. For task flow/management type applications suppose a manager reassign task while assignee tries to edit it. For any application having administration functionality suppose admin deletes or updates something that a user is using at the moment…
If you think your application don’t have multi-edit features, think once again. For example it is e-shop, where every user has it’s own chart and data is to no way shared among two users, right? Wrong! Normally e-shops keep track of how much items are still “in stock” and it decreases each time someone buys it. So what will happen if there is only one item left and two users try to buy it simultaneously…
Performance testing begins with two users …
I know three reasons why this heuristic is typically missed by tester: 1) it is not mentioned in testing handbook, I’ve seen 2) There are typically no requirements/use cases of multi editing described (as described above) 3) functional testers may think that it is task for performance testers to discover such issues as there are more than one user involved. On the other hand if there is a bug, then it is functional bug, not a performance or reliability, right? More over performance testing typically have poor functional coverage (functional coverage is not goal for non-functional testing, right?).
Final note – I’ve come to that heuristic only because of my performance testing experience. But as functional test lead I always ask my team to cover multi-editing cases. And we keep finding quite a lot of issues.
How to test
Preferably you will need two computers for that. You may apply pair-testing and simply try to work with the same data simultaneously. However I would suggest synchronizing activities done in both clients. If it is web-application fill in forms in both browsers, but press submit simultaneously or with a very little delay. If you work alone you could do it quite well if you have two hands (so if one arm is broken, you can’t do it :) – you could navigate to submit button using tab and then press space simultaneously (it is harder to do that with two mice)
If you test two different activities (e.g. deleting and updating) make sure you tested both cases – when deleing is prior to updating and vice versa.
Well know issue, still missed by developers
It is well know that multi-editing is a big concern for client-server applications: databases have commit/rollback mechanism, source control systems – merging functionality, Lotus Notes – “replication or save conflict” items. The same way some mechanism should be implemented in your application and you have to test it.
Requirements seldom mention this - only in case when working with shared data is explicitly defined by functional feature. An example is any queuing feature, i.e. task management systems may have queue of unassigned tasks and each user could get one task assigned to him the top of the queue.
Nevertheless, it is supposed implicitly that client-server is “multi-edit safe”. You would never write in requirements for client-server application that multiple users should be able to login to system simultaneously, this is self-evident, would you? And if it is not written, everyone tend to forget about it – both developer and tester.
Use Cases
There are so much use cases. For task flow/management type applications suppose a manager reassign task while assignee tries to edit it. For any application having administration functionality suppose admin deletes or updates something that a user is using at the moment…
If you think your application don’t have multi-edit features, think once again. For example it is e-shop, where every user has it’s own chart and data is to no way shared among two users, right? Wrong! Normally e-shops keep track of how much items are still “in stock” and it decreases each time someone buys it. So what will happen if there is only one item left and two users try to buy it simultaneously…
Performance testing begins with two users …
I know three reasons why this heuristic is typically missed by tester: 1) it is not mentioned in testing handbook, I’ve seen 2) There are typically no requirements/use cases of multi editing described (as described above) 3) functional testers may think that it is task for performance testers to discover such issues as there are more than one user involved. On the other hand if there is a bug, then it is functional bug, not a performance or reliability, right? More over performance testing typically have poor functional coverage (functional coverage is not goal for non-functional testing, right?).
Final note – I’ve come to that heuristic only because of my performance testing experience. But as functional test lead I always ask my team to cover multi-editing cases. And we keep finding quite a lot of issues.