Skip navigation.

Multi-User Functional Testing

functional test patterns | functional testing | non-functional testing | performance testing
In the February issue of Software Test & Performance read Karen Johnson's article about multi-user testing (pp. 20-23). Karen writes about a very rare subject – functional multi-user testing. Should admit that I started to read with a thought "one more article about performance testing" – but soon realized that it is about quite different subject. And yes, indeed, without functional multi-user testing, most of errors mentioned in article will slip through (won't re-tell the article – it is available on-line, you can read it yourself). Some may be found during performance testing (probably the most severe like deadlocks – if they are in the typical scenarios you included in performance testing) – and you will need to trace them down to the source, and probably it will be much later down the cycle.

I am afraid that very few companies really do functional multi-user testing (as described in the article) – although this is just a speculation based on my personal limited experience.

By the way, it is a good illustration that multi-user testing and performance testing are not synonyms. We can have single-user performance testing and multi-user functional testing.

Ainars, Thanks for the com

Ainars,

Thanks for the comments on my article. Multi-user testing may be picked up by performance testing but in my experience it's not. Not because it couldn't be but because by the time we fight for the tool, the environment or whatever else might be needed to execute perf testing, we're impatient to move onto tests with more users and to find other types of issues. It would likely be overkill to use a tool to find these types of multi-user errors.

Glad you found the article helpful.

Karen

...part of functional testing training...

Ainars,

It gets mentioned in passing fairly frequently, but I've not seen it called out separately and explicitly in any of the popular resources that I can remember. I could look up some specific examples if you are interested.

--
Scott Barber
Chief Technologist, PerfTestPlus
Vice President & Executive Director, Association for Software Testing
sbarber@perftestplus.com

It must be a part of functional testing training, mustn't it?

Karen is right. For me - a tester who does both functional and performance testing - it seems quite obvious to include it in functional testing. It seems that I've described a few months ago exactly what Karen is writing about in my blog multi-edit heuristic. I teach my colleagues to do so and I review their functional tests for covering multi-edit cases. But ... I feel like I'm quite unique.
I've never seen any functional testing book talking about this type of testing. Have I missed them? Any trainings you are aware of?

Another reason to suppliment tool vendor training...

I've been talking/writing/ranting about this one for years. I've always referred to it as "functional testing under load", but Karen's article is completely in alignment with what I teach about checking functionality under realistic usage scenarios.

This is just one more thing that the tool vendors don't talk about in their training (and barely support if at all with their tools). The reason this is a problem is because (in my experience, most) vendors sell their performance test training as "send your folks to this training and they will come back able to performance test your application" -- and management has a bad habit of believing it.

I don't fault the vendors for not teaching this (or many other topics) in their courses. I do fault the ones who mislead their customers into believing that the training is more than than it is. I fault the folks who purchase/choose/budget for training for *still* not figuring out that vendor training is about tools, not about testing.

The whole situation is sad. Tool vendors dominate the training market (at least in performance testing). Performance testers continues to lag behind other testers (and development roles) in maturity across the industry. Companies continue to experience significant, costly, avoidable, and embarrassing performance issues in production. Yet, the folks who are trying to educate the industry and the testers are largely dismissed by the vendors (who want the training dollars for themselves) and the managers (who have a tendency to believe the company with the largest annual revenue), thus perpetuating the problem.

It's all just very sad.

--
Scott Barber
Chief Technologist, PerfTestPlus
Vice President & Executive Director, Association for Software Testing
sbarber@perftestplus.com

Comment viewing options

Select your preferred way to display the comments and click 'Save settings' to activate your changes.