Performance - Related Terminology
Submitted by Alexander Podelko on Fri, 10/06/2005 - 05:34.
performance testing
It is interesting how much confusion is with performance/load testing terminology. There are many different definitions, but I still haven’t seen any precisely defining most of used terms. The best discussion is probably Load Testing Terminology by Scott Stirling (you need to scroll down a little to find it).
The main source of confusion probably is that people try to compare the definitions defining the process from different sides or confuse the process with the term. If you have a small red rubber ball, you won’t spend much time discussing what is difference between a small ball and a red ball. Or you won’t define a small ball by describing how you play baseball. Why that happens with testing?
Load Testing – testing when you load the system. Usually it means work of many (at least more than one) users.
Performance Testing - testing of performance of the system. It can be for one or many users.
If you run a test simulating many users and measuring response times how you should name it? That is the load AND performance test. You can refer to it as the load test or the performance test and both would be correct. They are not synonyms, but they describe different sides of the test. Do you need to name it the load test or the performance test? Whatever is more appropriate in the context (often whatever you like) – what is better to use: the small ball or the red ball?
The same with other adjectives we can use. What if we run a very long test under load and monitor that response times don’t degrade and there are no leaks? We can name it load performance reliability longevity memory leak (perhaps something else) test. How to refer to that test in short depends on what has more sense in the context. Whatever you name it, it is correct as far as it somewhat defines the test.
Does it impact name how we generate load (sitting several persons to do scenario manually, writing a script or using a load testing tools), how heavy is the load or is the system tuned? It adds more definitions, but it is still the load and performance test. For example, a 10-user manual load test of the tuned system.
Tuning and diagnostics are different processes from testing. Yes, quite often load testing is going together with tuning and diagnostics. Sometimes it is difficult to separate them. For example, performance testing of a mistuned system isn’t too meaningful. But still all three of them are different processes, and it is difficult to plan all this activities properly without their separation.
The main source of confusion probably is that people try to compare the definitions defining the process from different sides or confuse the process with the term. If you have a small red rubber ball, you won’t spend much time discussing what is difference between a small ball and a red ball. Or you won’t define a small ball by describing how you play baseball. Why that happens with testing?
Load Testing – testing when you load the system. Usually it means work of many (at least more than one) users.
Performance Testing - testing of performance of the system. It can be for one or many users.
If you run a test simulating many users and measuring response times how you should name it? That is the load AND performance test. You can refer to it as the load test or the performance test and both would be correct. They are not synonyms, but they describe different sides of the test. Do you need to name it the load test or the performance test? Whatever is more appropriate in the context (often whatever you like) – what is better to use: the small ball or the red ball?
The same with other adjectives we can use. What if we run a very long test under load and monitor that response times don’t degrade and there are no leaks? We can name it load performance reliability longevity memory leak (perhaps something else) test. How to refer to that test in short depends on what has more sense in the context. Whatever you name it, it is correct as far as it somewhat defines the test.
Does it impact name how we generate load (sitting several persons to do scenario manually, writing a script or using a load testing tools), how heavy is the load or is the system tuned? It adds more definitions, but it is still the load and performance test. For example, a 10-user manual load test of the tuned system.
Tuning and diagnostics are different processes from testing. Yes, quite often load testing is going together with tuning and diagnostics. Sometimes it is difficult to separate them. For example, performance testing of a mistuned system isn’t too meaningful. But still all three of them are different processes, and it is difficult to plan all this activities properly without their separation.