Load Testing
Submitted by Alexander Podelko on Mon, 11/10/2004 - 19:54.
performance testing
Testing of multi-user applications under realistic and stress loads is really the only way to ensure appropriate performance and reliability in production.
Load testing is emerging as an engineering discipline of its own, based on “classic” functional testing from one side, and system performance analysis and capacity planning from another side. The terminology is still vague in this field and the borders are fuzzy, but from a practical point of view, we probably can break down all testing into two big classes: those with a multi-user load (load, performance, stress, volume, etc.) and those without it (functional, regression, etc.). The different terms of each of these classes specify why we are testing and for what result we are looking, rather than what process is being used. There is no precise definition for each term; the exact meaning can differ from source to source. For example, performance testing could mean that we are more interested in response time; load testing could mean that we want to see the system behavior under a specified load, and stress testing could mean that we want to find the system’s breaking point. We still do the same things: apply a multi-user load and get some metrics. The difference is only in the details: what precise load we apply and what metrics are more important to us.
As all these terms reflect goals of testing, these two classes do not match the terms completely: we can do performance testing for one user measuring response times with a stopwatch, or test the functionality of the system while having a 100-user load in the background. Moreover, performance is a part of functionality in some sense. We would still refer to testing under multi-user workload as load or performance testing here and contrast it to classical, one-user functional testing.
Both classes are considered testing and have a lot in common. Still, load testing has significant specifics and requires some special approaches and skills. Quite often, applying the best practices and metrics of functional testing to load testing results in disappointments. It looks like many things that are trivial for an experienced performance engineer could drop from the attention of a person less experienced in this field that quite often results in unrealistic expectations, not optimal test planning and design, and misleading results.
Load testing is emerging as an engineering discipline of its own, based on “classic” functional testing from one side, and system performance analysis and capacity planning from another side. The terminology is still vague in this field and the borders are fuzzy, but from a practical point of view, we probably can break down all testing into two big classes: those with a multi-user load (load, performance, stress, volume, etc.) and those without it (functional, regression, etc.). The different terms of each of these classes specify why we are testing and for what result we are looking, rather than what process is being used. There is no precise definition for each term; the exact meaning can differ from source to source. For example, performance testing could mean that we are more interested in response time; load testing could mean that we want to see the system behavior under a specified load, and stress testing could mean that we want to find the system’s breaking point. We still do the same things: apply a multi-user load and get some metrics. The difference is only in the details: what precise load we apply and what metrics are more important to us.
As all these terms reflect goals of testing, these two classes do not match the terms completely: we can do performance testing for one user measuring response times with a stopwatch, or test the functionality of the system while having a 100-user load in the background. Moreover, performance is a part of functionality in some sense. We would still refer to testing under multi-user workload as load or performance testing here and contrast it to classical, one-user functional testing.
Both classes are considered testing and have a lot in common. Still, load testing has significant specifics and requires some special approaches and skills. Quite often, applying the best practices and metrics of functional testing to load testing results in disappointments. It looks like many things that are trivial for an experienced performance engineer could drop from the attention of a person less experienced in this field that quite often results in unrealistic expectations, not optimal test planning and design, and misleading results.