Another aspect of performance requirements
Submitted by Alexander Podelko on Mon, 06/11/2006 - 03:56.
non-functional testing | performance testing
One more aspect in addition to the discussion about performance requirements. If look at performance requirements from another point of view, we probably can classify them into business, usability, and technological requirements (not established terms, feel free to suggest better). Business requirements should come directly from the business before design starts. Something like "a customer representative should enter 20 requests per hour and the system should support up to 1000 customer representatives". So here we got that request should be processed in 5 minutes, throughput would be up to 20,000 requests per hour, and it could be up to 1,000 parallel sessions. Probably the main trap here is to tie these requirements to usability and technology immediately thus limiting the number of available design choices. If we speak about a web system, it is probably possible to squeeze all information into a single page or have a sequence of two dozens of screens. All information can be saved at once or each page of these two dozens can be saved separately.
Another aspect is usability requirements for response times. Many researches, for example, agree that users loose focus if response times are more than 8-10 sec. That is probably an important thing to remember and usability considerations may influence design choices (like breaking into several web pages). Still in some cases response time requirements may have a business meaning too: for example, to make sure that response times are not worse than competitors' (if think about responsiveness as a competitive advantage).
The example of the third category, technological requirements, is results of applying business requirements on the chosen design and apportioning them into components. For example, if we need to make a call to ten web services sequentially to show the web page and we want to show that web page in 3 seconds we probably want each web service to have response time about 200-250 ms. The more we know, the more accurately we can apportion overall response time to web services. Another example of technological requirements is resource consumption requirements. In simplest from it may be that CPU and memory utilization should be below 70% for the chosen hardware configuration.
So business requirements should be elaborated during design and development so we get some specific requirements (or goals) that can be verified during testing and monitored in production.
Another aspect is usability requirements for response times. Many researches, for example, agree that users loose focus if response times are more than 8-10 sec. That is probably an important thing to remember and usability considerations may influence design choices (like breaking into several web pages). Still in some cases response time requirements may have a business meaning too: for example, to make sure that response times are not worse than competitors' (if think about responsiveness as a competitive advantage).
The example of the third category, technological requirements, is results of applying business requirements on the chosen design and apportioning them into components. For example, if we need to make a call to ten web services sequentially to show the web page and we want to show that web page in 3 seconds we probably want each web service to have response time about 200-250 ms. The more we know, the more accurately we can apportion overall response time to web services. Another example of technological requirements is resource consumption requirements. In simplest from it may be that CPU and memory utilization should be below 70% for the chosen hardware configuration.
So business requirements should be elaborated during design and development so we get some specific requirements (or goals) that can be verified during testing and monitored in production.