I quickly found that my capacity of what I can do in addition to regular work is pretty limited: while I was writing a paper for the upcoming CMG conference and then helping a little with the conference organization as CPE (Computer Performance Evaluation) Subject Area Chair (CMG is a volunteer organization), I wasn't able to blog or update my site (probably more energy than time). Now this is mainly over: the CMG preliminary agenda was just published. It is time to summarize and move forward.

It is still not late to submit a paper to the CMG conference. Although the abstract deadline passed, the enforced deadline is only the paper deadline – June 13.

I believe that it is the best conference about computer performance. Conference proceedings from 1997 through 2005 are available to the public – probably the largest collection of performance-related papers (and most of them are practical).

I looked through the new LoadRunner certifications.

Accredited Integration Specialist in HP Performance Center v9. Two Prometric exams, $150 each.

Accredited Systems Engineer in HP Performance Center v9. Two exams, look somewhat similar to what Mercury had before, $750 each.

Almost all experts agree that pre-deployment "waterfall" performance testing (which, with the record/playback method, confused by many as the performance testing itself) is not enough - too little, too late. Actually it is just one very specific way of performance testing - with a full spectrum of other approaches, which are used so infrequently (at least as intentional performance testing techniques) that I don't recall finding any good classification. Thinking about it, I see several dimensions of performance testing which, although definitely correlated, probably might be considered somewhat independently - of course, just a raw idea for the moment, just an effort to order thoughts a little.

Looks like the pre-production validation approach to performance testing becomes typical for large corporations (if there is any at all):

-get the system ready
-develop all scripts requested (sometimes offshore)
-run them all together
-compare with the requirements provided
-allow some percentage of errors according to the requirements
-involve the development team if requirements were missed

Can't leave Ainars Galvans' posting unanswered. I think that it touches very important issues I am fighting for long time with, so this post is going beyond just a comment.

It is very interesting that I completely agree with Ainars on most items except final conclusions – which I completely disagree with. I suspect that it is rather terminology difference. So let's start with what I disagree.

See the message from the CMG Program Chair below. And yes: load and stress testing, benchmarking, performance optimization, software performance engineering, resource management, capacity analysis are among the major topics for the conference.

The CMG'08 conference committee is delighted to announce that they are now accepting abstracts for papers and workshops for the 2008 conference to be held in Las Vegas in December 2008.

In the February issue of Software Test & Performance read Karen Johnson's article about multi-user testing (pp. 20-23). Karen writes about a very rare subject – functional multi-user testing. Should admit that I started to read with a thought "one more article about performance testing" – but soon realized that it is about quite different subject. And yes, indeed, without functional multi-user testing, most of errors mentioned in article will slip through (won't re-tell the article – it is available on-line, you can read it yourself). Some may be found during performance testing (probably the most severe like deadlocks – if they are in the typical scenarios you included in performance testing) – and you will need to trace them down to the source, and probably it will be much later down the cycle.