Skip navigation.

Non-functional testing

Trends, Papers, and SOA

non-functional testing | performance testing
I quickly found that my capacity of what I can do in addition to regular work is pretty limited: while I was writing a paper for the upcoming CMG conference and then helping a little with the conference organization as CPE (Computer Performance Evaluation) Subject Area Chair (CMG is a volunteer organization), I wasn't able to blog or update my site (probably more energy than time). Now this is mainly over: the CMG preliminary agenda was just published. It is time to summarize and move forward.

CMG News

non-functional testing | performance testing | performance testing patterns | performance testing tools
It is still not late to submit a paper to the CMG conference. Although the abstract deadline passed, the enforced deadline is only the paper deadline – June 13.

I believe that it is the best conference about computer performance. Conference proceedings from 1997 through 2005 are available to the public – probably the largest collection of performance-related papers (and most of them are practical).

Multiple Dimensions of Performance Testing

non-functional testing | performance testing | performance testing patterns
Almost all experts agree that pre-deployment "waterfall" performance testing (which, with the record/playback method, confused by many as the performance testing itself) is not enough - too little, too late. Actually it is just one very specific way of performance testing - with a full spectrum of other approaches, which are used so infrequently (at least as intentional performance testing techniques) that I don't recall finding any good classification. Thinking about it, I see several dimensions of performance testing which, although definitely correlated, probably might be considered somewhat independently - of course, just a raw idea for the moment, just an effort to order thoughts a little.

Pitfalls of the "Waterfall" Approach to Performance Testing

design & development | non-functional testing | performance testing | performance testing patterns
Looks like the pre-production validation approach to performance testing becomes typical for large corporations (if there is any at all):

-get the system ready
-develop all scripts requested (sometimes offshore)
-run them all together
-compare with the requirements provided
-allow some percentage of errors according to the requirements
-involve the development team if requirements were missed

Brazil and Uruguay and Conferences, Oh my!

events | general software testing | industry recognition | non-functional testing | performance testing

This morning I arrived in Brazil. For the next 2 weeks I’ll be doing some consulting, some training, and speaking at 3 conferences spanning Brazil and Uruguay.

The first is Performance Summit 2008 hosted by Dell Brazil. The local organization team, including my friends and top-notch performance testers Carlos Panato and Walter Munstock, have already gone well out of their way to make me feel at home. Thanks guys!

Re: Performance testing and coverage

non-functional testing | performance testing | performance testing patterns
Can't leave Ainars Galvans' posting unanswered. I think that it touches very important issues I am fighting for long time with, so this post is going beyond just a comment.

It is very interesting that I completely agree with Ainars on most items except final conclusions – which I completely disagree with. I suspect that it is rather terminology difference. So let's start with what I disagree.

Multi-User Functional Testing

functional test patterns | functional testing | non-functional testing | performance testing
In the February issue of Software Test & Performance read Karen Johnson's article about multi-user testing (pp. 20-23). Karen writes about a very rare subject – functional multi-user testing. Should admit that I started to read with a thought "one more article about performance testing" – but soon realized that it is about quite different subject. And yes, indeed, without functional multi-user testing, most of errors mentioned in article will slip through (won't re-tell the article – it is available on-line, you can read it yourself). Some may be found during performance testing (probably the most severe like deadlocks – if they are in the typical scenarios you included in performance testing) – and you will need to trace them down to the source, and probably it will be much later down the cycle.