testingReflections.com - non-functional testing

CMG Final Agenda

Thu, 06 Nov 2008 14:36:13 -0600

CMG Final Agenda (in pdf) is finally available. Online agenda is available too.

Looks like CMG'08 is one more step towards performance testing community. I believe that CMG was the best conference about computer performance for a long time, but now we have significantly more papers related to performance testing and performance engineering. Conference proceedings from 1976 through 2005 are available to the public – probably the largest collection of performance-related papers (and most of them are practical).

Trends, Papers, and SOA

Wed, 06 Aug 2008 09:56:53 -0500

I quickly found that my capacity of what I can do in addition to regular work is pretty limited: while I was writing a paper for the upcoming CMG conference and then helping a little with the conference organization as CPE (Computer Performance Evaluation) Subject Area Chair (CMG is a volunteer organization), I wasn't able to blog or update my site (probably more energy than time). Now this is mainly over: the CMG preliminary agenda was just published. It is time to summarize and move forward.

CMG News

Thu, 29 May 2008 14:42:12 -0500

It is still not late to submit a paper to the CMG conference. Although the abstract deadline passed, the enforced deadline is only the paper deadline – June 13.

I believe that it is the best conference about computer performance. Conference proceedings from 1997 through 2005 are available to the public – probably the largest collection of performance-related papers (and most of them are practical).

Multiple Dimensions of Performance Testing

Thu, 24 Apr 2008 20:18:00 -0500

Almost all experts agree that pre-deployment "waterfall" performance testing (which, with the record/playback method, confused by many as the performance testing itself) is not enough - too little, too late. Actually it is just one very specific way of performance testing - with a full spectrum of other approaches, which are used so infrequently (at least as intentional performance testing techniques) that I don't recall finding any good classification. Thinking about it, I see several dimensions of performance testing which, although definitely correlated, probably might be considered somewhat independently - of course, just a raw idea for the moment, just an effort to order thoughts a little.

Training for Performance Testers in Brighton UK, May 08.

Sun, 06 Apr 2008 22:14:39 -0500

Scott Barber has teamed up with Rosie Sherry of DrivenQA to bring PTSS Training to Brighton & Hove, UK.

Sign-up now for Performance Testing for Managers or Performance Testing for Software Systems; A Heuristic Approach.

Pitfalls of the "Waterfall" Approach to Performance Testing

Mon, 24 Mar 2008 16:23:17 -0500

Looks like the pre-production validation approach to performance testing becomes typical for large corporations (if there is any at all):

-get the system ready
-develop all scripts requested (sometimes offshore)
-run them all together
-compare with the requirements provided
-allow some percentage of errors according to the requirements
-involve the development team if requirements were missed

Brazil and Uruguay and Conferences, Oh my!

Tue, 01 Apr 2008 23:13:02 -0500

This morning I arrived in Brazil. For the next 2 weeks I’ll be doing some consulting, some training, and speaking at 3 conferences spanning Brazil and Uruguay.

The first is Performance Summit 2008 hosted by Dell Brazil. The local organization team, including my friends and top-notch performance testers Carlos Panato and Walter Munstock, have already gone well out of their way to make me feel at home. Thanks guys!

Re: Performance testing and coverage

Tue, 11 Mar 2008 01:38:38 -0500

Can't leave Ainars Galvans' posting unanswered. I think that it touches very important issues I am fighting for long time with, so this post is going beyond just a comment.

It is very interesting that I completely agree with Ainars on most items except final conclusions – which I completely disagree with. I suspect that it is rather terminology difference. So let's start with what I disagree.

Multi-User Functional Testing

Sun, 02 Mar 2008 15:11:17 -0600

In the February issue of Software Test & Performance read Karen Johnson's article about multi-user testing (pp. 20-23). Karen writes about a very rare subject – functional multi-user testing. Should admit that I started to read with a thought "one more article about performance testing" – but soon realized that it is about quite different subject. And yes, indeed, without functional multi-user testing, most of errors mentioned in article will slip through (won't re-tell the article – it is available on-line, you can read it yourself). Some may be found during performance testing (probably the most severe like deadlocks – if they are in the typical scenarios you included in performance testing) – and you will need to trace them down to the source, and probably it will be much later down the cycle.

CMG Opens Its Content to the Public

Tue, 12 Feb 2008 13:30:21 -0600

The Computer Measurement Group (CMG) is making its conference proceedings from 1997 through 2005 available to the public. I believe that CMG holds the best practical conference in performance analysis, capacity planning, and related areas. In addition to the areas listed below, I'd definitely add performance testing. Here is the official mail CMG sent:

Education

Sat, 02 Feb 2008 23:08:05 -0600

With great interest read Who Killed the Software Engineer? (Hint: It Happened in College) by James Maguire as well as the original article Computer Science Education: Where Are the Software Engineers of Tomorrow? by Dr. Robert Dewar and Dr. Edmond Schonberg.

Performance Requirements

Sun, 06 Jan 2008 17:08:27 -0600

My performance requirements paper was published in the January issue of Software Test & Performance (pp.18-24).

It was simple: I just sent a draft – and now I am reading it printed. With a new name - or even two: it is referred as You Can Gauge Performance Without Requirements in one place and Gauging Performance in The Absence of Measures in another. Not to mention other minor improvement.

Good Advice

Sat, 05 Jan 2008 22:42:20 -0600

I like Good Advice by Graeme Simsion very much: it lists many real traps that you may fall into during consulting engagements (and, truly speaking, overall business communication). Actually, during performance engagements, I face all these issues again and again. While the issues are definitely generic, looks like they are more acute when you touch performance: usually too much egos and money involved, too much work may need to be done, etc.

Privacy, PR and the unpardonable "problematic" breach

Sat, 29 Dec 2007 02:37:56 -0600

At this time of year, various worst-of lists are appearing. Despite privacy being one of the few legally enforcable non-functional requirements for software, there are still some frightening incidents Even worse is the PR propaganda to try to excuse them.

While these are offences under the law, the fact is that many of them do not directly impact the people involved. A new privacy breach that is unforgivable has been occuring in Ohio in the U.S, Some adopted children visiting hospital are being called out with their birth names, and having their details displayed with their birth parents records, not their adopted parents. Some of these children may not even realize they are adopted. And of course, there is some PR as well, a government official said, “Obviously, when information like this is being inadvertently spit out by a computer, it is problematic”. I guess if there was an award for the understatement of the year, that may qualify!

AST Update: Smart Stuff for Career Software Testers

Fri, 22 Aug 2008 12:44:02 -0500

December Issue of the Association for Software Testing Newsletter Now Available