testingReflections.com - performance testing

WOPR11 Call For Proposals(CFP)

Sun, 10 Aug 2008 19:27:01 -0500

Theme: Reliability… what can we do?

People involved in developing, testing, and delivering hardware, software, or internet based applications must be able to ensure those solutions meet customer and user expectations… The question that follows – Exactly what are those expectations and how do you ensure they are being met?

WOPR11 will explore the topic of reliability with seasoned professionals, including architects, designers and performance and reliability testers.

Trends, Papers, and SOA

Wed, 06 Aug 2008 09:56:53 -0500

I quickly found that my capacity of what I can do in addition to regular work is pretty limited: while I was writing a paper for the upcoming CMG conference and then helping a little with the conference organization as CPE (Computer Performance Evaluation) Subject Area Chair (CMG is a volunteer organization), I wasn't able to blog or update my site (probably more energy than time). Now this is mainly over: the CMG preliminary agenda was just published. It is time to summarize and move forward.

5 Questions with Scott Barber by a Braidy Tester

Wed, 06 Aug 2008 01:29:44 -0500

I recently had the honor of being interviewed by Michael Hunter, a Braidy Tester, for Dr. Dobbs Portal. Check it out: 5 Questions with Scott Barber.

--
Scott Barber
President & Chief Technologist, PerfTestPlus, Inc.
Executive Director, Association for Software Testing
Co-Author, Performance Testing Guidance for Web Applications

"If you can see it in your mind...
you will find it in your life."

CMG News

Thu, 29 May 2008 14:42:12 -0500

It is still not late to submit a paper to the CMG conference. Although the abstract deadline passed, the enforced deadline is only the paper deadline – June 13.

I believe that it is the best conference about computer performance. Conference proceedings from 1997 through 2005 are available to the public – probably the largest collection of performance-related papers (and most of them are practical).

Scott Barber and AST now on Twitter

Mon, 19 May 2008 09:06:24 -0500

Recently I succumbed to peer pressure and tried Twitter. Twitter is a "micro-blogging" site. It took me a while to figure out what that meant and what value it has, but after trying it, I get it now. It's a great way to share little bits of information worth sharing, but that doesn't (or often doesn't *yet*) justify a whole blog post. I find that I like it. It's a quick and easy way to keep folks up to date on what I'm doing and/or thinking about, like giving conference talks, or trying to flesh out some half-baked performance testing ideas.

I don’t use math in performance testing, do I ?

Thu, 08 May 2008 08:26:55 -0500

I’ve seen testers recommending The Art of War or Weinberg books which are not about testing at all. I’ve also seen performance testers recommending knowledge of probability theory, statistics and modeling principles. I don’t apply this knowledge in performance testing myself – well at least not directly. I never think about things like distribution function, mean deviation, etc. Do you? Don’t I ?!

LoadRunner Certification

Fri, 11 Apr 2008 13:50:45 -0500

I looked through the new LoadRunner certifications.

Accredited Integration Specialist in HP Performance Center v9. Two Prometric exams, $150 each.

Accredited Systems Engineer in HP Performance Center v9. Two exams, look somewhat similar to what Mercury had before, $750 each.

Multiple Dimensions of Performance Testing

Thu, 24 Apr 2008 20:18:00 -0500

Almost all experts agree that pre-deployment "waterfall" performance testing (which, with the record/playback method, confused by many as the performance testing itself) is not enough - too little, too late. Actually it is just one very specific way of performance testing - with a full spectrum of other approaches, which are used so infrequently (at least as intentional performance testing techniques) that I don't recall finding any good classification. Thinking about it, I see several dimensions of performance testing which, although definitely correlated, probably might be considered somewhat independently - of course, just a raw idea for the moment, just an effort to order thoughts a little.

Training for Performance Testers in Brighton UK, May 08.

Sun, 06 Apr 2008 22:14:39 -0500

Scott Barber has teamed up with Rosie Sherry of DrivenQA to bring PTSS Training to Brighton & Hove, UK.

Sign-up now for Performance Testing for Managers or Performance Testing for Software Systems; A Heuristic Approach.

Pitfalls of the "Waterfall" Approach to Performance Testing

Mon, 24 Mar 2008 16:23:17 -0500

Looks like the pre-production validation approach to performance testing becomes typical for large corporations (if there is any at all):

-get the system ready
-develop all scripts requested (sometimes offshore)
-run them all together
-compare with the requirements provided
-allow some percentage of errors according to the requirements
-involve the development team if requirements were missed

Brazil and Uruguay and Conferences, Oh my!

Tue, 01 Apr 2008 23:13:02 -0500

This morning I arrived in Brazil. For the next 2 weeks I’ll be doing some consulting, some training, and speaking at 3 conferences spanning Brazil and Uruguay.

The first is Performance Summit 2008 hosted by Dell Brazil. The local organization team, including my friends and top-notch performance testers Carlos Panato and Walter Munstock, have already gone well out of their way to make me feel at home. Thanks guys!

Performance testing - still in the post-hoc ghetto of a test-driven world?

Mon, 17 Mar 2008 05:11:45 -0500

As some of you may recall, a couple of years ago I ran a workshop on Agile Performance testing.

You can find out more about this by searching for WOPR7.

The general theme, with the exception of one experience report, focused on post-hoc performance testing. A lot of work has been done over the last decade to advance functional test-driven development. When I say 'functional' I don't mean 'application level tests' (a.k.a. acceptance tests); I mean tests at the level of unit, acceptance and every level in between where these tests are concerned with functional behaviours (of the unit or the application respectively).

Performance: art and/or technique(s)?

Wed, 12 Mar 2008 10:32:24 -0500

What follows is nothing new. I only try to take another look to what have been written so far. Scott Barber recently commented The whole situation is sad. Tool vendors dominate the training market (at least in performance testing) . Perhaps I don’t care for training as I train my people myself, but I care for stereotype created by dominating vendors. I will bring my own understanding of by-effect caused by load tools bringing the nice load tools. No I don’t say that performance testing is no more an art. But there are certain limits we are yet to break.

Re: Performance testing and coverage

Tue, 11 Mar 2008 01:38:38 -0500

Can't leave Ainars Galvans' posting unanswered. I think that it touches very important issues I am fighting for long time with, so this post is going beyond just a comment.

It is very interesting that I completely agree with Ainars on most items except final conclusions – which I completely disagree with. I suspect that it is rather terminology difference. So let's start with what I disagree.

Performance testing and coverage

Mon, 10 Mar 2008 08:46:48 -0500

I’m doing something like performance testing as consultant now. First thing I say to my customer is this: I will not replicate real application usage! I will only emulate simultaneous clicking in a few places of your application: tiny functional coverage; incomplete environment and data; no emulation of unexpected events… It reminds me of a joke about physicists analyzing a spherical horse in a vacuum.