Agile Performance Testing?

Julian,
Thanks for the comment, I keep forgetting what a small world testing is really, I should have guessed you knew Mike and Scott. I am thinking of actually using the blogs here on www.testingreflections.com as a collaborative writing exercise to expand the thoughts and ideas and to get some more qualitative and quantified information together over and above the limited amounts I can generate through my own direct and company experiences. Hopefully by these shared discussions we will be growing the site in the manner Antony intended. May be it will become a WoprWeb where ideas are not just posted but exchanged and actively critiqued, now not only that be cool but powerful too.

Neill McCarthy
"Agile Testers of the World UNIT !"

Neill,
I'd love to learn more about you experiences and ideas. Please add me to the list of people for the draft material. I know Mike Kelly and Scott Barber reasonably well, so with your permission maybe we could share ideas (maybe you're on your way to running a de-facto WOPR style discussion :-)

Julian Harty

Scott,

What diagram do you refer to? http://www.perftestplus.com/approach.htm ?

Neil, very astute observation! The process flow diagram on perftestplus.com is fundamentally a hybrid model. It is what I developed to communicate the necessarily agile "middle-part" of a performance testing project to devout "proceduralists". The concept is simply that the beginning tasks can be treated sequentially, but once the first test is executed, all bets are off.

I’ve tried to develop a diagram that depicts a “fully agile” approach to performance testing, but I simply don’t have the artistic ability to do it in a way that clarifies more than it confuses. If you come up with one, I’d love to see it!

--
Scott Barber
Chief Technology Officer
PerfTestPlus, Inc.
sbarber@perftestplus.com
www.perftestplus.com

Alex,
At the moment I am aiming at two separate ideas/ concepts under the same banner I guess to some extent.
The first is Agile Performance Testing or Effective Performance testing on agile projects. In this I include the concept and the activity of using performance monitors early in iterations and exercising the user and technical stories with some performance monitoring running around the tests. I also run and plan for the performance aspects and scalability/ capacity aspects to be considered and monitored after each build/ optimisation in the automated regression testing to ensure we are not causing issues of performance within other areas of the code we have not optimised. So I am looking and we are to some extent realising some of the benefits of agility with early performance testing once there are meaningful artefacts.
The second aspect is making performance testing more agile in general even on non-agile projects. In part I have been doing this in a similar way to that mentioned by Scott Barber over on his excellent site [http://www.perftestplus.com/] , I came up with a similar flow to his process diagram prior to discovering his model and have been working with modelling the performance testing to User Stories early by using his UCML approach.
I am happy to distribute the paper as it matures. As I develop the ideas further, exercise them more and collect quantified results then I will also blog on them.
Neill McCarthy
"Agile Testers of the World UNIT !"

Sounds interesting, but what you really mean under "agile performance testing"? I am very interested to look at any materials you can share.

Alex Podelko
apodelko@yahoo.com

Mike,
That could well be part of it and also I think there is a general fear of the new in the European conferences at least, as the target market seems to be for new attendees and new entrants to testing, I know on each of the times I have gone and the "first timers" question is asked that seems to be the largest group. The EuroSTAR 2005 programme has been published and there are very few talks on leading edge subjects as I would define them, or deep technical areas which is a shame.
Anyway I think there is probably a separate blog in the ideas of what more experienced testers want out of conferences but yes, post the internal review this week, I am more then happy to forward the abstract for comment. Thanks for the offer.

Neill McCarthy
"Agile Testers of the World UNIT !"

Performance testing (and agile to some extent), seem to scare conference committees for some reason. I just had a performance test paper accepted to PNSQC, but for the most part, that's my first so far on that topic (that is not vendor specific). I think there is fear because the topic(s) is very challenging.

I would love to review your abstract if you have one handy...
-Mike