Skip navigation.
Home » blogs » neill mccarthy's blog

Agile performance testing - Index and start of an irregular series of blogs.

Submitted by neill mccarthy on Sat, 04/06/2005 - 17:26. agile | non-functional testing | performance testing | perspectives
[Textile]
Well an earlier blog "Agile Performance testing?":http://www.testingreflections.com/node/view/2186 has led to quite a lot of comments and information flow between other interested parties. It has also led to a number of links which have provided additional background reading which I am grateful for as they have help clarify my own thinking.
The original intent was to develop the concept and paper behind closed doors and circulate it once it had reached a higher level of maturity. However I am of the view this is one of those ideas that might just have legs and should be explored more fully in a public forum to assist it with its development. Hopefully this will lead to some early quantifying across a range of contexts and some qualitative sharing of experiences. Both of these types of feedback have real value as described by "Tim Van Tongeren":http://www.testgeek.com/cgi-bin/blosxom.cgi
So to some extent this is approach to the growing body of work will be agile & collaborative writing, hopefully once it has been developed at demonstrated it will become something that is used by others and can grow via conferences and workshops.

At the moment I have followed this idea up on around half a dozen projects. I do not have the benefits clearly quantified at the moment it feels right and sensible though. As meaningful, relevant and contextualised metrics are gathered, where they are not subject to NDAs or confidentiality clauses, they will be posted too as part of the series.
The list below may not be the final titles or even potentially the sequence they will appear in: yet they are the final logical sequence that I feel they will follow once it is formalised I am not sure where it is going though as a final destination: a paper, an article, a conference talk or remaining as a blog.

Currently the main hope for this evolving series of blogs is to open the discussion and allow for the idea to find its feet or be shot down in a flaming hail of bullets.

All comments thoughts and views are greatly received; if it does develop further, all contributors will be created with their input as is only right and proper: The working sections for the blogs at least are listed below, I suspect they will evolve overtime and the titles might change to protect the innocent. Links to the actual blogs will be added as they appear, now that "Antony Marcano":http://www.testingreflections.com/user/view/2 has finally got me to use the correct mark ups. Proposed Blog Titles in this series are listed below:

"Performance testing in Agile Projects: Notes on the series of blogs.":http://www.testingreflections.com/node/view/2286

Performance testing in Agile Projects: Concepts & early thoughts

Performance testing in Agile Projects: Approaches

Performance testing in Agile Projects: People and Skills

Performance testing in Agile Projects: Tooling

Performance testing in Agile Projects: Tool smiths and agile automation

Performance testing in Agile Projects: Exploratory Performance Charters.

Performance testing in Agile Projects: Metric Patterns

Performance testing in Agile Projects: Using and managing the information

Performance testing in Agile Projects: Emergent Architectures & Requirements

Performance testing in Agile Projects: Environments and data

Performance testing in Agile Projects: Managing Velocities

Performance testing in Agile Projects: Issues

Performance testing in Agile Projects: Modelling

Performance testing in Agile Projects: Retrospectives

Performance testing in Agile Project: Benefits

Performance testing in Agile Projects: Additional references and further reading

Agile Performance testing: Applying the benefits in other project approaches.
---
» neill mccarthy's blog | login or register to post comments

Count Me In

Submitted by David Gilbert on Wed, 06/07/2005 - 02:36.
Neill -- very interesting idea. As luck would have it, I am currently involved in a very large, long term engagement with lots of performance testing involved. While I can't officially sign my client up for anything, as I was explaining to someone recently...

"At this point in my career, I don't always TELL my client I am doing exploratory testing...they ask me to test something, and I do, then I bring them some results and explain them. Usually they are happy to have new, valuable information. If I just happened to DO some exploratory testing along the way, oh well."

We are in fact right now doing what I would call exploratory performance testing, so maybe I can contribute, provide some examples and metrics (with appropriate information obfuscation to protect the innocent...oh wait, none of us is really innocent...) and help flesh out the idea. Let me know how you would like all of this to proceed...public blog, email, etc. Besides, you know I can't resist anything with the word Exploratory in it. ;)


I will also be happy to contribute to Tool Smiths and Agile Automation in any way that I can, my experience is all Mercury, but a fair amount of it.


Sincerely,
David Gilbert
» login or register to post comments

re: process

Submitted by neill mccarthy on Tue, 05/07/2005 - 07:40.
Antony,
I agree the process/methods/practices used are a key part to all software development lifecycles and elements within in them including testing. From previous experiences of using the term "process" in agile talks before, I have found it not to be well received.
I however agree there is something missing still in it without this or a clear section on practices and the application of them, so thanks I will indeed pick these up. I agree with the view that there are two main threads, with the use of performance measures on current agile practices including TDD and the use of agile practices within traditional performance testing projects, if I remember correctly we discussed this briefly in the pub after the June SIGIST.
Thanks I am likely to take you up on the offer of discussing it further and will return the favour when you start producing papers yourself – I look forward to seeing you speak at some point soon.

Neill McCarthy
"Agile Testers of the World UNIT !"
» login or register to post comments

What about an outline process?

Submitted by Antony Marcano on Tue, 05/07/2005 - 06:54.
[textile]Neil,

Your headings cover most aspects with the exception of "process". Many find this word scary or off-putting. Process, however, plays an important role in agile approaches but, as you know, the process needs to be agile for it to be an Agile Process. This agility only works if the team are disciplined - i.e. they modify the process based on improvements not due to personal objections, say, to writing tests or pairing with a colleague.

In your early concepts and thoughts, I suggest that you might explore existing Agile Processes and how they apply to performance testing. For example, much of the text on how to employ Test Driven Development is targetted at functional testing (in the strictest sense, whether unit, integration, system testing, acceptance tests). Very little if any of the text available on TDD covers how to perform non-functional testing like performance testing. I believe that this alone is enough for an entire presentation.

It appears that your talk will be an overview of the subject. So in that light, I think there are two perspectives here:

* Performance Testing in Agile Processes
* Agile Processes for Performance Testing

The latter could apply to performance testing, regardless of what methodology is (or isn't) operating around it.

Whether there would be enough time to explore both in your paper/presentation is another story. Best of luck!!

"Antony Marcano":http://antonymarcano.testingreflections.com
» login or register to post comments

Comment viewing options

Select your preferred way to display the comments and click 'Save settings' to activate your changes.