Performance Testing Mythology 2

i'm no testing expert, and maybe the semantics of 'performance testing' are tied so tight they exclude the discovery of deadlocks, bottlenecks etc, but I have *never*, in 20 years of professional s/w engineering, found an application that didn't warrant some performance testing. it's all very well speaking of super fast processors, but let us not forget that disks and networks are slow. even if it isn't the app that is slow, it maybe the NIC that has been set as half-duplex in an environment where that causes excessive speed renogitiations.

regarding expense; i would have thought that the types of apps that the author thinks do not warrant testing can be performance smoke tested with little cost, using some cheap of free tooling, creativity a smart technical tester. after all, you don't have to end up in a world of thread monitoring, db lock monitoring tools and statistical algorithms for spreading user loads if a crude perf test shows you that everything is all well within tolerances.

I guess I should pull out all that stuff from my "Medieval Fascination" days if I'm gonna slay dragons! :P

Just remember, there is a fine line between bravery and stupidity. It's only by listening very carefully to folks like you who aren't on a personal crusade to overthrow the "Greedy & Evil Emperor" that we keep from becoming over-bold and crossing the line.


--
Scott Barber
Chief Technologist
CEO & President
PerfTestPlus, Inc.
sbarber@perftestplus.com
http://www.perftestplus.com

I'm allowed to say that 'cause I wrote it. ;)

What still amazes me, however, is that that article is causing a stir like I invented sliced bread. We have *sooooo* much work to do to *undo* the mythology promoted by the rich tool vendors with radio commercials, marketing departments and venture capitalists.

/sigh/ At least it's a more productive passtime than golf. :)

--
Scott Barber
Chief Technologist
CEO & President
PerfTestPlus, Inc.
sbarber@perftestplus.com
http://www.perftestplus.com

Are they though - I know that many of the test myths, automation myths and performance myths are built on sand, as were the pyramids, yet they still stand.
Scott indeed has covered many of the items in the article before and for enlightened engineers or thinking testers there are probably few new lessons there; however there are lessons in there worth repeating to the experienced and for the unaware.
For the experienced it is good to see others providing the weight of information to assist with moving to the tipping point where the early performance testing and the performance analysis are the norm. For the experienced it is good to have this learning reinforced and reiterated so we do not move into unconcious incompetance.
Long may Scott, Roland, Julian, Antony and other brave knights in test find and slay these dragons in public.


Neill McCarthy
"Agile Testers of the World UNIT !"

Ainars,
you should share these stories where you can, learning from others examples is a very powerful technique. Also providing the example helps to re-evaluate the activities at the time by allowing some critical thinking to be applied with the luxury of not being totally focussed on the immediate resolution of the issue.

Neill McCarthy
"Agile Testers of the World UNIT !"

Just found the featured article of last Better Software magazine talks among other about how do we communicate to our managers an approach for early project performance investigation . Though I found nothing new to me in that column, it shows that Perf. Mythology has weak positions, doesn't it?

Given the quantity of generated and reused code and increasing number of abstraction layers the performance issues are no more only “ineffective code/algorithm written”. There are more issues added like “ineffective data manipulation caused by assumption that code will be automatically optimized in another layer”. It is like assumption that memory leaking issues will be gone with move to JAVA...

It inspires me to share my experience of doing performance testing late in the project and detecting project-critical performance issues addressed by change of technology/architecture or tuning the 3rd party software used instead of optimizing slow algorithm (last issue I remember extended development and as a result iteration time for several months, which in turn indirectly caused the project not to be continued).

... but we've reached a point of validation -- a point where folks are willing to accept that it's probably a good idea. *That* is significant. Once we cross that threshold, it's up to us to demonstrate value and cost effectiveness with the early adopters and shout the successes with those early adopters from the highest mountains. It'll take some time - I mean, really, it only took five and a half years for some Harvard educated analyst to say what I've been saying since 2000... and I'm sure I wasn't the first to say it!

--
Scott Barber
Chief Technologist
CEO & President
PerfTestPlus, Inc.
sbarber@perftestplus.com
http://www.perftestplus.com