Verification & Validation in Performance Testing

Ainars,

It wasn't really a statement, it was rather a reflection. I was pointed to this issue and started to look around to confirm or reject the comment. While I completely agree that it isn't the most practical issue (at least in performance testing), it was documented and standardized long time ago. I see references to old IEEE standards/documents (unfortunately I don't have access to IEEE documents). Looking around, I was very surprised to see that people referring the same definitions interpret them completely differently. For example, as I read the definitions, "Verification is done through static testing, Validation - through dynamic testing" doesn't look correct for me. I didn't use this link because it was ideal - it looks like it is a kind of questions everybody included a couple words about, but I haven't found anything that was clear and comprehensive enough for me.

Not that it is a major issue for me or it makes a huge practical difference. The original thought was that we need to be consistent with a few terms that are well established. The next I was surprised how these "well established" terms are vaguely interpreted (sometimes "explanations" contradict each other).

Alex

Alexander,
I'm not sure how do you use the "investigate vs. validate notion" but as far as I use it and seen Scott using it - we as both talking about validation not verification.
According to link you provided Verification is done through static testing, Validation - through dynamic testing. Running tests using load tools is 100% dynamic testing. Static testing is when in the middle of project you talk with your architect to discuss performance attributes of architecture. This is done exactly in the investigation process.

P.S. I agree with Scott regarding difference between V&V (as he commented here). I could even tell you that this difference is the only item in testing and QA theory I never care to remember (and the only thing I learned by rote before certification exams that I passed some time ago)

...to coin a heuristic to make a distinction between performance tests that collect data without a pre-determined oracle and tests that collect data to compare against a pre-determined oracle. Both validation and verification include oracles. I chose validate simply because it rhymed with investigate. I never intended to try to draw a distinction between validate and verify.

In truth, I'd love to go back and say that somewhere in everything I've ever published including "Investigate vs. Validate". Unfortunately, that's not so easy to do when it's been published in print as much as it has.

So, at the end of the day, the difference between V&V is subtle, frequently misunderstood and not overly relevant *most* of the time, but that does not excuse my unintentional propagation of the mis-use of a term for the sake of a mnemonic device.

I assure you that this is an oversight that I'll resolve in my next major publication.

--
Scott Barber
Chief Technologist, PerfTestPlus
VP Operations & Executive Director, Association for Software Testing
sbarber@perftestplus.com