Ainars Galvans's blog

Grey-box techniques for excluding tests

Mon, 30 Jun 2008 07:33:18 -0500

Don’t test the reused code tested previously – you will have more time to test new code. This heuristic for a black-box tester is basically all the idea of this blog – one I was creating for a week … I started this as blog about techniques, but wanted to be clear about grey box. A research on the term ended quite a long story below. If you don’t care about it – scroll down to the last chapter describing the techniques.

How to teach grey-box testing?

Thu, 19 Jun 2008 01:34:59 -0500

When I’m teaching new testers I use (specific chapters from) Black Box Software Testing course by Cem Kaner and James Bach. Those are great materials. However there are some topics not covered to an extent that I would like to (probably because of the specific of software we are creating in our company). For instance multi-edit heuristic and environment-specific handling tests. Maybe those topics are more in grey-box than in black-box, but for sure they are not the only topics of grey-box...

Baltic/Nordic region events

Sat, 10 May 2008 05:49:39 -0500

If you live in a small country like me, you may or may not need to travel too far to get to testing conference. There is an annual Latvia test conference and I know numerous conferences in Sweden. Although I don’t like to go to conferences too much, I try to learn about them and their content as much as possible. Here I share what I know about my region conferences and hope to learn more from you.

Process VS People oriented test management

Fri, 09 May 2008 08:16:50 -0500

Once upon a time I cared about tester certification. I’m tired to be. There is but one lesson I learned - just as scripted and exploratory testing are two extremes of all the possible testing approach, there process-oriented and people(skill)-oriented test management styles. And it does matter which one your manager is… I don’t like to work with process-oriented managers – this is something to remember when new opportunities will appear either within my own organization or outside.

I don’t use math in performance testing, do I ?

Thu, 08 May 2008 08:26:55 -0500

I’ve seen testers recommending The Art of War or Weinberg books which are not about testing at all. I’ve also seen performance testers recommending knowledge of probability theory, statistics and modeling principles. I don’t apply this knowledge in performance testing myself – well at least not directly. I never think about things like distribution function, mean deviation, etc. Do you? Don’t I ?!

Stability test tool: batch file

Mon, 28 Apr 2008 02:31:17 -0500

This morning I realized that last Friday I did something that seems very simple to me, but may not be for young people not used to DOC (or at least windows95). I run a bat file test.bat containing following lines:

:loop
copy test\*.* import\*.*
sleep 600
goto loop

The same documents get copied into import directory each 10 minutes (600 seconds=10minutes). I was testing server who is able to pick up files incoming into this folder. It is quite typical to have such services to automatically process faxed documents and email attachments. But you could replace copy command with your process execution call or anything.

Reliability is not only about overload: RANDOMize your system state

Thu, 24 Apr 2008 11:15:18 -0500

I never used mnemonics. Never say never... Here is my own mnemonic. Double RANDOM stands for ReadRights AalternativeAccess, NoNetwork, DesynchronyzedData, OutOf(whatever resource), MinMax(installation). Depending on context (e.g. reliability requirements, etc.) I spend more or less time trying to emulate system states unexpected by developer and thus unhandled by code.

The only defect measure I would publish

Mon, 14 Apr 2008 08:58:43 -0500

Recently I’ve found my (sever years old) presentations with a lot of defect metrics and analysis. Only one slide I am proud of still – the percentage of “cancelled” bugs (drilled-down to percentage of reject reason: duplicated, not repeatable, etc.). What I like about this measure that it is direct – I’m really interested to keep the number of rejected bugs low – yes I’m interested in testers not reporting them in the first place.

Performance: art and/or technique(s)?

Wed, 12 Mar 2008 10:32:24 -0500

What follows is nothing new. I only try to take another look to what have been written so far. Scott Barber recently commented The whole situation is sad. Tool vendors dominate the training market (at least in performance testing) . Perhaps I don’t care for training as I train my people myself, but I care for stereotype created by dominating vendors. I will bring my own understanding of by-effect caused by load tools bringing the nice load tools. No I don’t say that performance testing is no more an art. But there are certain limits we are yet to break.

Performance testing and coverage

Mon, 10 Mar 2008 08:46:48 -0500

I’m doing something like performance testing as consultant now. First thing I say to my customer is this: I will not replicate real application usage! I will only emulate simultaneous clicking in a few places of your application: tiny functional coverage; incomplete environment and data; no emulation of unexpected events… It reminds me of a joke about physicists analyzing a spherical horse in a vacuum.

Is testing as different as economic systems?

Tue, 26 Feb 2008 05:54:52 -0600

I have lived in two out of 5 basic types of economic systems: planned (Soviet Union) and market (Latvia now) economy. I could draw certain similarity with notion of testing schools. … well at least to a degree of my interpretation of the notion. Based on my (perhaps limited) experience – I prefer to think about the reasons for implications described – in terms of decision making…

Do you keep notes about “the first below the line” tests?

Tue, 12 Feb 2008 09:25:26 -0600

We can’t test everything. So it would be logical to divide tests into must and may (if time permit)” test. To have some tests reserved... Nevertheless, in continuous integration projects I have “it’s now or never” rule as testing new features. I may postpone whole feature testing, but specific test ideas – never. I’m not sure if I’m right – that’s why I’m writing this.

Test Case Review

Thu, 31 Jan 2008 11:03:15 -0600

For years I was wondering why we disagree with people about exploratory testing. Recently (working close with them) I’ve realized part of their thinking: To review test case you have to document it. You have to document it in advance to test execution; otherwise you will run unapproved test cases. You can’t review test cases in exploratory testing because there are no test cases written in advance to test execution.

Interview hint: be careful using term Exploratory Testing

Tue, 22 Jan 2008 05:42:48 -0600

Term Exploratory Testing (ET) is gaining popularity at least in my region. I welcome and promote the popularity. As a test manager I use the term to indicate a bigger story for my team, to hypnotize upper management and for other good reasons.
But there is a something where I avoid this term. I have quite an experience with tester hiring and performance reviews. I don’t ask anyone about ET – not any more. Well… unless interviewee uses this term first. In most cases that’s a terrible mistake.

Usable = “used to”? How about progress?

Fri, 11 Jan 2008 09:24:41 -0600

...I’m used to manual transmission, however I agree that automatic transmission is somewhat more useful, but I’m simply used to manual one.
"Used to" is the most accepted oracle for usability testing. If the application seems a lot like any Microsoft application, then it’s usable (despite perhaps they hate Microsoft …). Now, in 2007 Microsoft did a bad thing – changed what we get used to during last 15 years.