Tester to look for the big picture in an Agile project

In this quite long blog I’ll tell you how I approach agile projects and why I think it is the right way. In short my idea is very simple – I do write down „the test ideas” at the very beginning of the project, while developers are doing the initial work and I have nothing to test yet. I analyze whatever information is available and ask for more to do that. When I’m done with the list I save it and forget about it. I test ignoring it. It’s sometimes late in the project when I dig it up to wonder how much of the ideas are covered. A lot of them does not make sense any more, others are somewhat covered, but I keep discovering that quite a lot of things I have missed – and I discover more bugs that I would otherwise missed. Why? In a hurry, testing one feature after another I’ve missed the big picture – that’s why.

How does agile differ ?

Guess who and on what have said those words:

I'll practically finish an area before I move on to another. It bugs me to walk away with areas unfinished. When you change your mind on things, and you will, you can always revisit an area later …

I will tell you where it comes from a bit later. Bet let me tell you how I found it first. I keep reading about manual tester’s role on agile projects, especially one following Scrum. Reading Mike Kelly on the very same topic, argue about the

value a skilled tester who is focused on the big picture can add to the project

. I wonder where does the term “the big picture” comes from as I’ve red it here and there even before the “era of agile”. Perhaps it comes from painting – a more mature area but also a creative one. So I found a particular painting methodology - technique talking about the big picture as a way to get an information about quality

You have to step back away from your painting frequently and look at it from a distance with your eyes squinted. This will visually break down the color values into monochromatic areas of light and dark, which will help you to establish an accurate value range.

So I start to think - if agile is about feedback which we want to have as soon as possible, then what agile teams really need is a person who is sitting back away from the painting with eyes squinted. Yes – a tester!

I’m afraid that in Scrum it is only the retrospective meeting when team tries to “walk away” and consider the values in the big picture. They are too busy finishing one area after another. It’s a bit too late, isn’t it?

Establishing an accurate value range for software

I’m not a painter, I’m not sure what exactly does this means, but I could guess. Once they have just another area painted they want to see if it changes the big picture. Adding an area may spoil down the value of previously painted areas. TDD is the best evidence that software developers tend to forget that features developed earlier may change their value as new functionality is added. It is not enough to simply run the same tests we considered to be enough when new functionality was not added. As a tester I do reevaluate and change a bit regression tests when I run them against subsequent version of software having functionality expanded.

So, the tester’s role, finally

So what do I do as a tester on agile projects? Yes, I do test each story – I test it manually in so called QA environment. Formally I do it to validate that the story is done. Actually I do rely on developer’s unit tests to confirm that. Well sometimes it appears that story is not correctly transitioned to QA environment – build and environment issues are discovered this way. However what I really do – I learn the feature, I do play with it to understand how user may value and use it. That’s something can’t be done up-front, or at least is much harder to be done before code is finished. Some people call it exploratory testing. I used to refer to it as Exploratory Learning followed by testing (applying the known testing techniques to) what was explored. It is especially essential to keep the big picture in mind as you do your exploratory learning, but step closer and forget the big picture as you do testing.

Adding a feature may influence established usage

Automated regression testing is a powerful tool to make sure that new features do not break old ones. But they miss that old feature usage patterns may change because of new features. Adding a comment field to a business item may change your business item naming approach, so that you may give the same name to two items. Well what I just wrote is an abstraction (I’m going “meta” some would say). Let me go even deeper before back to practice. I recently shared ideas coming from house building experience which ended up purchasing a house instead of building new one. Well we bought a garden with cherry, apple, pear and other trees. That’s a lot of fruits, much more than we could eat ourselves. So I have a new hobby – making home wine. 15 liters of cherry wine I made were quite good, so I’m making apple and pear ones now. Why do I tell this to you? My new hobby requires space to store the tools, to store the bottled and preparing wine. I’m really happy to have a huge cellar, which I never counted as a big value when buying the house. Adding trees to the big picture added the need to cellar.

Dragon is dead; long live the dragon

Yes, values changes. But that does not mean you should ignore the initial values. There was reasons for them and the reasons have not changed even if functional goal. I’ve used to work in an environment quite close to waterfall, where as a tester I spent a lot of time thinking and planning the tests to be done. I know that it takes time. Sometimes I considered more good tests only the next day or the next week. Besides I have quite a checklist in my head: web application – test different screen resolutions, font sizes, browsers, session timeout, back and refresh buttons, etc. for server the list is different. If there is something new to me (e.g. security) I may need to research that topic and come up with generic tests to perform (or tools like security vulnerability search tools). Once I done my research, analyzing and thinking I have a list of test ideas written down. I used to write them down in a generic test planning document. I typically have enough time in the beginning of the project or the new big release to do it, because that’s when developers does their big jobs, that agile people tend to call establishing infrastructure, while I’m still used to call it architecture/design and developing low-level, “core (API)” code. You don’t have any user story working to test until that. One I start getting user stories I test them on-by-one. I do exploratory testing around them and a lot of other good things, but I do not look into my test plan yet. I can’t tell you how do I decide when to look at it but it is quite late in a project. If there is some delivery date defined it should be close to delivery date but not too close to allow developers fix the bugs that I will find testing. And I know I will.

Why so late?

Wouldn’t it be better to provide this feedback faster? Perhaps I just need too much time “to walk away and get eyes squinted”. This may be a simple task for my brain to look at a painting, but quite time consuming – to look at the software. No wonder software industry has always tried to picture the software in forms of models (UML, ER, …) flow-charts, etc. I believe that with a practice my brains are used to analyze software much more efficiently than it did 10 years ago. I think as proficient painter may find more flaws in a painting within seconds compared to what novice will find in minutes. I’m afraid the same applies to software but seconds translates to hours or even days (depending on software size).