Skip navigation.
Home » blogs » Karen N. Johnson's blog

Bugs found away from the keyboard

Submitted by Karen N. Johnson on Sun, 11/02/2007 - 03:56.
I was walking along the beach when the idea of a bug came to me. I hadn’t been consciously thinking of the application I’d been testing but I stopped walking and thought through the scenario.

The next day, I walked into the office, pulled up to a test PC, executed the series of steps I’d thought of and watched the system crash.

It was a thrill. I felt like I had outwitted the program. I felt smart.

I told my boss about my find. He had asked me just a couple months before if my testing had evolved to the state where I was able to find bugs when I was away from the keyboard. This was back in 1995 and at time, I didn’t understand his question.

A few months later, his question made sense to me. Did I have to be at the keyboard or even in the office working on a test plan to find a bug? Not anymore.

There have been many different applications and for that matter, many different bosses since then. I continue to find bugs when I’m located in a variety of places but the thrill of finding a good bug is still a thrill. The bugs found away from the keyboard. They are the sweetest ones.

Well, this was true story. But to stay true to "what's in my blog?" what techniques can I share to help other testers develop skills to find bugs away from the keyboard? Here are three:

First, step away from the keyboard. Office life can be stifling and creativity is hard to develop if you work in an office where crazy beliefs such as you’re only working if you’re seated prevail.

Second, talk to people. People like database administrators and customer support can give you all sorts of ideas about what can go wrong. Problems from the past can become embedded in your head and help you find issues with the new release, new product, etc. Talk productively and it’s not disruption, it’s smart.

Third, be creative. How? Take a complex aspect of the application you’re testing and puzzle over it in your mind. Let your mind wander. Don’t focus on finding test cases; focus on understanding the system and different test conditions will likely come into focus.

When your testing evolves to being able to find bugs away from the keyboard, the thrill will have you addicted to testing. For me, the thrill continues.
» Karen N. Johnson's blog | login or register to post comments

I agree, just wanted to continue the thread

Submitted by Ainars Galvans on Fri, 16/02/2007 - 11:58.
First of all thanks Antony, you are right, it really does not matter when we say the bug is found: during design or execution. What matters to me - how to make stepping away from keyboard a legal testing practice? Because you know, my company does not pay time I drive back home even if I find bugs during that time. Why should I care then? Managers do not recognize power of this practice and never value skills for doing that – so why should I teach it to my team? Maybe this could only be based on enthusiasm as Antony told? Motivation - hope to earn credit though it. Which is good for salary negotiations. So you got paid after all, but it takes time – your time.
I started about session based exploratory testing because I feel like this could be a way to formally promote the above practice. Never tried it myself however.
» login or register to post comments

Stichomancy - another dark-art approach from Alan Richardson...

Submitted by Antony Marcano on Thu, 15/02/2007 - 19:23.
[textile]This reminds me of "LEWT1":http://www.testingreflections.com/node/view/2422 where "Alan Richardson":http://www.compendiumdev.co.uk/page.php?title=aboutus gave us a lightning talk on a technique he uses to get unstuck, at times when test-ideas dry up.

Alan has turned to dark-arts and the paranormal as inspiration in testing on more than one occasion... ;-)

The technique he described is called "Stichomancy":http://www.onelook.com/?w=Stichomancy. The basic idea being that you pick up a book (I prefer an unrelated book), flick to a random page, select a random sentence on the page, read it and try (if only abstractly) to relate what you've read, to the problem you are trying to solve (or software you are trying to test).

I've tried it and it works! Takes a bit of practice but when the pressure is on and you get stuck for ideas, it can be a fast way to get unstuck.

Antony Marcano
» login or register to post comments

Some agreement where there appears to be disagreement

Submitted by Antony Marcano on Thu, 15/02/2007 - 19:08.
[textile]I think that part of the problem is that English isn't Ainars' first language (one of three I beleive) and sometimes the points aren't made as clearly as if he were writing in Latvian (that's if we could read Latvian)... It also sometimes makes the tone of the remarks hard to calibrate...

There is one point that you both seem to agree on... (although implicitly) that you are:

1) First identifying a test
2) Second, executing the test that allows you to observe some behaviour

Beyond that, it becomes a philosophical question - when did you *find* the bug... when you thought of the test or when you observed the behaviour? Personally, I don't think a question like that matters in this context...

Equally, whether it is a form of subconscious exploratory testing also probably doesn't matter. It would be hard to say unless we could look inside our subconscious and know what was happening in our brains... At best, all we can hope for in this instance is post-rationalisation of how the test idea was inspired...

What is important is that Karen has provided some valuable and practical advice. Advice on how to use stepping away from the keyboard as a means to identifying test-ideas that we might otherwise not have considered.

Thanks again Karen for another valuable contribution to testingReflections.

Antony Marcano
» login or register to post comments

Let me try to explain

Submitted by Karen N. Johnson on Thu, 15/02/2007 - 13:29.
Ainars,

Testing ideas that come to mind when I'm not consciencely thinking about an application are interesting to me. These ideas are interesting when they're vivid and especially when they directly find a good bug. I don't always have this experience but I have had this several times. Is there a part of the brain that keeps working on challenges we're particularly interested in? I don't know but it feels that way. I know I'm not alone. I've worked with developers that have woken up with solutions clearly worked out to their current coding challenge. I'm not trying to be mysterious or elusive, most times I think - like Phil mentioned - in the car, on the train and I work towards dicovering creative ideas to provoke a system and expose a bug. But at times, some bugs just come to mind. I hope this clarifies.
» login or register to post comments

A good leader...

Submitted by Antony Marcano on Thu, 15/02/2007 - 10:41.
[textile]A good leader can motivate their people and raise interest and motivation.

I've worked with teams of testers who had just fallen into testing and hadn't taken the time to learn more about it. They were bored and demotivated and saw no way out.

I've joined, with my enthusiasm and interest and, although at first perceived as a sad geek who loves his job far too much, the team later joins me in my enthusiasm and revel in the fun and intellectual challenges that we experience together. I've done this as the official leader of the team and through guerilla leadership. I've done this with testers and developers...

When a tester tells you "You've renewed my interest in software testing and rejuvenated my commitment to it as my chosen career" (as stated by a tester I mentored last year) I know that I've succeeded.

No, it can't be taught, but with the right leadership it can be cultivated.

There are always some people you can't reach... but that is the way of the world.

Antony Marcano
» login or register to post comments

more clarifications, if you don't mind

Submitted by Ainars Galvans on Wed, 14/02/2007 - 08:15.
Karen,
Clarification was more a question less a statement. And let me ask you further. Quoting yourself: “I stopped walking and thought through the scenario. The next day, I walked into the office, pulled up to a test PC, executed the series of steps I’d thought of and watched the system crash “
Perhaps I don’t understand something but when I read this I see that you considered something new about the scenario if not whole new scenario. If I’m wrong, then how could it appear that you failed to execute the same scenario – the same series of steps the previous day? I never said that you or me driving back home are directly thinking about more tests to do. I believe we are simply letting our brain to analyze the information – we employ of “subconscience mind” power if you want. It does great things for us. Let me tell you what – I’ve been writing poems in my childhood, not bad, you know. What’s strange about writing them – all of them was written by my “subconscience mind”. I simply took pen in my hand and wrote any word that comes to my mind. Would I be more religious I would think of god telling me what to write.
I never said you are doing session based testing. I just say that session based testing is something aimed to legalize (doing it in the office) employment of “subconscience mind” power. Because you know – there are testers who employ their mind to think about their family (how to protect child from narcotics, etc) while walking along the beach and I don’t want to fire them for that.
» login or register to post comments

This is about subconscience vs conscience thinking

Submitted by Karen N. Johnson on Tue, 13/02/2007 - 21:42.
Ainars,

It maybe a case of taking a break but it's different situation when you're consciencely still churning to think of more conditions versus the situation I've described about when a bug crawls from your subconscience mind over to your conscience mind. And you've learned enough to pause your life and grab a hold of it.

I don't agree with your clarification. I do find bugs. I don't create them. I don't invent them nor plant them in the code. I find them and I report them. I do sometimes (of course) think of more tests and execute those tests. But those aren't the same bugs as the bugs that have come to me when I wasn't consciencely thinking about the application I've been testing.

I believe you're describing creative thinking and tenacity which are different skills in thinking through tests that expose additional system defects.

See if you can find a time when you're not thinking about your application and the perfect path to a nice find comes to you. This is the experience I'm writing about.

As for defining this as exploratory testing or any other label - this isn't an exploratory method and it's not based on the session-based experiences. This type of constant back-burner subconscience thinking can't easily be instilled or taught. I don't even know how to turn this type of thinking on and off myself (although I'd like to because I've been awake since 4:30am puzzling over a challenge.)
» login or register to post comments

This is Exploratory Testing, isn't it?

Submitted by Ainars Galvans on Tue, 13/02/2007 - 10:50.
Karen, let me first clarify something. You don't find the bugs, you consider more tests to run that have a great chance to find defect once you will be at your keyboard. I know this because I'm doing this a lot myself while driving home/to office, while eating my breakfast, etc. You know what – there are few more things I used to do away from keyboard: thinking how to repeat not-repeatable problems , if an issue I observed is a bug i.e. triaging, and more.
You sometimes need to take a break and let your brains to analyze the information you received during test execution to consider your next steps. This is opposite to classic test design techniques not supposing usage of such information (only static information channels such as design documents, code reading, etc.).
As far as I understand this is what James Bach and Cam Caner promotes as exploratory testing and this is why they are talking about session-based testing, as you have (officially) a time even in your office to analyze the information – in between the test sessions.
» login or register to post comments

you can't teach people to care

Submitted by Karen N. Johnson on Tue, 13/02/2007 - 03:57.
Hi Phil:

I think we can share and teach techniques. But I don't think there's a way to make people care or to be engaged in what they're working on if they don't have sincere interest and internal motivation. You can raise the bar but you can't create a bar within someone.
» login or register to post comments

roll on Monday morning

Submitted by Phil Kirkham on Sun, 11/02/2007 - 18:13.
yeah, I've found those sort of bugs, had a hunch over the weekend and couldn't wait to get into work Monday morning to try it out.

Is it a technique that can be taught though ? Or is it one of those things that marks out a great tester from someone who just does testing ? The testers I know switch off at 5:30 but I know I'll have my latest program background processing at the back of my mind
» login or register to post comments

Comment viewing options

Select your preferred way to display the comments and click 'Save settings' to activate your changes.