Exploratory Testing Excercise - how I solved it...

In a previous post I presented a challenge as an exploratory testing exercise

why is it an exploratory testing exercise?

At first, when my friend posed the question, I was just intrigued and quickly obtained the answer so I could answer her question. Then I realised that what I was doing was applying some of the skills I use when performing inspective exploratory testing... (arguably, we do the same with prospective testing too).

Appart from the obvious skills - like errm... using a computer... simplifying equations and the like - what made me stop and notice that what I was doing was akin to exploratory testing was when I realised that I was applying the scientific method. An approach many seasoned exploratory testers (such as Bach and Bach, Kaner, Kohl and I'm sure many others) have referred to in the past.

From What is the scientific method:

* 1. Observe some aspect of the universe.

* 2. Invent a tentative description, called a hypothesis, that is consistent with what you have observed.

* 3. Use the hypothesis to make predictions.

* 4. Test those predictions by experiments or further observations and modify the hypothesis in the light of your results.

* 5. Repeat steps 3 and 4 until there are no discrepancies between theory and experiment and/or observation.

When consistency is obtained the hypothesis becomes a theory and provides a coherent set of propositions which explain a class of phenomena. A theory is then a framework within which observations are explained and predictions are made.

Some testers work in environments where they are given little or no prior knowledge of what an application should do. Others may have detailed prior knowledge of what the application should do (e.g. you wrote the code or you were part of a team applying 'Acceptance Test Driven Development').

When you have detailed prior knowledge, it is important that you are aware of the potential for experiencing a confirmation bias... So you don't see only what you want to see, it helps to empty your mind or adopt a different persona in order to approach the problem with a clear head... but I digress.

In either case, when you are performing inspective exploratory testing one of the things you need to do (at one time or another) is answer the question 'what is it that the application is actually (or should be) doing ?'. The scientific method is a useful framework for answering that very question...

The Scientific Method Applied to this problem

The question my friend posed was... 'how does it know?' So I set about trying to find the answer...

Observe -> Hypothesise -> Test

The first thing I did was play the game to make some observations... I'll admit that I was momentarily amazed that it got it right... I played it again... and indeed it was right! Mystical wizardry? No!

I hypothesised that it would show me the same symbol for the same number... I tested this by playing the game again with exactly the same number... but this time it showed me a different symbol... Hmmmm... but it was the right symbol... hmmmm....

New hypothesis... the symbols were changing each time. I tested this hypothesis... it was correct.

A more detailed hypothesis...

Clearly, however, there is just a simple mathematical rule at play here. This isn't enough on its own as a hypothesis so I needed something better than 'a mathematical rule' to work with... Now, perhaps if I was paying attention more during Mathematics classes I would have known this rule and might have been able to skip this step...

The game asks you to think of two digit number... e.g. 23. Then it tells you to take each digit and add them as separate numbers... 2+3=5. Then it tells you to subtract that from the original number... 23-5=18.

For expediency, I simply put this into a spreadsheet and dragged the formula down from numbers 10 through to 39... instantly I spotted that the result of the formula was always a multiple of 9...

On reflection: what I should have done was take a screen shot at this point so that I could look to see if my symbol appeared anywhere else. This would have also revealed the 'multiples of 9' pattern.

Knowing me, I'd still want to know why so imagine I'd have still plugged the numbers into excel to solidify the hypothesis.

I checked this again by walking through the application, predicting that the symbols were the same for all multiples of 9 and that this symbol would not appear in any other positions...

Indeed I was right...

I predicted that if I ran through the game again, it would show a different symbol in each multiple of 9 (and in no other positions)...

Indeed I was right again...

Obviously this is so that you don't always see the same symbol every time you play... that would just give the game away :-)

Theorise

This wasn't enough for me... I want to know why!!!! Again, if I'd paid more attention in maths classes, I might have already known this... luckily I did pay enough attention to know how to do basic algebra...

So, if the first digit (the tens) is x and the second (the ones) is y... then the number is basically...

10x + y

The game tells you to add x + y and subtract that from the original number...

(10x + y) - (x + y)

So... to simplify the equation... hmmmm... algebra is a bit rusty... but let's see...

10x - x + y - y

Yep... well, y - y is 0 so, it's the same as...

10x - x

Which is the same as...

9x

i.e. always a multiple of 9.

Why does it matter? Well, for me, this gives me confidence in the conclusion I'd previously reached.

Now... what about those other symbols in all the other positions? hmmm...

Time to let it go methinks... perhaps I'll leave that one for you to solve.