Mike Kelly's blog

New home, same blog

Sat, 10 Nov 2007 12:17:04 +0000

I recently migrated my testingReflections.com blog over to my own website. You can find it at www.MichaelDKelly.com/blog.

It was a long time coming, I’m just generally too lazy to do things like that when the existing solution works just fine. I hope to get back to writing and blogging in the near future. I’m currently cooking up some new performance testing content and I’d like to get back to some basics and start writing about how I learn about testing again. I also have an inbox full of unanswered questions that I’ll start posting out here.

IWST Roundtable: Techniques for Exploratory Testing

Tue, 14 Aug 2007 12:46:36 +0100

Last month we held the July session of the Indianapolis Workshop on Software Testing. The topic was Techniques for Exploratory Testing and we covered a lot of cool stuff.

This month we will hold the follow up roundtable. I want to finish some of the experience reports we didn't get to (Jason, please come!!!) and I'd like to review the list I captured to see if we can identify any dynamics or trends from the data. Come, share, eat the M&Ms that Mobius puts out on the table.

Alter Ego

Mon, 06 Aug 2007 14:53:23 +0100

Alter Ego written by Dave Christiansen has just been published. I was one of the reviewers for the book and I think it's an excellent read. It has (almost) nothing to do with testing, but it's a great read for geeks who like fiction with a technology twist. And it is twisted!

Read it. You won't regret it. (You can even preview the first six pages online...)

Techniques for Exploratory Testing

Sun, 21 Oct 2007 14:21:49 +0100

Earlier this month we held the July session of the Indianapolis Workshop on Software Testing. The attendees were:

Andrew Andrada
Charlie Audritsh
Howard Clark
Michael Goempel
Jason Horn
Karen Johnson
Michael Kelly
Steve Lannon
Kelvin Lim
John McConda
Scott McDevitt
John Montano
Vishal Pujary
David Warren
Gary Warren

The topic we focused on for the five-hour workshop was 'techniques for exploratory testing'.

Starting a peer workshop

Fri, 26 Oct 2007 12:59:13 +0100

Someone recently emailed me and asked me about starting their own peer workshop. I have a small amount of experience in the topic. I've run a number of IWSTs (a small local workshop) and two WOCs (a longer three-day workshop). I've attended many other peer workshops, including WHET, WOPR, WOCT, STMR, STiFS, and AWTA. They are all in some way or another in the LAWST-style.

Here are bits of my reply:

Heuristics for Test Question Generation

Sat, 14 Jul 2007 01:37:51 +0100

[textile]
Today at the Workshop on Open Certification we came up with the following (non-ordered) heuristics that might be useful in test question creation:

1) Plausible buzzwords
2) True but irrelevant
3) Write but for the wrong reason
4) Some fool said it
5) My boss will believe it
6) Two conclusions from the same reason
7) Incomplete reason

What is a boundary (part 2)?

Wed, 11 Jul 2007 18:20:00 +0100

In this post, I gave a working definition of a boundary. That definition was "a boundary is any criteria by which I factor my model of what I'm testing."

James Bach challenged me to come up with three specific examples and to tell him how they are boundaries. With that, I pulled out my moleskin and drew three different models: a UCML model, a system diagram, and the model I used to test the time-clock application.

What is a boundary?

Sun, 08 Jul 2007 22:51:52 +0100

[textile]
Today was the second day of the WHET #4 (Workshop on Heuristic and Exploratory Testing). At the workshop, the question of "What is a boundary?" came up. Here's my answer...

Automation Bias

Sun, 08 Jul 2007 18:57:26 +0100

[textile] Yesterday at WHET #4, James Bach mentioned automation bias. A bit of research turned up Automation Bias in Intelligent Time Critical Decision Support Systems by M.L. Cummings.

While humans are typically effective in naturalistic decision making scenarios in which they leverage experience to solve real world ill-structured problems under stress, they are prone to fallible heuristics and various decision biases that are heavily influenced by experience, framing of cues, and presentation of information. For example, confirmation bias takes place when people seek out information to confirm a prior belief and discount information that does not support this belief. Another decision bias, assimilation bias, occurs when a person who is presented with new information that contradicts a preexisting mental model, assimilates the new information to fit into that mental model. Of particular concern in the design of intelligent decision support systems is the human tendency toward automation bias, which occurs when a human decision maker disregards or does not search for contradictory information in light of a computer-generated solution which is accepted as correct. Operators are likely to turn over decision processes to automation as much as possible due to a cognitive conservation phenomenon, and teams of people, as well as individuals, are susceptible to automation bias. Human errors that result from automation bias can be further decomposed into errors of commission and omission. Automation bias errors of omission occur when humans fail to notice problems because the automation does not alert them, while errors of commission occur when humans erroneously follow automated directives or recommendations.

Correcting my poor communication

Sun, 08 Jul 2007 17:10:17 +0100

[textile]
Last year I read Behind Closed Doors: Secrets of Great Management by Johanna Rothman and Esther Derby. The book is well written and is told in an interesting way. In the back of the book, there is a section titled Techniques for Practicing Great Management. In that section, I found a list of questions designed to help managers make the progress of any project contributor more visible. You might try asking questions like the following:

Bacon and boundary testing

Sun, 08 Jul 2007 16:48:28 +0100

Today was the first day of the WHET #4 (Workshop on Heuristic and Exploratory Testing). The topic for the workshop is boundary testing, and it's currently attended by:

Henrik Andersson
Ross Collard
Tim Coulter
James Bach
Jon Bach
Scott Barber
David Gilbert
Dawn Haynes
Doug Hoffman
Paul Holland

Slow pizza

Sat, 07 Jul 2007 02:17:49 +0100

[textile] I went to order a pizza last night and the website I normally use to do so had a new interface. It appeared to me that they had switched from what I believe to be a third-party solution to a homegrown solution. Here's what immediately went through my mind when I saw the change:

"Great, the company is going to loose my personal data because they are new to this and they probably don't know anything about security."

Fish Failover testing

Sat, 07 Jul 2007 01:43:48 +0100

Thanks to Doug Hoffman for pointing this out to me. Ignore the logo, it's worth watching. It's the type of stress testing we've all wanted to do.

July IWST: Techniques for Exploratory Testing

Sat, 30 Jun 2007 22:48:15 +0100

We are planning the July session of the Indianapolis Workshops on Software Testing.

Date: Saturday, Jul 21st
Time: 9:00 AM to 2:00 PM
Topic: Techniques for Exploratory Testing
Format: Workshop
Venue: DeVry University

We still have seats open. We are looking for people who are interested in becoming better exploratory testers and are willing to share real experience and techniques.

Developer to tester ratios, developers becoming testers, and quality criteria as a framework for selling testing value

Sat, 02 Jun 2007 23:31:46 +0100

[textile]
Earlier this week I was asked by a developer consultant who is a friend of mine for some links to research on developer to tester ratios. Being a helpful guy, I provided him with a link to a fairly good presentation on the topic put together by Dana Spears and Susan Morgan. In a follow up email, the consultant asked for something a bit more formal. He was trying to work with an organization to get them started with a testing practice and was looking for how to justify brining testers on board. They had a large population of developers with no testers.