Mike Kelly's blog

New home, same blog

Sat, 10 Nov 2007 06:17:04 -0600

I recently migrated my testingReflections.com blog over to my own website. You can find it at www.MichaelDKelly.com/blog.

It was a long time coming, I’m just generally too lazy to do things like that when the existing solution works just fine. I hope to get back to writing and blogging in the near future. I’m currently cooking up some new performance testing content and I’d like to get back to some basics and start writing about how I learn about testing again. I also have an inbox full of unanswered questions that I’ll start posting out here.

IWST Roundtable: Planning, Estimating, and Executing Performance Tests and Performance Engineering

Mon, 22 Oct 2007 10:03:53 -0500

This week we hold the October Indianapolis Workshop on Software Testing roundtable. The topic is Lessons Learned in Planning, Estimating, and Executing Performance Tests and Performance Engineering.

Date: Tuesday, Oct 23rd
Time: 6:00 PM to 9:00 PM
Topic: Lessons Learned in Planning, Estimating, and Executing Performance Tests and Performance Engineering
Format: Roundtable
Venue: WDD Software Consultants and Mobius Test Labs

Mobius Test Labs
9229 Delegates Row, Suite 290
Indianapolis, IN 46240
MAP

Email me with any questions at Mike@MichaelDKelly.com

IWST 2007 Calendar Change

Sun, 21 Oct 2007 08:21:27 -0500

To make room for another LAWST-Style peer conference here in Indianapolis, IWST has changed it's calendar. The November and December sessions on "Scripting for Testers and Test Automation with Open Source Tools" have been consolidated to one workshop on December 1st.

On November 16th and 17th there will be a new peer workshop in Indy. More information on that workshop soon to come.

If you would like to attend the Dec 1st IWST workshop, just let me know: mike@michaeldkelly.com

IWST Roundtable: Techniques for Exploratory Testing

Tue, 14 Aug 2007 06:46:36 -0500

Last month we held the July session of the Indianapolis Workshop on Software Testing. The topic was Techniques for Exploratory Testing and we covered a lot of cool stuff.

This month we will hold the follow up roundtable. I want to finish some of the experience reports we didn't get to (Jason, please come!!!) and I'd like to review the list I captured to see if we can identify any dynamics or trends from the data. Come, share, eat the M&Ms that Mobius puts out on the table.

Alter Ego

Mon, 06 Aug 2007 08:53:23 -0500

Alter Ego written by Dave Christiansen has just been published. I was one of the reviewers for the book and I think it's an excellent read. It has (almost) nothing to do with testing, but it's a great read for geeks who like fiction with a technology twist. And it is twisted!

Read it. You won't regret it. (You can even preview the first six pages online...)

Techniques for Exploratory Testing

Sun, 21 Oct 2007 08:21:49 -0500

Earlier this month we held the July session of the Indianapolis Workshop on Software Testing. The attendees were:

Andrew Andrada
Charlie Audritsh
Howard Clark
Michael Goempel
Jason Horn
Karen Johnson
Michael Kelly
Steve Lannon
Kelvin Lim
John McConda
Scott McDevitt
John Montano
Vishal Pujary
David Warren
Gary Warren

The topic we focused on for the five-hour workshop was 'techniques for exploratory testing'.

Starting a peer workshop

Fri, 26 Oct 2007 06:59:13 -0500

Someone recently emailed me and asked me about starting their own peer workshop. I have a small amount of experience in the topic. I've run a number of IWSTs (a small local workshop) and two WOCs (a longer three-day workshop). I've attended many other peer workshops, including WHET, WOPR, WOCT, STMR, STiFS, and AWTA. They are all in some way or another in the LAWST-style.

Here are bits of my reply:

Heuristics for Test Question Generation

Fri, 13 Jul 2007 19:37:51 -0500

[textile]
Today at the Workshop on Open Certification we came up with the following (non-ordered) heuristics that might be useful in test question creation:

1) Plausible buzzwords
2) True but irrelevant
3) Write but for the wrong reason
4) Some fool said it
5) My boss will believe it
6) Two conclusions from the same reason
7) Incomplete reason

What is a boundary (part 2)?

Wed, 11 Jul 2007 12:20:00 -0500

In this post, I gave a working definition of a boundary. That definition was "a boundary is any criteria by which I factor my model of what I'm testing."

James Bach challenged me to come up with three specific examples and to tell him how they are boundaries. With that, I pulled out my moleskin and drew three different models: a UCML model, a system diagram, and the model I used to test the time-clock application.

What is a boundary?

Sun, 08 Jul 2007 16:51:52 -0500

[textile]
Today was the second day of the WHET #4 (Workshop on Heuristic and Exploratory Testing). At the workshop, the question of "What is a boundary?" came up. Here's my answer...

Automation Bias

Sun, 08 Jul 2007 12:57:26 -0500

[textile] Yesterday at WHET #4, James Bach mentioned automation bias. A bit of research turned up Automation Bias in Intelligent Time Critical Decision Support Systems by M.L. Cummings.

While humans are typically effective in naturalistic decision making scenarios in which they leverage experience to solve real world ill-structured problems under stress, they are prone to fallible heuristics and various decision biases that are heavily influenced by experience, framing of cues, and presentation of information. For example, confirmation bias takes place when people seek out information to confirm a prior belief and discount information that does not support this belief. Another decision bias, assimilation bias, occurs when a person who is presented with new information that contradicts a preexisting mental model, assimilates the new information to fit into that mental model. Of particular concern in the design of intelligent decision support systems is the human tendency toward automation bias, which occurs when a human decision maker disregards or does not search for contradictory information in light of a computer-generated solution which is accepted as correct. Operators are likely to turn over decision processes to automation as much as possible due to a cognitive conservation phenomenon, and teams of people, as well as individuals, are susceptible to automation bias. Human errors that result from automation bias can be further decomposed into errors of commission and omission. Automation bias errors of omission occur when humans fail to notice problems because the automation does not alert them, while errors of commission occur when humans erroneously follow automated directives or recommendations.

Correcting my poor communication

Sun, 08 Jul 2007 11:10:17 -0500

[textile]
Last year I read Behind Closed Doors: Secrets of Great Management by Johanna Rothman and Esther Derby. The book is well written and is told in an interesting way. In the back of the book, there is a section titled Techniques for Practicing Great Management. In that section, I found a list of questions designed to help managers make the progress of any project contributor more visible. You might try asking questions like the following:

Bacon and boundary testing

Sun, 08 Jul 2007 10:48:28 -0500

Today was the first day of the WHET #4 (Workshop on Heuristic and Exploratory Testing). The topic for the workshop is boundary testing, and it's currently attended by:

Henrik Andersson
Ross Collard
Tim Coulter
James Bach
Jon Bach
Scott Barber
David Gilbert
Dawn Haynes
Doug Hoffman
Paul Holland

Slow pizza

Fri, 06 Jul 2007 20:17:49 -0500

[textile] I went to order a pizza last night and the website I normally use to do so had a new interface. It appeared to me that they had switched from what I believe to be a third-party solution to a homegrown solution. Here's what immediately went through my mind when I saw the change:

"Great, the company is going to loose my personal data because they are new to this and they probably don't know anything about security."

Fish Failover testing

Fri, 06 Jul 2007 19:43:48 -0500

Thanks to Doug Hoffman for pointing this out to me. Ignore the logo, it's worth watching. It's the type of stress testing we've all wanted to do.