I have always been curious about all sorts of things. How Stuff Works is a treasure trove for me - I didn't get much done the week I discovered it! I don't try to be an expert at everything I do, but there are a lot of subjects I want to learn (more) about. For example:

• Making automated testing so reliable and easy and fast that the cost of automating tests approaches zero.
• Taking manual testing from randomly banging on the app to finding lots of useful information fast. (Bunches of people are doing interesting work in this area; I want to learn about and understand and practice and do what they're doing.)
• Designing and writing elegant code.
• Desiging effective user interfaces.
• Designing fun user interfaces.
• Drawing.
• Designing functional houses that are also beautiful and intriguing and fun to live in.
• Designing logic games.
• Designing sidescrolling games.
• Designing applications and games that take advantage of the unique features of Tablet PCs.
• Designing workflows and applications for what computers will offer five, ten, and more years from now. (What if we had a full range of Tablet PCs, from tiny ultraportables up to monster 30"x40" slates? What if a tablet had both a pen digitizer and a touch screen? What if wireless internet access was ubiquitous? What if the pieces parts of computers were componentized so that you could have the screen out in front of you but secrete the fragile, heavy parts (e.g., hard drive) in your bag? What if you had multi-gigs of flash RAM? What if you had liquid lenses, so that your entire computer screen could also be a camera lens?)
• Making the education process useful and interesting and fun. (I'm not using a single thing I "learned" in college; how about you?)
• Making fitness and exercise something that people do because they want to do rather than something they think they should do.
• Investigating how the many different personality typing schemes integrate and overlap.
• Applying everything we know about how people learn to running a business where people enjoy working.
• Making listening to your body, eating the way your body wants you to eat, simple and intuitive and easy to do.
• General systems modeling.
• Visualizing data.
• Tap dancing. (Oh the rhythm fun!)
• Driving remote control cars.
• Flying remote control helicopters.
• Intentions, auras, channeling, psychics, past lives, spirit guides.
• Jazz piano. (I studied classical piano from kindergarten through senior high, but I haven't so much as touched a piano since.)
• Jazz drums.
• Jazz string bass.
• Classical guitar.
• Trikkeing.
• Roller skating (quads - inlines aren't my thing).

I used to start working on every idea or question I had the moment I had it. Needless to say this resulted in my doing a whole lot of not much on whole bunches of things! Now I try to focus on just a few interests at a time. My active projects this summer are Trikkeing, learning to draw, and finding and solving the pain points in our automation stack. Semi-active projects include skating, learning about general systems thinking, and pondering how to integrate Rapid Software Testing into the highly-automation-focused milieu that is Microsoft.

I would like to start a small research lab focused on these sorts of things. I am very curious to learn what happens if you take a couple great devs, a few great testers, a do-anything-with-hardware guru, a usability expert, and a talented designer (yes, I'm very open to one person filling multiple of these roles) - and me <g/> - and put them in an environment where they are compensated very well, have the freedom to try all sorts of zany ideas, and work closely with whomever the customers are. I think some very interesting outcomes would result.

If you (or someone you know) run or want to fund such a lab, or if you have a job where I would work with smart people to investigate and find answers to any or all of these topics, I am very interested! <g/> (Bonus points awarded if the job is walking or Trikkeing distance from my home, if the job involves lots of toys (i.e., books, training, tablety hardware), if it sends my wife and me overseas for weeks at a time, if y'all take the time to do things right the first time.)

*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great coding skills required.

They say that when you start thinking and dreaming in that foreign language you are trying to learn you are finally starting to internalize all that book learning with which you've been stuffing your brain. I wonder if that carries over to other skills...

This weekend I had a dream about a bug in a website. The details are fuzzy now, but evidently the website was supposed to do some sort of semantic analysis on my input and instead took it literally. Which of course is what computers always do!

I think I'm going to add another hallmark to my Hallmarks Of A Great Tester: Great Testers Dream Bugs. <g/>

My birthday present to myself this year was a Trikke:

Trikkes (that's a long "I" and just one syllable - "trike", not "trik-eee") are three-wheeled cambering vehicles, which is fancy speak for "lean side-to-side and turn the handlebars and they go fast". They are way fun to ride, but prepared to ache all over when you're done as pretty much your whole body gets into the act. Trikkes excel on the flat, slaloming down hills is a blast, and going up hills is a whole lot of work but very possible. (I'm still working on the going up hills part...) And you can freestyle on them just fine.

If you're in the Seattle area I'd be happy to give you a demo riding lesson. If you decide a Trikke is for you, please purchase through my affiliate site and help me feed my Tablet addiction! <g/>

One thing I've learned being a lead is that change takes time. And the bigger the change the longer it's going to take. Hence it follows that if your goal is to accomplish a big change in record time you had best break it into many many itty-bitty changes that you can punch out one after the other.

I am a big believer in Agile. It simply makes sense to me, and in my experience it works. But I've calmed down a bit about it. When I joined my current team I was rather an agilista. One of the first things I did was write a paper listing a raft of things that I thought needed to be changed: move to mini-milestones. Put more people working on fewer features at a time. Pair everything. I had five major points and a plethora of minor points.

Full credit to my management: they listened to my arguments and considered everything I had to say. But then came the counterarguments: The teams with whom we were working and/or dependent on were all doing multi-month milestones, and cross-synchronization would be problematic if we were working on a shorter schedule. The features weren't big enough to absorb more people working on them, and coordination of work would be too complicated, and we really needed to be moving forward on each of the features simultaneously. And on and on and on.

In case you hadn't guessed, none of my suggestions were enacted.

I can be pretty dense sometimes, but I got the message. <g/>

Which is not to say that I gave up. Far from it. But I did change my approach from a war cry to a whisper. I continued to argue that these changes would be beneficial, but I did so by talking with people one-on-one rather than talking at them in a great big meeting. I suggested books and articles for them to read. I highlighted other teams that were making similar changes. And lo and behold some quite interesting and useful and learningful discussions came about.

And guess where we are now? We've switched to one-month milestones. We've beefed up our feature teams into feature crews - i.e., more people working on fewer features. Portions of our automation stack have been developed using pair programming and test-driven design. And in what I consider my biggest victory, at a recent milestone retrospective scads of people suggested we do many of the things I have been promoting these past years but didn't credit me at all - they thought it was their own idea!

I certainly don't mean to say this all happened just because of me. Other agile proponents have joined the team. Agile is gaining steam at Microsoft and throughout the industry. Many of our partner teams have moved to mini-milestones. I do believe, however, that I am at least partially responsible. Credit might be useful come review time, but mostly I'm just glad that changes are happning, that they are proving useful, and that our already great team is becoming even better. And if they think it's all their idea, well, that simply means I'll have an easier time persuading them! <g/>

Change is good. Considered change is better. Change undertaken to solve real problems and with realistic expectations is Da Bomb.

*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great coding skills required.

A reader writes:

I am working on a client application that manipulates all sorts of COM objects. My job is to test the client application and to see if we are providing good enough support. So far I am writing test cases from the client application and testing if the COM objects are getting created or not. Since there are so many COM components out there, we are running into problem of how to test for each of them. Basically what I want to do is to test the client application for the interface it provides to the outer world.

I know COM works through two interfaces only IUnknown and IDispatch and then uses the QueryInterface to get the pointer to the other functionality supported by the interface. But the problem is how should I test this with so many controls. We also import/export data using this client application and this is another problem area. Do you have any suggestion. How guys at MS tested COM interfaces to/from say VB.

Ah, the joys of testing unbounded scenarios! Let me take each question in turn:

1) How can I verify my application correctly interacts with the COM objects it hosts?

You may or may not have an explicit spec for it, but your application has a contract with its COM objects as to what it will do when. You need to verify your application honors this contract. One solution is to write a COM object that simply reports which methods are called and what data is supplied for each call. Then you can drive your client application however you like and verify that it calls your COM object as you expect.

2) How can I verify my application correctly handles misbehaving COM objects?

This is the other side of the equation: verifying what happens when a COM object does not honor that aforesaid contract. Here also perhaps the simplest solution is to write a COM object that will do exactly what you tell it to do. You can pass these commands via some sort of script, or you could have a UI of some sort that lets you drive your custom object. Either way, you can see how your client app reacts to specific types of misbehavior.

3) How can I verify my application correctly imports data?

Once again your application has a contract with whatever is providing the data it imports. Go through that contract with a fine-toothed comb and test the combinations and scenarios your app needs to handle. You may be able to create much of this data yourself, but even when that is the case you'll want to do some testing with whichever data sources your customers use: other applications, XML transforms, perhaps even your own application! Try to get files from your customers; some will be happy to provide real live data, others will give you sanitized data, and yet others won't be willing (or able) to provide anything.

Well-formatted files are one thing, but you also need to know what happens when your app ingests rotten data. An easy way to generate loads of dirty data is to take a valid file and fuzz it: change one or more characters to something else. You can work through the file character by character, mutate values randomly, or use the file format's spec to target specific values - anything that seems likely to futz up the file. Then load each fuzzed file into your application (hopefully via an API - even targeted fuzzing will generate hundred or thousands of files!) and see what happens.

4) How can I verify my application correctly exports data?

This is straightforward: put your application into a particular state, export the data to every supported format, then verify the exported data. If your app supports custom exporters, think about writing one that simply dumps the data your app gives it. This makes for a simple way to verify your application is giving its exporters correct information, which will help you determine whether export bugs are caused by your app or a particular exporter.

5) Any other suggestions?

In every case I suggest you also review your bug history for your application. While you will likely find many of the test cases yourself, customers are wily creatures that have a tendency to (ab)use your application in ways you would never predict. Whether you find yourself running out of ideas for test cases or you just want some jumping off points, reviewing problems your customers have reported will give you a big help.

*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great coding skills required.

The opening keynote at STAR East this morning was James Whittaker (who I'm happy to say has joined Microsoft!) talking about security testing. James focused on three areas:

• Input. One common source of security bugs is unhandled/mishandled input of, shall we say, a special nature. SQL injection for example: entering ' OR 1=1 -- as a user name. If this "user name" is stuffed directly into a SQL query such as
      SELECT * FROM USERS WHERE USERNAME='<username>' AND PASSWORD='<password>'
  , this attack turns that query into
      SELECT * FROM USERS WHERE USERNAME='' OR 1=1 -- AND PASSWORD='<password>'
  - which returns every last record, since "1=1" is always true and everything after the "--" is turned into a comment. Oops.
• Environment. Muck with the environment and see what happens. For example, Microsoft Internet Explorer used to have a bug where it did not supply a path when it dynamically loaded its parental ratings DLL. So if you used a tool like Holodeck or Process Explorer to discover that IE loaded a DLL named MSRATINGS.DLL, and then you copied any random file side-by-side to iexplore.exe as MSRATINGS.DLL, IE would follow standard file-searching behavior and load your trojan DLL. The dynamic load would of course now fail and so IE would simply disable parental controls. You can bet thirteen-year-olds everywhere know about this hack! <g/>
• Logic. Ponder how a feature might be implemented and then look for a way to break it. For example, anytime an ActiveX control tries to load, IE pops a message box asking whether you want to let it run. IE used to have a bug where if the load was executed from script and the load was attempted multiple times - in a loop, say - you would only be asked the first time. That's nice of IE, isn't it, to not repeat a question you've already answered? Just one problem: it acted as though you had said "Yes, please load it" even if you answered "No, don't load that nasty evil thing"! All your base are belong to us, indeed.

Each of these exploits requires sophisticated security tools. Like Notepad. Which was James' point: security hacking is often not a sophisticated process. Which means there's no reason why we can't find the problems before we ship.

Of course, if you *like* seeing your company's name on the morning news you can skip security testing...

Speakers at STAR East are given the opportunity to submit a paper in support of their presentation. In addition to serving as the unlive version of the presentations, the papers are judged and the best awarded Best Paper.

Guess who won this year? <g/>

(Co-won, actually; I tied with someone else. But I'm still pretty happy!)

Thanks to everyone who helped make my STAR East presentation "Hallmarks Of A Great Tester" this afternoon go off without a hitch! Speaking to five hundred people was quite experience, let me tell you. <g/> The unlive version is now up.