testingReflections.com - design & development

De-smelling by refactor or rewrite: some thoughts

Sun, 17 Jan 2010 01:14:39 +0000

Is there a tipping point when rewriting code should be favored over refactoring ? It is often hard to decide. There are some important factors that can help. Code smells cluster, whether as bugs that are detected, or potential failures lurking in immature code that may break during future updates or maintenance. Steve McConnell wrote (in “Rapid Development”) that “Error prone modules tend to be more complex than other modules in the system, less structured and unusually large. They often were developed under excessive schedule pressure and were not fully tested”. He also quotes indicative studies that these modules are much more expensive to develop. Steve’s metric is 10 defect per 1000 lines of code to mark a module for further work. If it appears to be an error prone module, he advises rewriting it. Other indicators are large concentrations of bugs, large number of checkins, or large amounts of fix time spent in comparison to similar modules. While error-prone modules are typical in traditional development, agile practices like well executed TDD should minimize them. High cyclomatic complexity of code is also a good indicator of the need for simplification.

The hard thing is determining the tipping point between refactor and rebuild. The easy thing is that because of clustering, there is often a lot of low hanging fruit, and focused effort on a small portion of the code should have a massive impact on the code quality.

For more on bug clusters and other productivity ideas, see my Google tech talk “Building Software Smarter”, http://www.tinyurl.com/80-20-rules

Short and long game thinking, tests driving design and CRAP metrics

Thu, 28 May 2009 05:59:56 +0100

Kent Beck recently posted To Test or Not to Test? That’s a Good Question on the complex “theory versus practice” issue of always automating tests, where he states,”Then a cult of [agile] dogmatism sprang up around testing–if you can conceivably write a test you must”. By classifying projects into long game and short game, he argues that ROI becomes a major issue on whether a test stays manual. He says “Not writing the test for the second defect gave me time to try a new feature”, but several people commented that this was a technical debt tradeoff, and Guilherme Chapiewski noted he had done the same thing with a Proof of Concept that went live then he had to rewrite major chunks later. It is interesting that this ROI discussion is reflecting the experiences of the pre-agile functional automation community. Back in November 2001 (Wow! Long time ago!!), I posted to the Agile Testing list some considerations for not automating . While many of these were from the context of two separate development teams and the automaters using expensive test tools, the risks of incomplete automation and insufficient ROI dominate. The benefits of having the same people both develop the code and the tests are great, and beyond my experience when I wrote that post.

I think the ROI issue for code-based tests will go away over time. Much of the creation of code-based tests is mechanical. Just as programming languages replaced assembler and took care of fiddly details (what registers to use, low level comparisons etc) and build utilities replaced simple text file include statements, I think that soon it will be standard practice to have tool-created unit testing to handle mocking, dependency injection and assert-based testing. Mocking was originally very manual, then tools were developed. Dependency Injection was very manual,then tools were developed. For assert-based testing, we’ve already seen Agitar’s tools , zentest and now pex amongst others. I think these tools will become standard, just as coverage tools are now standard in IDEs when they originally were luxuries costing tens of thousands of dollars. Another variation of this is tools like Celerity recently blogged about by Jeffrey Frederick. Celerity is a fast way to run GUI web tests, but could be handled as a mechanical translation not a manual one. Some meta language could generate Celerity and selected browser tests in a single step.

Mechanically generated tests are cheap to produce and overcome ROI issues. However, they only reflect the current code. The benefits of test design infusing the coding approach are missing. If tests are not being automated for whatever reason, some analysis of the refactoring risk should be done, at least to know where and what the error-prone code is. One way of doing this is using the Agitar-created CRAP metric , which Bob Martin recently blogged about as a way to keep design clean. While I currently believe all code should be created test first wherever possible, techniques like the CRAP metric can highlight the complicated bits for refactoring where possible. While it may be a great intellectual challenge, there is no need to refactor a complex industry standard algorithm. [Aside: is there an inherent advantage to doing test first design all the time? Perhaps, just as renaissance masters only painted and sculpted hand and faces and left the rest to their workshop staff, we only need to focus on core functions for test first and do the rest test last?]

As Kent says,”By insisting that I always write tests I learned that I can test pretty much anything given enough time.” Time is often a rare commodity, so Kent argues compromises are often needed in short goal projects. As Ron Jeffries said in a comment on Kent’s post, “My long experience suggests that there is a sort of knee in the curve of impact for short-game-focused decisions. Make too many and suddenly reliability and the ability to progress drop substantially.” I hope that advancements in mechanical generation of tests don’t push us into a short game perspective, impacting the use of hand crafting tests to drive design. At the same time, metrics that can be run as part of the build to highlight areas for refactoring on all projects are proving valuable (and I’m looking forward to state coverage ). By any measure, these are interesting times we live in. Long live long game thinking!

Software Performance Engineering

Wed, 12 Nov 2008 04:34:31 +0000

Trying to make some suggestions for Building Responsive and Scalable Applications panel I contemplated for a while on this topic.

For a long time we have Software Performance Engineering addressing these issues. As it described in the books and papers of Dr. Connie Smith and Dr. Lloyd Williams, it looks rather as a methodology for me.

The short and the long of IT: two videotaped presentations

Sun, 07 Jun 2009 05:58:45 +0100

Since the middle of the year, I’ve presented and facilitated about 12 hours of sessions at 2 traditional and 3 open space conferences, plus a Googleplex visit. Two presentations are now on video, both filmed on the other side of the world from my usual location of Melbourne, Australia.

A lightning talk (at the functional tools workshop held before Agile 08 in Toronto, Canada) called Shades of Green discusses how the “green” passing tests of functional automation may not be as green as they seem. Note the static pose to stay within camera range, compensated for by the wildly waving arm. And yes, the audience was not limited to a leg and a foot.

An order of magnitude longer at around 50 minutes, a Google Tech Talk (filmed at the San Francisco Googleplex) called 80:20 Rules! Building Software Smarter looks at formal and informal ways to get significant improvements in creating software, including various puzzles and questions for viewers. I had looked at some tech talks by other people I know, and they had been watched around 1000 times over a year or so. It looks like I may hit that mark only a few days after the video was posted which is great. I hope my talk inspires people to build their software smarter. Can I turn “shades of green” into a similar talk? Probably not!

ISubmitBlogPosts - a nice twist on Hungarian Notation for Interfaces...

Mon, 23 Mar 2009 06:17:34 +0000

I've been pairing with Andy Palmer over this last week. I have to say it's been a lot of fun... and I think we've learned a lot from the experience. One of the things I learned from Andy this week was an innovative use of Hungarian notation for interfaces... Andy told me about Udi Dahan's presentation on intentions and interfaces (PDF)

Pitfalls of the "Waterfall" Approach to Performance Testing

Mon, 24 Mar 2008 21:23:17 +0000

Looks like the pre-production validation approach to performance testing becomes typical for large corporations (if there is any at all):

-get the system ready
-develop all scripts requested (sometimes offshore)
-run them all together
-compare with the requirements provided
-allow some percentage of errors according to the requirements
-involve the development team if requirements were missed

The P28 virtual fence: Borderline success or virtual vaporware?

Wed, 05 Mar 2008 10:06:01 +0000

I’ve already noted some of the issues around the virtual fence here. In May 2006, George W. Bush called it “the most technologically advanced border-security initiative in American history”. In a flash of originality, the 28 mile virtual fence in Arizona was called Project 28, or P28 in Chertoff-speak. Micheal Chertoff has now accepted the fence system, on the basis that “all of the defects” in the prototype project were either “cured” or “immaterial.” He also said “In some form or fashion, technology is going to be virtually every place on the border, but it’s not necessarily going to be in the configuration of P28,” The chair of the House Committee on Homeland Security, Representative Bennie Thompson, sees it differently, “The poorly structured contract that prevented the line Border Patrol agents from pointing out obvious flaws and caused an overreliance on contractors has resulted in a system that has been described as providing ‘marginal’ functionality at best,” I wonder if he was tempted to say the Border fence had borderline functionality. [grin]

The technology stack on top of each tower looks impressive . The project is anything but impressive , with many of the issues typical in the IT industry: badly underestimated effort, not involving users, and not enough testing, While there was evidently a push for reuse of existing systems and components, it caused many difficulties: trying to base the system around a law enforcement dispatch system (!) that couldn't scale, trying to use off-the-shelf components that weren't designed to integrate, and a lack of standards for the sensors. The builder of the system, Boeing, has only taken 3/4s of the $20 million fee, granted a $2 million credit, and apparently paid $40 million to get this far on a project that will now finish in 2011, not the end of this year (which was already a year and a half past the original schedule). That will only be phase one!

CMG Opens Its Content to the Public

Tue, 12 Feb 2008 19:30:21 +0000

The Computer Measurement Group (CMG) is making its conference proceedings from 1997 through 2005 available to the public. I believe that CMG holds the best practical conference in performance analysis, capacity planning, and related areas. In addition to the areas listed below, I'd definitely add performance testing. Here is the official mail CMG sent:

Education

Sun, 03 Feb 2008 05:08:05 +0000

With great interest read Who Killed the Software Engineer? (Hint: It Happened in College) by James Maguire as well as the original article Computer Science Education: Where Are the Software Engineers of Tomorrow? by Dr. Robert Dewar and Dr. Edmond Schonberg.

Performance Requirements

Sun, 06 Jan 2008 23:08:27 +0000

My performance requirements paper was published in the January issue of Software Test & Performance (pp.18-24).

It was simple: I just sent a draft – and now I am reading it printed. With a new name - or even two: it is referred as You Can Gauge Performance Without Requirements in one place and Gauging Performance in The Absence of Measures in another. Not to mention other minor improvement.

From the Mailbox: Software Development: Art or Science?

Thu, 18 Oct 2007 20:47:12 +0100

Here’s a question that I didn’t realize I had much to say about until I read my own response.

The Question:
Software Development: Is it an art or a science? An age old question I know, but what do you think and why?

My Response:

I refer to new software development as a scientific art. I've seen some maintenance work, platform porting, etc. that has been almost entirely mechanical -- I'm not sure what that counts as, but I certainly didn't witness anything "artistic".

What Skills Performance Testers Need and How to Get Them?

Mon, 15 Oct 2007 15:31:39 +0100

From time to time I see questions on different forums asking what skills are necessary for performance testers. There were pretty interesting discussions. Looks like most experts agree that performance testing requires more skills than just knowledge about how to create a script for a particular load testing tool. While it is still possible to imagine a performance tester in a large corporation with deep specialization who only creates scripts and mechanically runs them while other performance experts monitor the system and analyze results, I don't see many perspectives neither for this person, nor for the approach. Systems become so complicated now that the sum of specialized expert views doesn't give the whole performance picture.

Guerilla Approach

Thu, 04 Oct 2007 20:06:44 +0100

Mark Friedman reviewed Neil J. Gunther's new book "Guerilla Capacity Planning: A Tactical Approach to Planning for Highly Scalable Applications and Services"
in MeasureIT (free registration may be needed - I hate registrations, but this one is worth efforts if you care about performance: it is a very good newsletter).

"Shot in the Dark" anti-pattern

Sat, 02 Sep 2006 03:00:50 +0100

Kirk proposed the "Shot in the Dark" anti-pattern. By my opinion, a must reading for all working with performance issues.

Predicting the Path of the Storm

Tue, 27 Jun 2006 18:11:11 +0100

Recently, three unrelated events have given birth to a new idea in my head, and I wanted to share it with you. So, to properly set up the background, the three unrelated events:

At the recent CAST, James Bach presented a keynote, “Against Certification”. During that presentation, one of the things he did was to review some of the “Body of Knowledge” documentation upon which such certifications were based. As he reviewed this documentation, one of the things I was struck by was an underlying pattern of motive driving much of the documentation. That motive was the ability to predict, manage, and control the SDLC. Much of this “Body of Knowledge” for test certification was obviously written by managers, not testers.