Skip navigation.
Home » blogs » Antony Marcano's blog

CRAP or CRAPpier?

Submitted by Antony Marcano on Wed, 14/11/2007 - 07:58. code analysis & inspection

If you've been following the testingReflections feed then you would have noticed announcements about CRAP4J.

CRAP is a metric recently devised to help developers avoid obtain feedback about certain elements of code quality.

CRAP = "Change Risk Analysis and Predictions"

CRAP4J is the java implementation of a tool that calculates and presents the metric.

There are only two dimensions (currently) - cyclomatic complexity and code coverage... CRAP is a function of the two.

This is great and I encourage it... but are these the only two facets of code that we need to measure to evaluate quality in terms of risk of change?

I doubt it.

I don't discourage people from using CRAP4J, however I do suggest that people understand the limitations of the metric... there's a whiff of leaky abstractions here that it would be all too easy for people to read too much into the metric if they don't understand the value add of the metrics that it is based upon. Worse still, placing too much faith in the metric without understanding it may lead to fixing one problem and introducing a new, worse "risk of change" issue that CRAP isn't measuring (yet).

One thing in particular bothers me ever so slightly...

Some very smart people we talked with believe that if you have 100% test coverage, your risk of change should be 0. Bob and I, however, believe that more complex code, even if it’s fully protected by tests, is usually riskier to change than less complex code.

Although this may be a belief - a hypothesis - is it proven?

Jason Gorman has been asking these sorts of questions about 'Code Quality Metrics' and the 'change risk' for a while now. He's been challenging the assumptions behind several of code-quality metrics. It turns out that many 'code quality' metrics are based more on belief than actual scientific proof. Jason is challenging these beliefs, scientifically, and I recommend following Jason's blog to keep up to date on where he is with his efforts.

Perhaps some of his research will influence how CRAP, as a metric, evolves.

So, do draw upon tools like CRAP4J but be wary of its basis and understand the metric... supplement it with other metrics... scientifically prove that it is reducing 'risk of change' when you have less CRAPpy code and be wary of the metrics equivalent of leaky abstractions.

» Antony Marcano's blog | login or register to post comments

SWING base applications

Submitted by ppathak on Thu, 15/11/2007 - 05:25.
Respected Sir,
I am exploring into using the OpenSTA in my org. Before I spend more time, I qucikly wanted to know that if this tool can be equally used for SWING based applications .
Your answer of yes OR no would be very helpful to explore further on the tool.

Thanks for taking pain to reply, in advance.
- Prabhu
» login or register to post comments

Good post

Submitted by Alberto Savoia on Wed, 14/11/2007 - 17:16.
Hi Antony,

Good post. We (Bob and I) take great pains to ensure that people don't abuse, misuse, or overuse CRAP and understand its limitations. We are also trying to be very clear about the fact that CRAP is designed to evolve and change as people gain experience with it - which is the only way to come up with a sensible and up-to-date metric IMO.

Given the amount of buzz and attention that CRAP is generating, I feel a strong responsibility to ensure that it's used properly.

I have had mixed feelings about metrics since I started studying and using them a couple of decades ago. I saw them used with great success to detect and catch problematic code, improve development practices, etc. But I've also seen them abused and misused by both developers and managers.

Based on my observations, the key difference between metric use and misuse/abuse/overuse is the intelligence and experience of the people using them. Smart and experienced developers/managers know how to use metrics. Not-too-smart or inexperienced developers/managers often misuse them.

A key slogan about metrics (in general, not just software) is to make sure that YOU use the metric and don't have the metric use YOU.

Alberto

PS We have been talking about CRAP evoloving through use for a while and we are about to release a new version of Crap4J that gives users the option to share their CRAP data and other project stats anonymously. This is the first step toward getting feedback and building a database that, hopefully, over time will give us lots of empirical data. Stay tuned.
» login or register to post comments

Comment viewing options

Select your preferred way to display the comments and click 'Save settings' to activate your changes.