Most Software Development Metrics Are Misleading And Counterproductive

[article]
Summary:
The software development industry has a poor track record for developing and employing useful software metrics. This is because most of the metrics selected are tangential to the true goal of software development—delivering business value—and instead focus on software attributes and accounting measures. Metrics such as lines of code per developer week, function points created, hours worked, or budget consumed appear important measures, but they have dangerous and counterproductive implications.

The software development industry has a poor track record for developing and employing useful software metrics. This is because most of the metrics selected are tangential to the true goal of software development—delivering business value—and instead focus on software attributes and accounting measures.

Metrics such as lines of code per developer week, function points created, hours worked, or budget consumed appear important measures, but they have dangerous and counterproductive implications. The use of these metrics rewards the wrong behaviors; the phrase "you get what you measure" highlights the problem. By tracking lines of code written, visible and unconscious incentives to generate lots of code are established. On the surface this may seem attractive, as a manger of a project it is gratifying to see lots of code being written, but what is really required is functionality completed, business value generated, and customers satisfied.

The more code generated the harder a system is to maintain and extend. With incentives like lines of code written, how do value-adding activities like refactoring simplifications appear? Reducing 20,000 lines of code to 15,000 is a good thing, but from the lines-of-code perspective it looks like the project is going backwards.

Be Aware Of The Hawthorn Effect
During the 1920's Elton Mayo conducted a number of worker productivity experiments at the Western Electric Company in Chicago. [1] These experiments included taking a selection of office workers into a controlled environment where the lighting brightness could be varied and then measuring their productivity. Mayo found that in the new brighter conditions, workers in the test group {sidebar id=1} were more productive than they had been in the main office. After increasing the lighting levels further, their productivity went up yet again. (These findings must have seemed promising for Western Electric shareholders who sold electrical lighting systems to offices.) However, as a last control experiment, Mayo reduced the lighting brightness to the default level and measured worker productivity one last time. Despite the dimmer conditions, productivity went up yet again and it was concluded that it was not the lighting levels at all but the fact that people were being measured that increased their productivity.

These experiments and the widely accepted phenomenon that by measuring something you will influence it gave rise to the Hawthorne Effect (named after the Western Electric factory). Because we will likely influence what we measure, we should be very careful about what we visibly measure. Lines of code written and hours worked are great examples of bad things to visibly measure on software projects, as they will likely lead to bloated code and quickly consumed budgets. Instead, we should use the Hawthorn Effect to our own advantage and measure things we would happily influence.

An example of a good metric is features completed compared to features remaining . It is simple and relevant to the true goal of the project, getting the project finished to the satisfaction of the customer. Donald Reinertsen, author of "Managing the Design Factory," offers this advice for metric selection:

First, they should be simple, the ideal metrics are self-generating in the sense that they are created without extra effort in the normal course of business. Second, they should be relevant. One test of relevance is whether the metrics focus on things that are actually controllable by the people being measured. Psychologists have found that when people think that they can control something they are more motivated to control it. Measuring people on things they can not control simply causes stress, dissatisfaction, and alienation. A third characteristic is that they should be focused on leading indicators. Accountants like lagging indicators

About the author

TechWell Contributor's picture TechWell Contributor

The opinions and positions expressed within these guest posts are those of the author alone and do not represent those of the TechWell Community Sites. Guest authors represent that they have the right to distribute this content and that such content is not violating the legal rights of others. If you would like to contribute content to a TechWell Community Site, email editors@techwell.com.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!