Most Software Development Metrics Are Misleading And Counterproductive


that can be measured very accurately, but these point to things that have past. Managers, on the other hand, prefer an imperfect forecast of the future to a perfect report on the past. It is better to measure the size of a test queue than it is to measure the processing times of individual tests because test queue size is a leading indicator of future delays in test processing.  [2]

Scoring Our Metrics
Given these guidelines for metrics, let's assess how traditional and agile metrics perform.

Traditional metrics:

    • Lines of code written–poor, does not reward simplification, leads to code bloat.
    • Hours worked150–poor, leads to long hours, burn-out, defects, consumed budgets.
    • Function points delivered–poor, not relevant to true goal of project, effort to generate.

It is also fair to say that these three common metrics are also backwards looking, focussing only on the past  On today's high change projects they are not good indicators for the future. Running projects by relying on backwards-facing views is a little like driving your car using only the rear view mirror and fuel gauge. You can see what you have run over and how much longer we have to go before the fuel runs out. However, we need to look ahead and make adjustments (steer) in light of leading (forward-looking) metrics.

Agile metrics:

    • Features completed verses features remaining–good, simple, and self generating.
    • Customer satisfaction–good, simple, and relevant to true goal.
    • Cycle times–good, simple yet forward looking.

Agile Metrics Explored
Features completed verses features remaining is a vital metric. At the end of the day it is the business value generating features rather than lines of code or function points that add value to the customer. "Features completed" means accepted by the customer (or proxy customer) -- not just coded and unit tested–as features do not add any business value until they are really done. In fact, written but unsigned-off code represents inventory. Following the guidelines of lean manufacturing, inventory should be minimized as it signifies effort invested without a return.

Cumulative Flow Diagrams (CFDs) are great ways to show features completed verses features remaining (see Figure 1).   


Figure 1 - Sample Cumulative Flow Diagram

This chart shows the features completed verses the features remaining for a fictional project that is still in progress. The blue area represents all the planned features to be built. This number has risen from 400 to 420 in June and then to 450 in August as additional features were added to the project. The yellow area plots the work in progress, and the green area shows the total number of features completed. (CFDs are a little more complicated than burn-down charts but have the advantage of illustrating scope variances separately from productivity. A flat spot on an "Effort Remaining" burn-down chart could be the result of lower productivity or additional scope being added to the project.)

Using CFDs, the rate of progress is indicated by the gradient of the green features completed line. Cycle time (how long features take to be coded and accepted) can be determined by assessing the size of the yellow in progress area. Customer satisfaction can be captured via a "Green," "Yellow," or "Red" satisfaction ranking from the ambassador user at regular meetings. It is better to learn of a potential issue when it is still a vague uneasiness from the customer than after it manifests its self as a real problem. If "Green," "Yellow," and "Red" are too regimented, then a line drawn on a range from happy face to unhappy face, as recommended by Rob Thomsett in his book "Radical Project Management," can

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.