Building a Meaningful Metric Mousetrap


This approach looks at total effort and includes the set-up time and the on-going effort to maintenance the metric. Calculate the hours:




Set-up Effort
(define and automate process)


On-going Effort

12 (1 monthly)

Effort per Year



Determining value is critical factor. If the perceived benefit exceeds the total effort hours in a year (e.g., the value-rating is greater than 1 when perceived value is divided by hours), then the metric may be considered a value-add to the organization or application. In this case, if we divide the perceived benefit of 75 by the total effort hours in a year of 37, we get 2.03 value-rating (rounded up). When using this approach, you will often find that the effort is greater than the perceived benefit (e.g., value-rating is less than 1 and should therefore not be considered as a potential metric).

Comparing a Potential Metric
When embarking on a metrics program, it is important to look at a number of potential metrics in order to determine their value-rating in relation to other metrics. As stated in the early section, gather input by a) uncovering problem areas in the organization, the application team, or project and b) identifying what value-added metrics are used in other organizations. Discuss the metrics with those benefitting from it and ensure they would, in fact, use the metric for improvements.

Of those potential metrics that remain, go through the steps described above (e.g., understanding the benefit of the metric, determine effort to collect the metric, identify who benefits most from the metric, and assess value of the metric). Then compare the potential metrics together. Here is an example of a metrics comparison table that uses the value-rating as its gauge.

Note: it includes the "time to build the product" metric discussed above:


Potential Metric Name

Perceived Benefit



Time to Build
the Product




Build Errors




Code Volatility




Change Control





In this case, we see four potential metrics. Of the four, the SCM metrics program may only choose to pursue the "time to build the product" and the build errors metric since it has the highest metrics value-rating.

Note: there will be times when the value-rating is below 1 but the organization still wants to proceed with a metric. In these cases, it is important to communicate the risk of the metric being perceived as not adding value.

Monitoring the Metric
As time goes on, any metric that has been established and is produced on an on-going basis should be revisited on a periodic basis. The reasons are two-fold: First, it is important to periodically check if the metric is actually being used. If it is not being used, then the metric should be discontinued. Second, the value-rating of the metric should be re-evaluated. Is the benefit still perceived to be high as time moves forward? Is the effort more or less than what was initially calculated? In some cases, once a metric is used and drives a positive change in the organization, then the metric has done its job and may no longer need to be generated (or generated less frequently then on a monthly basis).

It is also important to allow the data to collect for a few cycles prior to prescribing a target for a metric. It is important to identify the range of scores a metric produces over a reasonable time period to understand what "average" is and what are variations. Once this is done, then a target for improvement

About the author

Mario  Moreira's picture Mario Moreira

<strong>Mario Moreira</strong> is a Columnist for the CM Journal, a writer for the Agile Journal, an Author, an Agile and CM expert for CA, and has worked in the CM field since 1986 and in the Agile field since 1998. He has experience with numerous CM technologies and processes and has implemented CM on over 150 applications/products, which include establishing global SCM infrastructures. He is a certified ScrumMaster in the Agile arena having implemented Scrum and XP practices. He holds an MA in Mass Communication with an emphasis on communication technologies. Mario also brings years of Project Management, Software Quality Assurance, Requirement Management, facilitation, and team building skills and experience. Mario is the author of a new book entitled “<strong><a href=";camp=213761&amp;cre... Configuration Management for Agile Teams</a></strong>” (via Wiley Publishing). It provides an Agile Primer and a CM Primer, and how to adapt CM practices for Agile Teams. Mario is also the author of the CM book entitled, “<strong><a href=" Configuration Management Implementation Roadmap.</a></strong>” It includes step-by-step guidance for implementing SCM at the organization, application, and project level with numerous examples. Also consider visiting Mario’s blog on CM for Agile and Agile adoption at <a href=""></a>.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Nov 09
Nov 09
Apr 13
May 03