# Building a Meaningful Metric Mousetrap

[article]

This approach looks at total effort and includes the set-up time and the on-going effort to maintenance the metric. Calculate the hours:

 Task Hours Set-up Effort (define and automate process) 25 On-going Effort (monthly) 12 (1 monthly) Effort per Year 37

Value
Determining value is critical factor. If the perceived benefit exceeds the total effort hours in a year (e.g., the value-rating is greater than 1 when perceived value is divided by hours), then the metric may be considered a value-add to the organization or application. In this case, if we divide the perceived benefit of 75 by the total effort hours in a year of 37, we get 2.03 value-rating (rounded up). When using this approach, you will often find that the effort is greater than the perceived benefit (e.g., value-rating is less than 1 and should therefore not be considered as a potential metric).

Comparing a Potential Metric
When embarking on a metrics program, it is important to look at a number of potential metrics in order to determine their value-rating in relation to other metrics. As stated in the early section, gather input by a) uncovering problem areas in the organization, the application team, or project and b) identifying what value-added metrics are used in other organizations. Discuss the metrics with those benefitting from it and ensure they would, in fact, use the metric for improvements.

Of those potential metrics that remain, go through the steps described above (e.g., understanding the benefit of the metric, determine effort to collect the metric, identify who benefits most from the metric, and assess value of the metric). Then compare the potential metrics together. Here is an example of a metrics comparison table that uses the value-rating as its gauge.

Note: it includes the "time to build the product" metric discussed above:

 Potential Metric Name Perceived Benefit Effort Value-Rating Time to Build the Product 75 37 2.03 Build Errors 85 45 1.89 Code Volatility 45 60 0.75 Change Control Volatility 55 45 1.22

In this case, we see four potential metrics. Of the four, the SCM metrics program may only choose to pursue the "time to build the product" and the build errors metric since it has the highest metrics value-rating.

Note: there will be times when the value-rating is below 1 but the organization still wants to proceed with a metric. In these cases, it is important to communicate the risk of the metric being perceived as not adding value.

Monitoring the Metric
As time goes on, any metric that has been established and is produced on an on-going basis should be revisited on a periodic basis. The reasons are two-fold: First, it is important to periodically check if the metric is actually being used. If it is not being used, then the metric should be discontinued. Second, the value-rating of the metric should be re-evaluated. Is the benefit still perceived to be high as time moves forward? Is the effort more or less than what was initially calculated? In some cases, once a metric is used and drives a positive change in the organization, then the metric has done its job and may no longer need to be generated (or generated less frequently then on a monthly basis).

It is also important to allow the data to collect for a few cycles prior to prescribing a target for a metric. It is important to identify the range of scores a metric produces over a reasonable time period to understand what "average" is and what are variations. Once this is done, then a target for improvement