Building a Meaningful Metric Mousetrap


Metrics provide data points that can both benefit and endanger and organization.   Metrics can be used positively to build a better organization and can be used negatively to punish organizations and people therein.  Many times, those that use the metrics negatively do it purposefully, but other times, they are not aware of the way they are using them.  This is why it is important to have a metrics culture that apply metrics in a positive manner, provides an understanding of the metric, and then actually utilize metrics to manage an organization. 


Often times an organization collects numerous measures and generates numerous metrics. In many of these cases, it is unclear why certain metrics are being collected. In the long run, a majority of these are not actually used to manage an organization; they become ignored and deemed worthless. It is the combination of ignored metrics and the negative use of metrics that begins the downfall of the perceived value of metrics and, therefore, leads to a very negative metrics culture.

Constructing a Value-Added Metric
There are many challenges to establishing a positive metrics culture. One of the primary objectives to building a positive metrics culture is to ensure they are solving a problem and designed to be value-added.

It is important to gather input prior to establishing metrics. For input, consider the following:   It is necessary to uncover problem areas in the organization, the application team, or project. It is also important to identify the value-added metrics in use by other organization.. The possibility of effectively using metrics to alleviate the pain felt by groups in certain areas can be a big motivator to implementing metrics.

The key building blocks to value-added metrics are: 1) ensuring the organization understands the benefit of the metric and how it can be used to improve the organization; 2) understanding the level of effort to collect the metrics (both establish and maintain); and 3) providing clarity on who really benefits from and will use the metric. By assessing the benefit versus the effort, a value-rating can be assigned to each metric. Those that have high value-ratings can be implemented. Once a metric is in place, monitoring should then occur to verify if the metric is actually being used to manage the problem area or organizational change. If not, discard the metric. For the sake of this discussion, let us say that the application team has identified that build times are a major problem and impacts delivery times to test and worse to production. The application team is unclear of how long they should expect a build to take and build times are perceived to range drastically in duration. Let's explore the key building blocks in more detail by proposing a metric called "time to build the product" within the software configuration management (SCM) field.

Understand the Benefit of the Metric
When considering the benefit of the metric, first take a moment to describe it. In the case of "time to build the product", the description is, a metric that determines the average duration to build (e.g., initiate, compile, package, and smoke test) a product from start to finish.

Next consider what areas that the metric can be used for (a.k.a., the benefits). This can be a consensus-driven discussion where the problem it solves or the opportunities it gains are considered. Essentially, this drives the "why" (e.g., why would we want to establish this metric). Several examples of why the metric can be of benefit and how it can be used to improve an application team's effort if establishing a "time to build the product" metric are:

·         Identifying potential problems when there are large deviations from average build times or when build times get longer and longer.

·         Setting customer expectations for scheduling and planning. If it always takes 2 hours to build an application, and management tasks for a build in 30 minutes, it's much better to have hard numbers ready to explain why that's not possible.

·         Measuring expected build time gains (or losses) caused by implementing:

1.       New functionality in applications

2.       System change/improvements



About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.