Building a Meaningful Metric Mousetrap


Build process changes/improvements

Determine Effort to collect the Metric
Now that folks are aware of the benefits and use of the metric for improvement, it is time to consider the level of effort needed to collect the metric. This is critical, as there are many cases where the effort to collect the metric outweighs the perceived benefit. If this is the case, then it is not worth proceeding forward with the metric. Total effort should include the setup effort and the ongoing maintenance effort. Here is an example of considerations in setting up a "time to build the product" metric:

·         Define the process

·         When the build is initiated, record a time stamp. At the start of each step in the build process (e.g., build, compile, package, and smoke test) a record time stamp, and at the end of the build process record another time stamp. The data should be written into a build log

·         Determine average times for each step in the build process

·         Automate process

·         Script the process so it can automatically run

·         Send output to a log (average times for each step)

·         Test the automated process

·         Adjust as needed

Identify Who Benefits most from the Metric
It is important to identify who benefits most from the metric. Engaging with these people will help give you an understanding if and how they may use the metric. Here is an example of those that may benefit most from the "time to build the product" metric:

·         SCM engineers for build time monitoring, scheduling, and for having a baseline of data to improve build times

·         Development for scheduling and awareness of build time

·         Project Management for scheduling and awareness of build time

Assess Value of the Metric
Is the metric a value to an application team or organization? This is a critical step. Once the benefits of the metrics are known, the effort to construct and maintain the metrics are identified, and it is clear who will benefit from the metrics, then a comparison of the benefits versus the effort should occur. It should include input from those that will use the metric.

Typically the values used to determine the effort of a build is measured in hours or days. The tricky part is to establish meaningful values to determine the benefit of a metric. This can be subjective from person to person. A tip is to consider how many hours in a given year are worth setting up and maintaining a metric. Is 50 hours per year reasonable or 100? Using this value can provide a basis to measure effort against. Then it is important to ask the people who may benefit from the metric if the benefit is worth the effort. Using the "time to build the product" metric as an example, let's compare the perceived benefit versus the effort.

Perceived Benefit
Working with the group that would benefit most from the metric (from the "who benefits most and uses the metric" section), it was determined that 100 hours was a reasonable amount of time to set up and maintain a metric of this nature. Using 0-100 as the range, ask each person to determine the perceived benefit of "time to build the product" metric (from 1-100 with 100 being the highest):



Perceived Benefit

SCM Engineer #1


SCM Engineer #2


Lead Developer
(representing Development)


Project Manager






Determine the effort to establish the "time to build the product" metric.

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.