Building a Meaningful Metric Mousetrap

[article]

Build process changes/improvements

Determine Effort to collect the Metric
Now that folks are aware of the benefits and use of the metric for improvement, it is time to consider the level of effort needed to collect the metric. This is critical, as there are many cases where the effort to collect the metric outweighs the perceived benefit. If this is the case, then it is not worth proceeding forward with the metric. Total effort should include the setup effort and the ongoing maintenance effort. Here is an example of considerations in setting up a "time to build the product" metric:

·         Define the process

·         When the build is initiated, record a time stamp. At the start of each step in the build process (e.g., build, compile, package, and smoke test) a record time stamp, and at the end of the build process record another time stamp. The data should be written into a build log

·         Determine average times for each step in the build process

·         Automate process

·         Script the process so it can automatically run

·         Send output to a log (average times for each step)

·         Test the automated process

·         Adjust as needed

Identify Who Benefits most from the Metric
It is important to identify who benefits most from the metric. Engaging with these people will help give you an understanding if and how they may use the metric. Here is an example of those that may benefit most from the "time to build the product" metric:

·         SCM engineers for build time monitoring, scheduling, and for having a baseline of data to improve build times

·         Development for scheduling and awareness of build time

·         Project Management for scheduling and awareness of build time

Assess Value of the Metric
Is the metric a value to an application team or organization? This is a critical step. Once the benefits of the metrics are known, the effort to construct and maintain the metrics are identified, and it is clear who will benefit from the metrics, then a comparison of the benefits versus the effort should occur. It should include input from those that will use the metric.

Typically the values used to determine the effort of a build is measured in hours or days. The tricky part is to establish meaningful values to determine the benefit of a metric. This can be subjective from person to person. A tip is to consider how many hours in a given year are worth setting up and maintaining a metric. Is 50 hours per year reasonable or 100? Using this value can provide a basis to measure effort against. Then it is important to ask the people who may benefit from the metric if the benefit is worth the effort. Using the "time to build the product" metric as an example, let's compare the perceived benefit versus the effort.

Perceived Benefit
Working with the group that would benefit most from the metric (from the "who benefits most and uses the metric" section), it was determined that 100 hours was a reasonable amount of time to set up and maintain a metric of this nature. Using 0-100 as the range, ask each person to determine the perceived benefit of "time to build the product" metric (from 1-100 with 100 being the highest):

 


Role




Perceived Benefit




SCM Engineer #1




80




SCM Engineer #2




90




Lead Developer
(representing Development)




60




Project Manager




70




Average




75



 

Effort

Determine the effort to establish the "time to build the product" metric.

About the author

Mario  Moreira's picture Mario Moreira

Mario Moreira is a Columnist for the CM Journal, a writer for the Agile Journal, an Author, an Agile and CM expert for CA, and has worked in the CM field since 1986 and in the Agile field since 1998. He has experience with numerous CM technologies and processes and has implemented CM on over 150 applications/products, which include establishing global SCM infrastructures. He is a certified ScrumMaster in the Agile arena having implemented Scrum and XP practices. He holds an MA in Mass Communication with an emphasis on communication technologies. Mario also brings years of Project Management, Software Quality Assurance, Requirement Management, facilitation, and team building skills and experience. Mario is the author of a new book entitled “Adapting Configuration Management for Agile Teams” (via Wiley Publishing). It provides an Agile Primer and a CM Primer, and how to adapt CM practices for Agile Teams. Mario is also the author of the CM book entitled, “Software Configuration Management Implementation Roadmap.” It includes step-by-step guidance for implementing SCM at the organization, application, and project level with numerous examples. Also consider visiting Mario’s blog on CM for Agile and Agile adoption at http://cmforagile.blogspot.com/.
 

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!