Metrics provide data points that can both benefit and endanger and organization. Metrics can be used positively to build a better organization and can be used negatively to punish organizations and people therein. Many times, those that use the metrics negatively do it purposefully, but other times, they are not aware of the way they are using them. This
is why it is important to have a metrics culture that apply metrics in a positive manner, provides an understanding of the metric, and then actually utilize metrics to manage an organization.
the long run, a majority of these metrics do not actually get used to manage an organization. These metrics become ignored and deemed worthless. It is the combination of ignored metrics and the negative use of metrics that begins the downfall of the perceived value of metrics and leads to a very negative metrics culture.
Constructing a Value-Added Metric
There are many challenges to establishing a positive metrics culture. One of the primary objectives to building a positive metrics culture is to ensure they are solving a problem and designed to be value-added.
It is important to gather input prior to establishing metrics. For input, consider a) uncovering problem areas in the organization, the application team, or project and b) identifying the value-added metrics other organizations are using. The possibility
of effectively using metrics to alleviate the pain felt by groups in certain areas can be a big motivator to implementing metrics.
The key building blocks to value-added metrics are: 1) ensuring the organization understands the benefit of the metric and how it can be used to improve the organization; 2) understanding the level of effort to collect the metrics (both establish and maintain); and 3) providing clarity on who really benefits from and will use the metric. By
assessing the benefit vs. the effort, a value-rating can be assigned to each metric. Those that have high value-ratings can be implemented. Once a metric is in place, then monitoring should occur to verify if the metric is actually being used to manage the problem area or organizational change. If not, discard the metric.
For the sake of this discussion, let us say that the application team has identified that build times are a major problem and impacts delivery times to test and worse to
production. The application team is unclear of how long they should expect a build to take and build times are perceived to range drastically in duration. Let's explore the key building blocks in more detail by proposing a metric called "Time to Build the Product" within the Software Configuration Management (SCM) field.
When considering the benefit of the metric, first take a moment to describe it. In
the case of "Time to Build the Product", the description is, a metric that determines the average duration to build (e.g., initiate, compile, package, and smoke test) a product from start to finish.
Next consider what areas that the metric can be used for (a.k.a., the benefits). This can be a consensus-driven discussion where the problem it solves or the opportunities it gains are considered. Essentially, this drives the "why" (e.g., why would we want to establish this metric). Several examples of why the metric can be of benefit and how it can be used to improve an application team's effort if establishing a "Time to Build the Product" metric are:
- Identifying potential problems when there are large deviations from average build times or when build times get longer and longer.
- Setting customer expectations for scheduling and planning - if it always takes 2 hours to build an application, and managementasks for a build in 30 minutes, it's much better to have hard numbers ready to explain why that's not possible
- Measuring expected build time gains (or losses) caused by implementing:
- New functionality in applications
- System change/improvements
- Build process changes/improvements
Determine Effort to collect the Metric
Now that folks are aware of the benefits and use of the metric for improvement, it is time to consider the level of effort needed to collect the metric. This is critical because there are many cases where the effort to collect the metric outweighs the perceived benefit. If this is the case, then it