Metrics and Process Maturity

query them easily as you need. Perhaps they come out as numbers, perhaps as graphs as well. Great. Even better if you can export your data to a graphical slicer and dicer (e.g. Excel spread sheet).

Next you need to be able to dig down into the metrics - why is this number what it is. Click on the extra long or extra short bar on your bar graph and identify the specific items. Or zoom in on a cell of your tabular summary. This is more than an problem solving exercise - it's a learning exercise. You want to be able to understand the metrics, not just see them. You want to investigate anomalies too. Unfortunately, over-night generated metrics (as opposed to interactive ones) will require a bit more work to dig.

Finally, you need to be able to specify what you want to look at. Perhaps your queries are grouped by related areas. For example, metrics which are shown across development streams, or metrics dealing with the administration of your CM tool (or your environment). Here's what I would recommend.

• Specify the domain of your metric - what set of objects are you looking at.
• Specify the attribute you want to measure.
• Specify the granularity of your measurement.
• Group your metrics - tag them if possible so that metrics may appear in more than one grouping.

The good thing about a CM repository is that it has all of the history in it (I hope!). You shouldn't have to worry about saving your metrics because you can reproduce them at any time. OK, some metrics may take longer to produce than others - for example if you have tens of thousands of files and you're computing function point metrics, these are not going to pop up in a point-and-click response time - at least not until we go through another 10 years of technology advances. So you may want to save some of your metrics.

Ideally, you want to be able to set an arbitrary context view and compute many of your metrics based on that. You want to be able to point to a specific stream and compute metrics for it. To a specific build and compute metrics for it. To a part of the organization and compute metrics for it. This is data mining, but in a revision-rich repository containing, hopefully, everything you've ever wanted to know about metrics but were afraid to ask.

Too much data? The next level of optimization is the ability to look through your metrics and tell you the concerns. This doesn't have to be a 22nd century artificial intelligence capability. It can be a simple set of checks, hard-coded for each metric, looking for large deviations from the norm and giving metric specific interpretations for these deviations. For example, department xyz has an actual to projected effort ratio which is 3 times that for the rest of the project. If your metrics are easy to specify, you won't mind spending a bit of effort to automatically interpret and hi-lite the results. If your metrics are hard to specify and implement, you'll likely not get past the specification step. If most of your metrics are easy, and a few really difficult, focus on the easy ones first. Again, the value of a good tool cannot be under-estimated. I generally have to specify a single line to add a new metric to my list - grant it, not something like computing function points, but more like the above cited metrics. If I had to write a program (or even worse, steal somebody's