The Next Generation: Software Metrics

The CM process will demand traceability - from the requirements, through the tasks, to the software changes, to the builds and onto verification.  Your CM tools must be able to handle these processes and must be able to collect data in a reliable manner. 
If your CM tools/processes dictate that you should not check in your source code until the build manager requests your changes, then it's no use trying to use the CM tool to measure how long it takes to complete a feature implementation.  The completed code could be sitting on somebody's personal disk for weeks.  If your CM tool is a file-based
tool as opposed to a change-based tool, don't expect it to give you any metrics with respect to logical changes. If your data is stored within the version control file, don't expect to be able get at it easily.

If your processes don't require referencing an approved feature or accepted problem report as part of starting a new software change, expect a lot of variation on the traceability metrics.  If it's not easy to search for potentially duplicate problems in your repository,
expect a lot of duplicate problem reports, which could further skew your data.

It's one thing to say process first then find the right tool, but perhaps its better to say give me a tool that will both do the job and help me to define and continually refine my process so that the process decisions are enforced as they are made.  Better yet, give me that same tool, but with a strong process supported out of the box.  And for certain, customization must be easy to do so that the experts who want the metrics don't have to run to a technical resource to have them supported.

Metrics - What's Useful, What's Overhead
So when I'm developing metrics for a project, what's useful and what's overhead?  The answer depends on numerous variables which are constantly changing.  Define the metrics you might find useful.  Create a large list if you want.  Then turn the presentation of them on and off as required.  Group them into dashboards according to role.  Add to
the list as you discover additional problems or trends.

Make sure that the data you're using is reliable.  Make sure that the processes support gathering good data.  Make sure that the interpretation of the data is unambiguous.  Make the important metrics widely visible, especially if the team members performance can affect the results.  Even consider competitive metrics that pit one part of the team or one project against another.

Good process, good tools, good presentation. And meaningful metrics. These will make your metrics useful.

About the author

Joe Farah's picture
Joe Farah

Joe Farah is the President and CEO of Neuma Technology and is a regular contributor to the CM Journal. Prior to co-founding Neuma in 1990 and directing the development of CM+, Joe was Director of Software Architecture and Technology at Mitel, and in the 1970s a Development Manager at Nortel (Bell-Northern Research) where he developed the Program Library System (PLS) still heavily in use by Nortel's largest projects. A software developer since the late 1960s, Joe holds a B.A.Sc. degree in Engineering Science from the University of Toronto. You can contact Joe at farah@neuma.com