Establishing Effective Software Metrics for the Measures You Want


In his CM: the Next Generation series, Joe Farah gives us a glimpse into the trends that CM experts will need to tackle and master based upon industry trends and future technology challenges.

Depending on my role in the project, I may want a specifically designed dashboard. If my goal is to monitor product quality perhaps I want to look at metrics such as:


·         The number of problems introduced per 100 changes within a product development stream

·         The number of problems raised each week for a given product development stream

·         Test suite completion and success rates over time for a given product development stream

If I'm a project manager, my mix will be different. The point is that I don't want a few pre-canned metrics; I want the ability to easily define which metrics are necessary for me at any one time. Ideally I can sit down and invent a metric on the fly, interactively, as a result of a trend that I've noticed over the past few weeks and I can turn it on or off as necessary. Metrics are useful for evaluating product quality. They are also useful for evaluating process quality. What should happen when we improve the process in this way? Let's measure and make sure it does happen.

Metrics are also useful for evaluating teams. This is a scary one. Nobody wants their weaknesses pointed out. "Sure we have twice as many bugs, but we do three times as much work." How do you establish fair team metrics? This is where interpretation is tricky. Do you want more changes from a team member or fewer - what does it indicate? More functionality completed? Additional rework frequently happening? Incremental feature development versus big bang? A lot of the interpretation will depend on your process guidelines and procedures. A lot will simply reflect the different work habits of your team members.

So perhaps you want to overload metrics here and over time prune out the ones that aren't really useful or yield ambiguous interpretations. When your metrics can be easily specified and collected into a dashboard or two, you'll start asking better questions and hopefully will improve your decision making. Maybe it will be as simple as noticing that 20% of the staff disappear at March break so that better be accounted for in the planning phase.

4. CM processes and applications that can reliably collect data:remember that one of the key problems in establishing metrics is interpretation. For good interpretation, you need to collect meaningful and reliable

About the author

Joe Farah's picture Joe Farah

Joe Farah is the President and CEO of Neuma Technology and is a regular contributor to the CM Journal. Prior to co-founding Neuma in 1990 and directing the development of CM+, Joe was Director of Software Architecture and Technology at Mitel, and in the 1970s a Development Manager at Nortel (Bell-Northern Research) where he developed the Program Library System (PLS) still heavily in use by Nortel's largest projects. A software developer since the late 1960s, Joe holds a B.A.Sc. degree in Engineering Science from the University of Toronto. You can contact Joe at

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!