In his CM: the Next Generation series, Joe Farah gives us a glimpse into the trends that CM experts will need to tackle and master based upon industry trends and future technology challenges.
I find it rare to see a project with too many metrics. Quite the opposite is usually the case. The problem is that establishing metrics is a lot more work than identifying what you want to measure. It's hard to collect the data. It's hard to mine the data, and then to present the comparisons and trends which are the metrics is no simple feat.
However, as CM tools move more and more into the realm of ALM and beyond, we'll see some changes. As integration across the lifecycle becomes more seamless, the focus will move to dashboards that not only provide standard metrics, but which are easily customized to mine and present the data necessary.
The goal of software metrics is to have a rich collection of data and an easy way of mining the data to establish the metrics for those measures deemed important to process, team and product improvement. When you measure something and publish the measurement regularly, improvement happens. This is because a focus is brought on the public results.
However the interpretation of the measure must be clear. For example, are glaciers falling into the ocean a sign of global warming (weakening and falling) or global cooling (expanding ice pushing the edges into the ocean)? A wrong interpretation can be very costly. So a good rule is to understand how to interpret a measure before publishing it.
Looking at the software world, is the number of lines of code generated per month a reasonable measure? I want my team to create a given set of functionality using the minimal number of lines of code. So larger numbers are not necessarily a good thing and may be a bad thing. Lines of code generated is not a good measure for productivity. However, averaged over a project, and compared from project to project (using similar programming technology), it is a fair measure of project size and is also useful for overall "bugs per KLOC" metrics.
The point here is that you must select metrics that can clearly be interpreted. And when you publish these regularly, the team will tend toward improving the metric because they understand the interpretation and how it ties back to what they're doing.
Creating Effective Metrics - What's Needed
I don't intend to go into detail about which metrics I find useful and which I don't. Instead I want to focus on how to build an infrastructure so that you can easily generate metrics and test out their effect on your project. The idea is that if it is easy to create metrics, they will be used to
improve your process, team and product.
So what do we need?
1. A Single Next Generation Repository Across the Life Cycle
I'm sure there are a lot of you that can balance multiple repositories and do magic to get data from one to another, knitting multiple data sources together into a fountain of data. But no matter how you cut it, there's no replacement for a single repository that can hold the data for your entire application life cycle.
I don't really want to mix transient data, such as compile results, with persistent data, such as management information. But I want all of the persistent information in one place, easily accessible. I want the schema for this data under control of a small team or a single person who understands that the entire team needs all of the data, and who has a thorough understanding of the ALM processes.
I don't want to jump through security hoops as I hop from one database to another. I don't want to put up with server problems and network issues because my data is distributed through multiple repositories. I don't want to learn different access methods for each application. I want a single uniform way to access my data. Maybe it's relational. Even better is a next generation hybrid repository which provides both relational and data networking so that I don't have to map from the real world model to the relational and then back again to retrieve the data. And, this being a CM world, if it support data revisions (not just source code revisions), all the better, with perhaps some advanced large object handing for things such as source code, problem descriptions and