If certain files are more prone to parallel development than others, it may be an indication of a "code smell" and perhaps some restructuring should be done to minimize dependencies and/or checkout contention and merge conflicts. (See several of the techniques described in John Lakos' book Large Scale C++ Design). If certain files are more frequently edited than others, than we may want to look closely at which of those files significantly impact build-time (for incremental/dirty builds). It too may be an indication that the file should be split-up and/or its code refactored, not necessarily because of checkout-contention, but to minimize rebuild time (which brings to mind more techniques from Lakos' book).
In general, these metrics would be used to help us find/regain our team's "sweet spot" to dynamically adjust the successful balance between integration frequency, commit-frequency, change granularity, build/test "completeness", and when and how often to run full and complete builds and test-suites. A few of them would also help raise a yellow flag about situations where refactoring (or process improvements) might help minimize merging effort and build-times.
(Im)Parting Words of Advice: Principles and Guidelines for Gathering CM Metrics
In this article, we introduced a lot of potentially new terms and concepts from Lean and Theory of Constraints and explored how they apply to SCM. Many possible measurements were proposed without much time devoted to which ones will prove most useful or when to attempt measuring them (you probably don't want to start with all of them at once). Giving such advice can be very perilous without being situationally-aware of the individual parameters and constraints of project and team. We instead attempt to give some concise "meta-advice" to those attempting this sort of thing.
Two almost diametrically opposed concerns for CM metrics in an agile environment are that:
- On the one hand, Agile's insistence on very small/tight feedback loops with ultra-high frequency makes for a lot of fine-grained details to attempt to measure in order to gain insight into what's going on
- On the other hand, Agile's insistence on "people and interactions over process and tools makes it extremely challenging to collect such metrics transparently and unobtrusively.
So we need to find an appropriate balance between the value of the metrics and the cost of gathering them. After all, we don't want to create waste (muda) as a result of our metric process itself.
Value-Up! Agile metrics need to reward and reinforce the mental shift from the traditional “work-down” attitude to the newer “value-up” attitude espoused by Sam Guckenheimer in his book Software Engineering with MSVSTS. Primary measures promote the notion that:
“Only deliverables that the customer values (working software, completed documentation, etc.) count. You need to measure the flow of the work streams by managing queues that deliver customer value and treat all interim measures skeptically.”
Measure Up! Don’t use metrics to measure individuals in a way that compares their performance to others or isolates the value of their contributions from the rest of the team. The last of the seven principles of Lean software development tells us to “Optimize across the whole.” When measuring value or performance, it is often better to measure at the next level-up. Look at the big-picture because the integrated whole is greater than the sum of its decomposition into parts. Metrics on individuals and subparts often create suboptimization of the whole, unless the metric connects to the “big picture” in a way that emphasizes the success of the whole over any one part. Mishkin Berteig writes the following regarding “Measure Up!”
“In a single-team situation this means that individuals are measured and rewarded based on team performance (their sphere of influence). In a multi-team environment, that means that the group of teams should be measured as a group and compensated as a group. This will encourage all teams to work towards the success of the overall project. I personally believe there is some room for individual-based compensation, but the way it is handled needs to be done so that it does not encourage sub-optimal behavior.”
Measure Wisely! Applying the principles of “measure up!” and “value-up” may sound great in theory, but it still doesn’t tell you what to measure. It may give some general advice, but is lacking in specific criteria. Deborah Hartmann and Robin Dymond wrote a paper on Appropriate Agile Measurement that gets into more specifics about criteria and checklists to use when selecting measurements for agile projects and teams.