Lean Metrics for Agile Software Configuration Management

[article]

If certain files are more prone to parallel development than others, it may be an indication of a "code smell" and perhaps some restructuring should be done to minimize dependencies and/or checkout contention and merge conflicts. (See several of the techniques described in John Lakos' book Large Scale C++ Design). If certain files are more frequently edited than others, than we may want to look closely at which of those files significantly impact build-time (for incremental/dirty builds). It too may be an indication that the file should be split-up and/or its code refactored, not necessarily because of checkout-contention, but to minimize rebuild time (which brings to mind more techniques from Lakos' book).

In general, these metrics would be used to help us find/regain our team's "sweet spot" to dynamically adjust the successful balance between integration frequency, commit-frequency, change granularity, build/test "completeness", and when and how often to run full and complete builds and test-suites. A few of them would also help raise a yellow flag about situations where refactoring (or process improvements) might help minimize merging effort and build-times.

(Im)Parting Words of Advice: Principles and Guidelines for Gathering CM Metrics
In this article, we introduced a lot of potentially new terms and concepts from Lean and Theory of Constraints and explored how they apply to SCM. Many possible measurements were proposed without much time devoted to which ones will prove most useful or when to attempt measuring them (you probably don't want to start with all of them at once). Giving such advice can be very perilous without being situationally-aware of the individual parameters and constraints of project and team. We instead attempt to give some concise "meta-advice" to those attempting this sort of thing.

Two almost diametrically opposed concerns for CM metrics in an agile environment are that:

  • On the one hand, Agile's insistence on very small/tight feedback loops with ultra-high frequency makes for a lot of fine-grained details to attempt to measure in order to gain insight into what's going on 
  • On the other hand, Agile's insistence on "people and interactions over process and tools makes it extremely challenging to collect such metrics transparently and unobtrusively.

So we need to find an appropriate balance between the value of the metrics and the cost of gathering them. After all, we don't want to create waste (muda) as a result of our metric process itself.

Value-Up! Agile metrics need to reward and reinforce the mental shift from the traditional “work-down” attitude to the newer “value-up” attitude espoused by Sam Guckenheimer in his book Software Engineering with MSVSTS. Primary measures promote the notion that:

Only deliverables that the customer values (working software, completed documentation, etc.) count.  You need to measure the flow of the work streams by managing queues that deliver customer value and treat all interim measures skeptically.

Measure Up! Don’t use metrics to measure individuals in a way that compares their performance to others or isolates the value of their contributions from the rest of the team. The last of the seven principles of Lean software development tells us to “Optimize across the whole.” When measuring value or performance, it is often better to measure at the next level-up. Look at the big-picture because the integrated whole is greater than the sum of its decomposition into parts. Metrics on individuals and subparts often create suboptimization of the whole, unless the metric connects to the “big picture” in a way that emphasizes the success of the whole over any one part. Mishkin Berteig writes the following regarding “Measure Up!”

In a single-team situation this means that individuals are measured and rewarded based on team performance (their sphere of influence). In a multi-team environment, that means that the group of teams should be measured as a group and compensated as a group. This will encourage all teams to work towards the success of the overall project. I personally believe there is some room for individual-based compensation, but the way it is handled needs to be done so that it does not encourage sub-optimal behavior.

Measure Wisely! Applying the principles of “measure up!” and “value-up” may sound great in theory, but it still doesn’t tell you what to measure. It may give some general advice, but is lacking in specific criteria. Deborah Hartmann and Robin Dymond wrote a paper on Appropriate Agile Measurement that gets into more specifics about criteria and checklists to use when selecting measurements for agile projects and teams.

About the author

Brad Appleton's picture Brad Appleton

Brad Appleton is a software CM/ALM solution architect and lean/agile development champion at a large telecommunications company. Currently he helps projects and teams adopt and apply lean/agile development and CM/ALM practices and tools. He is coauthor of the book Software Configuration Management Patterns, a columnist for the CMCrossroads and AgileConnection communities at Techwell.com,  and a former section editor for The C++ Report. You can read Brad's blog at blog.bradapp.net.

About the author

Steve Berczuk's picture Steve Berczuk

Steve Berczuk is a Principal Engineer and Scrum Master at Fitbit. The author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, he is a recognized expert in software configuration management and agile software development. Steve is passionate about helping teams work effectively to produce quality software. He has an M.S. in operations research from Stanford University and an S.B. in Electrical Engineering from MIT, and is a certified, practicing ScrumMaster. Contact Steve at steve@berczuk.com or visit berczuk.com and follow his blog at blog.berczuk.com.

About the author

Robert Cowham's picture Robert Cowham

Robert Cowham has long been interested in software configuration management while retaining the attitude of a generalist with experience and skills in many aspects of software development. A regular presenter at conferences, he authored the Agile SCM column within the CM Journal together with Brad Appleton and Steve Berczuk. His day job is as Services Director for Square Mile Systems whose main focus is on skills and techniques for infrastructure configuration management and DCIM (Data Center Infrastructure Management) - applying configuration management principles to hardware documentation and implementation as well as mapping ITIL services to the underlying layers.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!