The Illusion of Control in Software Configuration Management

[article]
Summary:
We explore why it is important to understand the context of agile techniques when you are trying to build a more agile software configuration environment and how people can fool themselves into thinking that their non-agile environment has more control.

High performance development environments arise out of a group of people using approaches in the correct context. What seems to be a good solution in one environment will cause pain in another because some of the elements that made the solution a success in one situation were missing in the second. This is what patterns and pattern languages are all about. This month we explore why it is important to understand the context of Agile techniques when you are trying to build a more agile SCM environment, and how people can fool themselves into thinking that their non-agile environment has more control.

Control
From time to time I give talks to groups of software developers at various companies about Software Configuration Management Patterns and Agile SCM. Two of the most common frustrations that people express about their development process are:

    • Instability in the codeline: The latest code does not compile, or pass tests
    • Very long pre-check-in processes. Either there is a long series of hurdles to check in code such as tests or reviews, or the team uses an Integration Token in a way that means that there is a long wait to do any check-in (or both)

What is amusing is that you often hear about both of these issues from people in the same organization at the same time! The integration token or extensive test suite was started as a way to address the code instability problem, yet the end result is that neither output nor quality improves in any real way.

The heart of the problem here is a classic software development problem: that Integration is difficult. Slowing down the check-in process gives the illusion of control, but that is only a short term fix.

What often happens is that after a being frustrated by a long line of broken builds due to testing and check-in controls that are too loose, the team swings the process the other way towards heavy process and control applying the theory that slower is safer. The problem is that the slower process still permits breaks from time to time, while encouraging larger integration intervals. These larger integration intervals break the team’s rhythm and also increase the risk that there will be problems come integration time.

A Story
Here is a common situation in many organizations. After a series of dark and stormy release cycles during which the release is late, and has poor quality, someone in the team decides that the way to fix the chaos is to impose more control on the development process. If the problem is poor quality, the answer is to do more testing. The proposed answer is rarely “to test more effectively” except in the sense that “more” is often viewed as “better.” The rule comes down that before check-ing code in, developers need to run a long (or very long) suite of tests.

The quality of the codeline does improve for a while, and developers are happy that they are spending less time working with broken code. Velocity increases, and developers start adding features. There is more change in the codeline, and this change is good.

Then strange things start happening. Otto makes some changes, runs the test suite, which by now takes 90 minutes to run. Actually he makes a lot of changes since it takes a while to run the tests, and he wants to batch as many changes into his check in as possible. Unknown to Otto, Sue has also started making some changes. Both Otto’s and Sue’s test pass and they check in their code. Brad updates his workspace from the codeline and discovers that the code does not compile, much less pass tests. What has happened is that Otto and Sue made incompatible changes. Because the tests took so long to run, they made lots of changes, increasing the chances that the changes would conflict. Figure 1 illustrates this.

Pages

About the author

Steve Berczuk's picture Steve Berczuk

Steve Berczuk is a Principal Engineer and Scrum Master at Fitbit. The author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, he is a recognized expert in software configuration management and agile software development. Steve is passionate about helping teams work effectively to produce quality software. He has an M.S. in operations research from Stanford University and an S.B. in Electrical Engineering from MIT, and is a certified, practicing ScrumMaster. Contact Steve at steve@berczuk.com or visit berczuk.com and follow his blog at blog.berczuk.com.

About the author

Brad Appleton's picture Brad Appleton

Brad Appleton is a software CM/ALM solution architect and lean/agile development champion at a large telecommunications company. Currently he helps projects and teams adopt and apply lean/agile development and CM/ALM practices and tools. He is coauthor of the book Software Configuration Management Patterns, a columnist for the CMCrossroads and AgileConnection communities at Techwell.com,  and a former section editor for The C++ Report. You can read Brad's blog at blog.bradapp.net.

About the author

Robert Cowham's picture Robert Cowham

Robert Cowham has long been interested in software configuration management while retaining the attitude of a generalist with experience and skills in many aspects of software development. A regular presenter at conferences, he authored the Agile SCM column within the CM Journal together with Brad Appleton and Steve Berczuk. His day job is as Services Director for Square Mile Systems whose main focus is on skills and techniques for infrastructure configuration management and DCIM (Data Center Infrastructure Management) - applying configuration management principles to hardware documentation and implementation as well as mapping ITIL services to the underlying layers.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Nov 09
Nov 09
Apr 13
May 03