Testing is Essential to Agile SCM

[article]
Summary:
Rather than being an afterthought for SCM, an appropriate testing strategy is what enables an SCM in an agile environment. To be more agile, you need to avoid the silo-based perspective of development, SCM, and testing being three different disciplines. Instead, think about how the processes in one part of your development ecosystem affects what you can do in the others.

Occasionally we hear the question, "What does testing have to with software configuration management, anyway?" Testing is essential for agile SCM environments and agile CM environments are built on testing.  To explain this idea, we need to talk about what SCM is and how testing helps.

A software configuration management system is in place to manage change. In a 1990 Software Engineering Institute Technical Report, Susan Dart said, “The goals of using CM are to ensure the integrity of a product and to make its evolution more manageable.”

The traditional approaches to software change management focus on the software lifecycle and identifying and controlling what changes happen in what context. Identification of the facts of changes is important.  What matters most to many stakeholders is less the fact of the change, but rather the impact of the change to them. People care that their system works as expected, that the functionality was added as desired and not removed accidentally. We want to ensure that the value of our configurations is increasing over time rather than decreasing! To verify these things you need to test the working application. In this sense, you can't fulfill the goals of SCM without testing.

As a system evolves, you want to know at what point risks increase. The more frequent and automated the testing, the easier it is to identify the point in time when a system is in a configuration that does not work.  This is where something like continuous integration comes in. In his recent book Continuous Integration: Improving Software Quality and Reducing Risk, Paul Duvall discusses how a CI system is a central part of the software quality lifecycle.  It allows testing not only for functionality, but also for various metrics.
We will now describe the trade-offs between and identification-based approach to SCM and a verification-based approach.



Stability and Progress



CM environments balance:

  • Stability:  How certain you can be that the code works at any given point
  • Progress:  How quickly a codeline evolves to encompass new features or fixes. How to set the balance varies depending on the needs of the project. Stability is always important since, without stability it's difficult, if not impossible, to make progress. Stable in this sense means that little changes, but by this definition, stable can become stale rather quickly. A more useful definition of stable is that the quality of the code is staying as good as it was. With this sense of stable positive change can happen.
One way to ensure stability is to add controls and processes to ensure quality, but it is easy to make the controls interfere with the business of an organization: delivering new functionality. The classic example of this is requiring changes go through extensive (manual) review before being worked on and then requiring extensive (and time-intensive) testing before making a change. The idea behind this is well meaning: software is complicated and we don't know the effect of a change, so let's be very cautious in making changes. These rules will improve stability, at least on the surface but at a great risk to your rate of progress. In our previous article,  The Illusion of Control we discuss that it is often better to have more visibility into effects of a change rather than attempt to guarantee that a change will be "safe." For more on how to emphasize transparency over traceability, see our 2007 September article Lean-Agile Traceability: Strategies and Solutions.

Another approach is to understand that it is extremely difficult to understand the effect of a change and, instead, change the criteria for approving a change to do what it was expected to do, and did not break any existing functionality. You can do this by having a codeline policy that requires that new code have:

  • New unit tests
  • Pass a workspace build, including unit tests
  • Pass an integration build, including all unit and integration tests

About the author

Brad Appleton's picture Brad Appleton

Brad Appleton is a software CM/ALM solution architect and lean/agile development champion at a large telecommunications company. Currently he helps projects and teams adopt and apply lean/agile development and CM/ALM practices and tools. He is coauthor of the book Software Configuration Management Patterns, a columnist for the CMCrossroads and AgileConnection communities at Techwell.com,  and a former section editor for The C++ Report. You can read Brad's blog at blog.bradapp.net.

About the author

Steve Berczuk's picture Steve Berczuk

Steve Berczuk is a Principal Engineer and Scrum Master at Fitbit. The author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, he is a recognized expert in software configuration management and agile software development. Steve is passionate about helping teams work effectively to produce quality software. He has an M.S. in operations research from Stanford University and an S.B. in Electrical Engineering from MIT, and is a certified, practicing ScrumMaster. Contact Steve at steve@berczuk.com or visit berczuk.com and follow his blog at blog.berczuk.com.

About the author

Robert Cowham's picture Robert Cowham

Robert Cowham has long been interested in software configuration management while retaining the attitude of a generalist with experience and skills in many aspects of software development. A regular presenter at conferences, he authored the Agile SCM column within the CM Journal together with Brad Appleton and Steve Berczuk. His day job is as Services Director for Square Mile Systems whose main focus is on skills and techniques for infrastructure configuration management and DCIM (Data Center Infrastructure Management) - applying configuration management principles to hardware documentation and implementation as well as mapping ITIL services to the underlying layers.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!