Testing's Role in the Software Configuration Management Process

In this article, the authors cover how testing can support any SCM environment and how testing can be part of the SCM process. They outline at patterns, roles, automation, and the never-ending search for simplicity.

Some CM practitioners object to placing an emphasis on testing when discussing software configuration management, believing that testing is the domain of a quality assurance organization and developers. Such a perspective, while traditional, is flawed, as the goals of the QA, SCM, and development processes are all closely connected. QA engineers are concerned with identifying what they are testing and being able to reproduce results. As SCM engineers, we want to be able to verify that configurations are valid.

Testing is essential for agile configuration environments and, perhaps, for any configuration management environment.

Best Practices, Attitudes and Principles 

We will start with some thoughts on last month's theme of best practices, as we consider testing to be one.

Many people think the term “best practice” means one specific practice is better than all the rest for a particular problem. We believe that it was intended to mean that it is one of many best practice solutions, even for a given problem. This also involves the things that repeatedly prove themselves to work better than many initial or naive solutions. This doesn't mean that each one is the absolute best at what it does, but rather that each is proven to be the best among the other options that had been tried at the time.

Agility and Feedback

Agile software development practices are based on feedback and adjustment. Testing is one way of providing feedback, and the agile practice of continuous integration (CI), supported by testing is a key feedback mechanism. Paul Duvall's book, Continuous Integration, explains that your CI build can not only tell you if the software compiles, but can also give you test results, metrics on, among other things, test coverage, and serve as a window into the overall quality of the system.  The right kinds of tests enable those using the SCM process to assign some sort of quality metric to a configuration. The trick is figuring out what the right kinds of tests are.

Attitude and Principles 

An important point is the attitude and way in which we go about our jobs of developing software or managing configurations.

Continuous improvement (or “kaizen” from in Japanese) is an elegant expression of some powerful ideas (from http://en.wikipedia.org/wiki/Kaizen) :

To be most effective kaizen must operate with three principles in place:

  • Consider both the process and the results so that actions to achieve effects are made apparent;
  • Systemic thinking of the whole process and not just what is immediately in view (i.e., big picture, not solely the narrow view) in order to avoid creating problems elsewhere in the process; and
  • A learning, non-judgmental, non-blaming approach and intent will allow the re-examination of the assumptions that resulted in the current process.

While kaizen (often associated with the Toyota Production System) usually delivers small improvements, the culture of continual aligned small improvements and standardization yields large results in the form of compound productivity improvement. There is a need to be aware of what we are doing and the effects of it, which brings a different quality of approach, and allows us to question current practices and seek to improve them.

This attitude and approach will lead us both to consider and adopt appropriate patterns and also to implement them in such a way that we can measure and improve (or even discard them if they turn out to be inappropriate for our particular requirements).

About the author

Brad Appleton's picture Brad Appleton

Brad Appleton is a software CM/ALM solution architect and lean/agile development champion at a large telecommunications company. Currently he helps projects and teams adopt and apply lean/agile development and CM/ALM practices and tools. He is coauthor of the book Software Configuration Management Patterns, a columnist for the CMCrossroads and AgileConnection communities at Techwell.com,  and a former section editor for The C++ Report. You can read Brad's blog at blog.bradapp.net.

About the author

Steve Berczuk's picture Steve Berczuk

Steve Berczuk is a Principal Engineer and Scrum Master at Fitbit. The author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, he is a recognized expert in software configuration management and agile software development. Steve is passionate about helping teams work effectively to produce quality software. He has an M.S. in operations research from Stanford University and an S.B. in Electrical Engineering from MIT, and is a certified, practicing ScrumMaster. Contact Steve at steve@berczuk.com or visit berczuk.com and follow his blog at blog.berczuk.com.

About the author

Robert Cowham's picture Robert Cowham

Robert Cowham has long been interested in software configuration management while retaining the attitude of a generalist with experience and skills in many aspects of software development. A regular presenter at conferences, he authored the Agile SCM column within the CM Journal together with Brad Appleton and Steve Berczuk. His day job is as Services Director for Square Mile Systems whose main focus is on skills and techniques for infrastructure configuration management and DCIM (Data Center Infrastructure Management) - applying configuration management principles to hardware documentation and implementation as well as mapping ITIL services to the underlying layers.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!