Testing is Essential to Agile SCM

the criteria for approving a change to be that the change did what it was expected to do, and did not break any existing functionality. You can do this by having a codeline policy that requires that new code have:

  • New Unit Tests
  • Pass a workspace build. including unit tests
  • Pass an integration build (including all unit and integration tests)

Each of these rules can also be validated through an automated process; test coverage tools can allow you "test" whether new code has unit tests, for example, testing not only functional compliance (the code still works based on the tests) but process compliance (the metrics that we consider important are also met). Continuous Integration: Improving Software Quality and Reducing Risk has excellent practical advice on how to use your build process to measure various quality and policy metrics.

Another difference between this point of view and traditional SCM is that traditional SCM is often "event-based", focused on baselines, individual changes, etc. Lean/Agile SCM is focused on managing the flow of change across the value stream. In an event-based model the fact that a developer made a change is of primary interest, and our infrastructure is focused on tracking (and perhaps preventing) the developers from making a change. In the model we're discussing here the item of interest is the impact that the developer's change had on the system, and we can use criteria like code quality to initiate action, rollback changes, etc. The difference is one of priorities and focus. We care about tracking the various events, as they allow us to recover when things start going in the wrong direction. But we want to report on the impacts, not simply  the events.

Tests
Much has been written about the different kinds of tests: Functional Tests, Integration Tests, Unit Tests and who write them (developers, QA Engineers, etc), so we won't cover that here. It is important to understand that there are many places in the Software Development Ecosystem Timeline where testing happens and each has a impact on SCM.

  • During Coding: software developers write unit tests for any changes/additions that they are making, and frequently run the unit tests for other parts of the code to ensure that their change did not break anything. This might also be a good time to extend the functional test suite. When a developer feels that their work is ready to share, she updates her workspace, does a final build and runs the unit test suite.
  • Once code is submitted an automated integration build is run. This build might run all the unit tests as well as any integration tests.
  • Periodically (nightly or more frequently as possible) longer running automated regression or functional tests are fun.
  • As new featured appear on the scene, manual testing can happen for those features that do not yet have automated tests.

Roles
The "SCM is Testing" position also frustrates people because it seems to be placing an QA function (Testing) in the hands of another team (Release Engineering). To be able to successfully respond to change you need to forget about those boundaries, and think of SCM as being an element (perhaps a central element) of the software development environment.

As we mentioned above, there is a sense in some circles that testing is not relevant to the SCM community because testing is not part of build management or release engineering. The problem with this idea is that if SCM is about ensuring the integrity of the product, what other mechanisms do we have to do this? While we often speak in favor of cross functional teams

About the author

Brad Appleton's picture
Brad Appleton

Brad Appleton is a software CM/ALM solution architect and lean/agile development champion at a large telecommunications company. Currently he helps projects and teams adopt and apply lean/agile development and CM/ALM practices and tools. He is coauthor of the bookSoftware Configuration Management Patterns, a columnist in The CM Journal and The Agile Journal at CMCrossroads.com, and a former section editor for The C++ Report. You can read Brad's blog at blog.bradapp.net.

About the author

Steve Berczuk's picture
Steve Berczuk

Steve Berczuk is an engineer and ScrumMaster at Humedica where he's helping to build next-generation SaaS-based clinical informatics applications. The author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, he is a recognized expert in software configuration management and agile software development. Steve is passionate about helping teams work effectively to produce quality software. He has an M.S. in operations research from Stanford University and an S.B. in Electrical Engineering from MIT, and is a certified, practicing ScrumMaster. Contact Steve at steve@berczuk.com or visit berczuk.com and follow his blog at blog.berczuk.com.

About the author

Robert Cowham's picture
Robert Cowham

Robert Cowham has long been interested in software configuration management while retaining the attitude of a generalist with experience and skills in many aspects of software development. A regular presenter at conferences, he authored the Agile SCM column within the CM Journal together with Brad Appleton and Steve Berczuk. His day job is as Services Director for Square Mile Systems whose main focus is on skills and techniques for infrastructure configuration management and DCIM (Data Center Infrastructure Management) - applying configuration management principles to hardware documentation and implementation as well as mapping ITIL services to the underlying layers.