Agile SCM and Requirements
At the start of our last article on the applicability of some principles of martial arts we talked about the importance of requirements when deciding on a martial art, so it seems fitting this month to look at the implications of requirements for agile SCM.
The process of eliciting requirements (the phrase "requirements gathering" rather brings to mind wandering around an alpine meadow with a basket under your arm plucking one juicy requirement after another while singing along with Julie Andrews!) is a specialist area with many of its own techniques and practices.
Configuration Management and Requirements
From a configuration management point of view, what is important about the requirements (and analysis) phases?
- Change Control - tracking and verifying changes to the requirements themselves for the purposes of reporting on status. Also change control for the whole development in terms of controlling changes to requirements and code.
- Baselining of Requirements - see below.
- Traceability and Auditability - linking code and other changes to the requirements for them, becoming increasingly more important in a Sarbanes Oxley (and related corporate governance regulations) world. Also verifying that the changes that have been implemented are valid and necessary (no back doors introduced!).
Requirements and Change
From an agile standpoint we are going to take an iterative development approach as a given. Note also, that the very process of releasing an interation means that we have a complete working system, thus ensuring that issues around system integration, or build and installation/release are not allowed to fester in a neglected corner, coming back to bite us at the most inopportune moment.
A recent discussion in the blogosphere (How Two Hours Can Waste Two Weeks with responses You Call This Agile? shows an informative level of discussion taking place helping refine the thinking needed to apply agile methods and not get blind-sided by pure theory.
Baselining Requirements
In classical configuration management, the purpose of a "functional configuration audit" is to ensure that the "baselined" software version correctly+completely+consistently satisfies the corresponding "baselined" requirements
In the above diagram, tests (test cases and executable tests) are also baselined, and refer to the appropriate baselined requirements which they are testing.
In some agile methods, automated unit and higher level tests provide a level of "executable requirements spec" which minimizes intermediate artifacts, and makes things easier to maintain and keep consistent (satisfying lean development principles).
Reducing Complexity During Development
At an even higher level, we are thus talking about communication and collaboration with our customers, management and others. From an agile point of view we wish to make this as easy as possible and with minimal extra effort. This will also satisfy the "Lean" principles are of minimizing intermediate artifacts and minimizing redundancy and dependency, and non-essential artifacts. This will start enabling us to achieve trustworthy transparency.
How can we make it easier to track what is going on during development and to be able to control and communicate (report on) the status of all our configuration items?
The easiest way to do this is to reduce the number of items we need to track. This includes not only the CIs in our system, but also to reduce the number of changes to those CIs in any given iteration. At first glance this may seem just not to be possible - we can't unilaterally just reduce the number of CIs in the system!
But there are of course very standard ways of making information easier to grasp by grouping it, and compartmentalizing it.
Some approaches:
- Component based development (when done well) can reduce the complexity of the development problem by "chunking up" functionality into higher level components. Laura Wingerd has a nice analogy: "it is easier to organise a coach load of 40 adults, each of whom has responsibility for their own packing and turns up at the appropriate time, than a car load of 4 children for whom you have to do all the packing yourself". Thus we aren't necessarily reducing the number of items to track, but we are reducing the degree of granularity at which tracking is necessary. This is not feasible without also adhering to sound principles of modularity (factored-ness) and coupling and cohesion and encapsulation.
- Reduce No of Changes in progress - see below.
- Reducing Number of Changes in Progress (Cognitive Overload) This relates to:
- working in smaller (consistent) chunks and integrating more frequently.
- consistent configurations (such that a codeline increases in value over time and doesn't regress)
- reducing WIP (work in progress, or inventory in lean terminology)
- Keep mainline buildable and with known status
- Use unit testing and other testing frameworks to ensure quality and state
A traditional method for marking status in a code line is to use labels to label a set of files and particular revisions of those files. This can then indicate which requirement they implement and what state that implementation is at.
Labels vs Change Sets. In this diagram, there are a number of individual files in a codeline, each with their own revisions. The revisions are checked in change set by change set, and yet the user has chosen to use labels to record known configurations, rather than referring to specific change sets.
In the above example, using labels which "cut across" change sets means that there are many more things to keep track of, and it is very unclear what the status of the codeline at any particular change set is. If we can train our developers to check in logical and consistent change sets, then it is generally much easier to keep track of the status of the codeline. We hope that our codeline is increasing in value, change set by change set. Each change set follows standard OOD principles: coupling, cohesion and encapsulation.
Obviously in real life, mistakes are made, but simple practices such as checking in unit tests with any changed code, and ensuring that all unit tests for the whole system run successfully before you commit your change set, make this achievable.
For those occasions where you wish to commit a partial implementation of a feature, which might either destabilize the mainline, or not increase its value, then use of a Task Branch will allow you to make those changes and integrate them back to the mainline in a single consistent change set.
The Pros and Cons of Refactoring
Refactoring is a very useful set of techniques for restructuring an existing body of code, altering its internal structure without changing its external behavior. It is aimed at making the code more easily understandable and easier to modify (respond to change) without introducing errors and unintended side effects.
Refactoring will often reduce the number of code modules in a system, reducing redundancy, so that a change has to be made in fewer places (sometimes it will increase the number but only if there are other benefits such as use of a well known pattern or to make it more easily understandable). This has the benefit of reducing the number of CIs in the system.
The problem with refactoring from a change control perspective is that it is in itself a change to the system (which needs to be controlled and reported on). Refactoring is not explicitly to address requirements, but it is vital for overall health of the system, and to ensure future ability to respond to change.
Thus we need to find the right balance between desirable refactoring and less desirable - experience is the key guide here, and fortunately there are a lot of positive experiences available. Simple guidelines such as not mixing refactoring changes in the same change set as a feature implementation or a bug fix help significantly.
Conclusion
Requirements are vital for effective software development - ensuring that when we climb our ladder we find it is leaning against the right wall! Iterative development helps significantly to address this by ensuring improved feedback. And the other agile processes all help to contribute to this.
By tweaking slightly the traditional SCM principles which are so valuable, and applying agile and lean thinking, we can be even more effective in our SCM.
