Some CM practitioners object to placing an emphasis on testing when discussing software configuration management, believing that testing is the domain of a quality assurance organization and developers. Such a perspective, while traditional, is flawed, as the goals of the QA, SCM, and development processes are all closely connected. QA engineers are concerned with identifying what they are testing and being able to reproduce results. As SCM engineers, we want to be able to verify that configurations are valid.
Testing is essential for agile configuration environments and, perhaps, for any configuration management environment.
Best Practices, Attitudes and Principles
We will start with some thoughts on last month's theme of best practices, as we consider testing to be one.
Many people think the term “best practice” means one specific practice is better than all the rest for a particular problem. We believe that it was intended to mean that it is one of many best practice solutions, even for a given problem. This also involves the things that repeatedly prove themselves to work better than many initial or naive solutions. This doesn't mean that each one is the absolute best at what it does, but rather that each is proven to be the best among the other options that had been tried at the time.
Agility and Feedback
Agile software development practices are based on feedback and adjustment. Testing is one way of providing feedback, and the agile practice of continuous integration (CI), supported by testing is a key feedback mechanism. Paul Duvall's book, Continuous Integration, explains that your CI build can not only tell you if the software compiles, but can also give you test results, metrics on, among other things, test coverage, and serve as a window into the overall quality of the system. The right kinds of tests enable those using the SCM process to assign some sort of quality metric to a configuration. The trick is figuring out what the right kinds of tests are.
Attitude and Principles
An important point is the attitude and way in which we go about our jobs of developing software or managing configurations.
Continuous improvement (or “kaizen” from in Japanese) is an elegant expression of some powerful ideas (from http://en.wikipedia.org/wiki/Kaizen) :
To be most effective kaizen must operate with three principles in place:
- Consider both the process and the results so that actions to achieve effects are made apparent;
- Systemic thinking of the whole process and not just what is immediately in view (i.e., big picture, not solely the narrow view) in order to avoid creating problems elsewhere in the process; and
- A learning, non-judgmental, non-blaming approach and intent will allow the re-examination of the assumptions that resulted in the current process.
While kaizen (often associated with the Toyota Production System) usually delivers small improvements, the culture of continual aligned small improvements and standardization yields large results in the form of compound productivity improvement. There is a need to be aware of what we are doing and the effects of it, which brings a different quality of approach, and allows us to question current practices and seek to improve them.
This attitude and approach will lead us both to consider and adopt appropriate patterns and also to implement them in such a way that we can measure and improve (or even discard them if they turn out to be inappropriate for our particular requirements).