Agile Software Configuration Management: Communications and Documentation

The closer the feedback loops, the more accountability becomes shared between customers and development. And the more frequently it is shared, the more decisions become a matter of collaboration rather than negotiation: We can quickly try something, see the result, and then adjust/correct our speed and direction by working together on small micro-sized increments of functionality at a time.

While many of the terms we use to discuss Agile SCM come with "conversational" meanings, we need to understand the limits of the language and understand the best that we can do is to find a metaphor that starts us on a path to understanding. In as much as Agile SCM is based on acknowledging the limits of our abilities to control every aspect of development, learning about Agile SCM needs to be grounded in the understanding that words have limits.

Documentation - How much is too much?
One of the biggest lessons learned is with deciding how much is the appropriate amount of "textual" documentation required to effectively support an Agile SCM solution.  It's safe to say that every project differs in the amount of documentation it requires.  So what questions should we ask to help decide how much is the right amount?  Maybe the appropriate place to start is to define the basic reasons why we create "textual" documentation in the first place (from a software perspective).

• To consistently share knowledge across many people in a scaleable way: Although writing is certainly not the most effective form of communication, it does scale in that you can share knowledge across many people with relatively less effort than talking to those same people individually.
• Provide a persistent media from which to collaboratively build knowledge: As we work with others to build knowledge, it often helps to have the knowledge-to-date persist so that we can build on it.
• Make knowledge persist over time: There exists knowledge that is somewhat stable and offers value in a persistent form.

Agile and Lean methods suggest that since knowledge changes over time, we should delay decisions as late as possible and decide as "low" as possible. This translates into writing as few documents as possible as late as possible, and when required, use documents that are truly central to the actual point of the project, for example generating software.  Don't spend valuable resources writing documents that will never be read again, or become obsolete in a short period of time.

This seems to make sense, yet something still may not feel right with taking such a strong stance on documentation.  Maybe it's because of the "It's the way we've always done it" syndrome, or maybe it's because we're thinking of trying to apply a single principle to all projects.  This agile approach to documentation flies in the face of traditional formal traceability, something that's necessary to support large and complex software development efforts.  It's all about choosing the appropriate amount of traceability to satisfy the needs of your project.  Document when you have a REASON to document and understand the reason.

Whence Formal Traceability?
The mandate for formal traceability originated from the days of Department of Defense (DoD) development with very large systems that included both hardware and software, and encompassed many geographically dispersed teams collaborating together on different pieces of the whole system. The systems were often mission critical in that a typical "bug" might very likely result in catastrophic loss of some kind (loss of life, limb, livelihood, national security, or obscenely large sums of money/funding).

At a high level, the purpose of formal traceability was three-fold:

1. Aid project management by improving change Impact Analysis (to help estimate effort/cost, and assess risk).
2. Help ensure Product Conformance to requirements specs (i.e. ensure the design covers every requirement, the implementation realizes every design element and every requirement).
3. Help ensure Process Compliance (only the authorized individuals worked on the things [requirements, tasks, etc.] they were supposed to do).