Trustworthy Transparency over Tiresome Traceability
Many in the agile community, especially the eXtreme Programming community, see red whenever they encounter that maddening “T-word”: traceability. Almost instantaneous distrust sets in against those who would dare to utter it, much less recommend it. On the other side of the fence, we have many agile skeptics who misunderstand (or have all too often seen misapplied) the Agile Manifesto's tenet of "working software over comprehensive document" to mean "We're Agile! We don't do documentation!"
The position of this month's article on traceability is more "lean" than "agile." We base this on the XP & Scrum centric views that were expressed in the March, 2004 YahooGroup discussion thread Why Traceability? Can it be Agile? The "tests over traceability" discussion is probably a valid summary of the XP/Scrum perspective from that thread.
From his recent stint at Microsoft, David Anderson would probably say something more along the lines of "transparency over traceability", where we acknowledge the important goals that traceability is trying to fulfill, but don't necessarily accept many of the traditional ways of trying to attain it. David, in particular, has written about "trustworthy transparency" and "naked projects". These are projects that are so transparent and visible in their status/accounting that they seem "naked". Here is an excerpt from David on the subject of Changing the Software Engineering Culture with Trustworthy Transparency:
"Modern software tooling innovation allows the tracking of work performed by engineers and transparent reporting of that work in various formats suitable for everything from day-to-day management and team organization to monthly and quarterly senior executive reporting. Modern work item tracking is coupled to version control systems and aware of analysis, design, coding and testing transitions. This makes it not only transparent but trustworthy. Not only can a tool tell you the health of a project based on the state of completion of every work item, but this information is reliable and trustworthy because it is tightly coupled to the system of software engineering and the artifacts produced by it. The age of trustworthy transparency in software engineering is upon us. Trustworthy transparency changes the culture in an organization and enables change that unleashes significant gains in productivity and initial quality. However, transparency and managing based on objective study of reality strains existing software engineering culture as all the old rules, obfuscation, economies of truth, wishful thinking and subjective decision making must be cast aside. What can you expect, how will you cope and how can you harness the power of trustworthy transparency in your organization?"
In the past we have written about The Trouble with Tracing: Traceability Dissected where we described:
- The ten "commandments" of traceability
- Nine common complaints about traceability
- Eight reasons for traceability
- The seven functions of SCM
- The six "facets" of traceability
- The five orders of traceability
- The four "rings" of enterprise/stakeholder visibility
- The three driving forces for traceability
- The two overarching objectives of traceability
- One ultimate mission/vision: fostering trust
To make a long (but hopefully interesting) story short, we concluded that the overreaching objectives of traceability are:
- Transparency: the ability to readily view all the information we are concerned with
- Identification: the ability to identify our concerns so we can separate independent sets of concerns and cohesively associate the related ones.
These are supposed to be ideals that help to engender trust. Can we achieve transparency and identification (and hence "navigable knowledge-spaces") without more traditional traceability methods? If so, what are the different ways of doing it?
Our CM backgrounds strongly differ from the many vocal opinions expressed in XP community when it comes to the use of tools for tracking requests/changes. We are strongly in favor of using a "good" tracking tool. Index cards are a great and valuable "tool" for eliciting dialogue and interaction with the "customer" and some of us even use them for this purpose, along with post-it notes. From a CM-perspective, though, index cards alone simply do not suffice as a serious means of storing, tracking, sorting, searching, slicing & dicing development change requests. We do believe an extent of traceability is necessary, and that it's not necessarily "agile," but that it can be, and should be, "lean" and streamlined. It should also serve the purpose of transparency, visibility and status-accounting rather than being a goal in itself.