on short term memory, etc., we need to address the sociological aspects of our profession rather than mechanical or chemical aspects. How many of us work an entire eight-hour day without interruptions? How often do you start a one-hour task in the morning and find that at day's end you have not finished it--and the next day, as you start over, you've forgotten a critical aspect of the program that was your next task the day before.
Item six has multiple consequences. To be effective in software development, the real root cause usually requires the person making the mistake to be involved in the analysis. If there is fear of retribution (scapegoating), the incentive to identify the root cause is eliminated. The second issue is the time relationship between discovery of the root cause and the chance to prevent the problem in the next development cycle.
I came across an organization that was doing RCA on production failures with the expectation of significantly improving quality. Their typical release schedules were twelve to eighteen months. This meant problems found in the requirements or early design activities and eliminated from the next release cycle would have a twelve- to eighteen-month delay before showing up as improved quality in the next release--not what they wanted. To be effective, the time delay between the error introduction, discovery, root cause analysis, repeat of the activity that introduces the error, and impact on the next development or production cycle should be as short as possible.
The third issue with item six is "what to do with the result" of the RCA. For major catastrophes, finding the scapegoat is often the real reason behind RCA, as lawyers and victims get in line for compensation. In software, we are looking to prevent future occurrences, which means we need to change something: process, development environment, work environment, etc. Change means some effort or cost will be incurred to make the change. If this cost is not budgeted, how will it happen? All too often this falls into the "now a miracle occurs" part of the plan--or lack of plan--for preventive action. Whether the preventive action is as simple as a checklist update or complex as changing the development environment or process, allocating some budget for this endeavor is mandatory. Telling your development teams to do something in zero time or at zero cost sends a message that the activity isn't worth much.
To effectively apply RCA in your organization:
- Make RCA a formal, budgeted activity
- Avoid scapegoating
- Involve the people who made the original error
- Do RCA on groups of failures, looking for common causes
- Understand both temporal delay and causality (linkage of cause to significance of the failure)
Typically we use the terms error, fault, defect, and failure in sequence to correctly describe what happens. A person makes an error which introduces a fault in the product. This fault results in a defect when the program executes, which may or may not result in a failure visible to the user. Casual writing quite often uses the terms bug or defect to mean all four. Usually context will tell you which is which, and, hopefully, I (hopefully) was consistent in this article.