major influence on risk. For example, Raytheon has achieved zero deviation from plans and budgets over several years. They used a $1million/year (for 1,000 software engineers) for 8 years to do continuous software process improvement. They report that the return on this investment was $7.70 per $1 invested on improving processes such as requirements, testing and Inspection itself. Their software defect rate went down by a factor of three [DION95].
Using Inspection, analysis of the identified defects to find process improvements is carried out in the Defect Prevention Process (DPP). DPP was developed from 1983 at IBM by Robert Mays and Carole Jones and, is today recognized as the basis for SEI CMM Level Five. The breakthrough in getting DPP to work, compared to earlier failed efforts within IBM, was probably in the decentralization of analysis activity to many smaller groups, rather than one 'Lab Wide' effort by a Quality Manager. This follows what the quality Guru Dr. W Edwards Deming taught the Japanese; factory workers must analyze their own statistics and be empowered to improve their own work processes.
Analysis of 'root causes' of defects is very much a risk analysis effort [HALL98] and a handful of my clients are reporting success at doing so. But, most are still working on other disciplines like Inspection and others elsewhere in this paper.
Principle 9. Delegate personal responsibility for risk
People must be give personal responsibility in their sector for identification and mitigation of risks.
To back up communicating about risk, people must be given ownership of the risks in their sector (e.g. allocating ownership/sign off of IE tables and giving people specific roles within Inspections).
Principle 10. Contract out risk
Make vendors contractually responsible for risks, they will give you better advice and services as a result.
I would like to point out that contracting for products and services gives great opportunity to legally and financially control risks by squarely putting them on someone else's shoulders.
The effect of contracting a risk to someone else is that:
- you have gotten rid of the risk in some senses, but if they fail, you will still be affected!
- the supplier (assuming they get the risk) will be more motivated to take steps to eliminate the risks,
- be motivated to tell you exactly what you have to do the avoid being hit by risks,
- might come up with a more realistic bid and time plan to cope with the risks.
Risks can be handled in many ways and at many levels. I have tried to point out some risk management methods which are not so well known or well treated in existing literature. Hopefully, the need to fully integrate risk management into all the development and maintenance processes is clear.
Table 6 and Table 7 recap the ideas presented in this paper. Table 6 is a set of policies for risk management [See GILB98 for more detail]. Table 7 contains 'Twelve Tough Questions' to ask when assessing risk.
TABLE 6: Policy Ideas for Risk Management
EXPLICIT RISK SPECIFICATION
All managers/planners/engineers/testers/quality assurance people shall
immediately in writing, integrated in the main plan, specify any uncertainty, and any special conditions which can imaginably lead to a risk of deviation from defined target levels of system performance.
NUMERIC EXPECTATION SPECIFICATION
The expected levels of all quality and cost attributes of the system shall be specified in a numeric way, using defined scales of measure, and at least an outline of one or more appropriate 'Meters' (test or measuring instruments for determining where we are on a scale).
The requirements levels shall be qualified