Without return on investment (ROI) calculations for the software inspections process, you cannot know the true benefit of those inspections. In this article, Ed Weller makes some assumptions about the cost of inspections and tries to estimate the savings from reduced test cost. He also provides a spreadsheet for doing "what-if" analysis of different savings based on inspection effectiveness, and how much defect removal in test might cost.
As we near the thirtieth anniversary of the development of the inspection process by Michael Fagan at IBM, I continue to be amazed at the number of software development organizations that do not use this powerful method to improve quality and productivity.
I recently had a discussion with someone about Return on Investment (ROI) for the process. He said they did not do ROI calculations, as they knew they were getting a good benefit from requirements and design inspections. He then said, "We do not do code inspection as they take too much time." In the absence of any ROI calculations, how would he know?
Let's take a look at this situation and explore some relatively easy ways to evaluate either potential or actual benefits from inspections. It really does not take a large amount of data, and the cost to do the analysis is minimal. For the potential benefit, we will make some assumptions about the cost of inspections and try to estimate the savings from reduced test cost. I'll also provide a spreadsheet for doing "what-if" analysis of various different savings based on inspection effectiveness, and how much defect removal in test might cost.
The inspection process itself will not be explained in any detail, as it has been explained in many books and publications (see a short list at the end of the article).
The savings can be calculated as follows:
Savings = (Original Test Cost) - (New Test Cost) - (Cost of Inspection)
How each of these is derived is shown below.
If you are not already performing inspections and therefore do not have your own data, the following rules of thumb can be used:
One thousand lines of source code will take about 50 hours of effort for a four-person team at the recommended preparation and inspection rates of 150 lines per hour. (The number is actually 53, but rounding to 50 makes the analysis easier.)
A 35-page design or requirements document will take about 50 hours of effort for a five-person team at the recommended rate of seven pages per hour for preparation and inspection.
We will use these numbers as baselines for ROI calculations. In addition, there is the defect rework cost for the defects found in the inspections.
Based on questions I have asked in test measurement tutorials I have taught over the last four years, most organizations have little idea of their test cost. Again, there are easy ways to approximate these costs if you do not have the actual data. Most organizations can identify the start of the last test stage before shipping a product to the customer. Typically called System Test, it is conducted with more control and measurement than the Unit or Integration Test stages that precede System Test. There is usually some sort of problem-tracking system in place, and often there are closure codes that identify defects that are fixed as well as other closure reasons (user error, duplicate problem, etc.).
To develop the cost per defect found in this test stage, take the development and test resources on the project and divide that by the number of defects. There are several assumptions that are
|Calculating the Economics of Inspections||25.5 KB|