Note that in these situations some amount of "surround material" typically is reviewed to ensure the change is correct in context. The amount of surround material is hard to specify by any formula. Approximations will work reasonably well, but counting surround material to get a more accurate relationship between effort and size is one of the more difficult issues to be addressed in the inspection process.
The real problem in the example was that using full pages as the size measure resulted in a defect density control that appeared to be controlled. When the size was measured in a way more consistent with the effort spent, we saw a defect density control chart where nearly half the data points were outside the control limits. Understanding why the two results were different and, in particular, identifying the cause of the defect density variation were important from both a quality and productivity perspective.
Were we not finding defects, or were we wasting time inspecting functional specifications that had no defects? If the latter case, what was the cause? Was it the product or function that was less complex, unique to certain persons, or a function of a different upstream process?
These are not easy questions to resolve, and corrective action is obviously different for each cause. In this case it would be necessary to look at more than just the data. Understanding the culture, the people performing the process, the end result of products delivered to test and eventually the customer, and the earlier process steps all proved important to the final outcome.
Some key points to remember:
- Use statistical and quantitative analysis to understand and improve your business
- Do not let "making good control charts" be the goal
- Ensure relationships shown in the control charts actually reflect the work being done and the relationship of the data in the real world
- Understand that some of the process data you evaluate is based on product characteristics (defect density) that may have causes outside or unrelated to the process being analyzed
- Look outside the process and product to the people executing the process for help
- If CMMI Level 4 doesn't help your business, why are you doing it (see No. 1)
® CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University