Grady , and even some recent work [17, 23]. Take for example, an estimated defect population for ten thousand source lines of code expressed in defect density metrics. One thousand defects would not be uncommon. Watts Humphrey has found the estimated defect population for this size of program to be as much as fifteen to one hundred and fifty percent in many cases . It would take approximately seven hundred and eight Inspection hours to eliminate as much as nine hundred defects in a well run Inspection, while yielding 1.27 defects per hour. Ninety more defects could be found by testing using over eleven hundred hours, for a total cost of eighteen hundred and fifty-two V&V hours using lifecycle methodologies.
Testing alone would require over eleven thousand hours to eliminate almost nine hundred defects from the same population, in an extremely well run testing process. Testing would result in the delivery of over one hundred defects, should an unwary software manager commit to performing over eleven thousand hours of testing.
Contemporary lifecycle methodologies would only require four hundred hours to eliminate all of the defects from the same population . This effort would include development time as well. This would be an improvement of seventy eight percent over traditional lifecycle methodologies, and an improvement of ninety seven percent over testing. The empirical results are overwhelming and undeniable.
Costs of Methodologies
Assuming that Rayleigh theory is correct, it would not be advisable to use testing-based V&V methodologies. Rayleigh models predict that an overwhelming population of defects exists in all software lifecycle artifacts by the time testing begins, if lifecycle methodologies are not employed [9, 11, 21, 22]. Using testing alone would overwhelm the testing process, software lifecycle resources, and inevitably result in passing a large defect population onto software customers and users.
Hewlett-Packard reports saving over three hundred and fifty million dollars in software lifecycle costs by the corporate-wide use of Inspections, between 1989 and 1998 . Inspections are the linchpin of lifecycle methodologies [8, 11, 17].
Raytheon plans on using the aid of lifecycle methodologies to save as much as three and a half billion dollars in corporate operating expenses . Raytheon will be highly dependent on the defect elimination properties of lifecycle methodologies described in this paper.
At least three conclusive positions can be identified by this paper:
- Testing is inefficient and happens too late in the lifecycle: Testing is at least ten times less efficient than Inspection-based life cycle methodologies [12, 13]. In addition, testing begins far too late to begin addressing the latent defect populations predicted by Rayleigh models [9, 11]. Most contemporary software managers would not concede to performing minimal testing, much less carry out a program of highly structured testing (based on the author's experience).
- Lifecycle frameworks are inundating, nonmethodological, and not easily understood: Lifecycle frameworks are encyclopedic and taxonomic in nature [6, 7]. Lifecycle frameworks are non-methodological, requiring them to be tailored from a catalog of techniques . Lifecycle frameworks lack metrics and measurement-intensive processes that Inspections have [6, 7]. There is not one published or publicly available case study on the quantitative benefits of lifecycle frameworks (based on the author’s experience).
- Lifecycle methodologies are fast, efficient, measurable, and accurate: The costs and benefits of lifecycle methodologies are some of the most measurement intensive industrial examples of software lifecycles [11, 12, 17, 21, 23]. The inefficiencies of testing-based V&V approaches are extremely well documented [12, 13]. Lifecycle frameworks have been obsoleted by upstart lifecycle methodologies (based on the author's experience).
- "IEEE Standard Glossary of Software Engineering Terminology," IEEE Standard