The Cost of Software Quality

required to get the software out the door initially does not see testing as an investment. Quality is given lipservice when the only priorities are shipping something—anything—on a given schedule and within a given budget.

You’ll also want to get a better grip on what quality actually costs your organization. As Crosby points out in Quality Is Free , most organizations that haven’t looked at quality costs carefully significantly underestimate those costs. Campenella describes rigorous accounting techniques in Principles of Quality Costs. However, first-order estimates—provided you are conservative—can be used to build a solid, credible business case. Ask programmers, technical support personnel, testers, business analysts, and managers (development, technical support, help desk, IT, sales, and marketing) what percentage of their time they spend dealing with each kind of failure, internal and external. To turn these percentages into dollars—or euros, or pounds, or yen, or whatever—figure out what a staff-hour costs in your organization. For example, $100 per hour is typical in the United States for technical organizations, including costs like health insurance, facilities, and other overheads beyond just salary. Add in other costs as appropriate. For example, in one company I talked to, their response to field failures was to fly a programmer to the client's site, along with sufficient tools to fix the bug, and keep him there until the problem was fixed, which was typically about a week. Last minute airfare, hotel costs, meals, and car rental added about $2,000 to the $4,000 cost associated with the programmer's lost time. (For additional ideas on doing this analysis, see my book, Managing the Testing Process, Second Edition, Mark Fewster and Dorothy Graham's book, Software Test Automation, or Johanna Rothman's web site, www.jrothman.com.) Your management team will probably be surprised at what quality costs your organization right now, and how little value it's getting from that expenditure.

However, having supportive, far-sighted management is only a necessary—not a sufficient—condition for achieving positive returns on your test investment. Just as in the stock market, there are right and wrong ways to invest. Picking the right tests, managing the appropriate quality risks, using the proper tools and techniques, and driving testing throughout the organization will result in optimal returns, while failure in any one of these areas can mean disappointing or even negative returns. In the next few columns, I'll explore each of these topics.