of the project, this estimation metric is good for system test fix cost. But, look at how high Avery's cost to fix is. In reality, Avery's project met its beta release date (20 work days in system test), at a very low risk of disappointing the customers. Dan's test time took two months (40 work days), and even though the team has fixed 125 defects, they have over 300 defects not fixed. Avery's
cost-to-fix is high, because his team worked hard to prevent defects before system test. In fact, using the estimation technique above, Avery's cost to fix a defect is highly inflated in system test, because they found and fixed most of the defects beforehand. Because Avery's project detected and fixed most of the defects before system test, the estimation technique is not valid. Instead, Avery's project can calculate the actual cost of finding and fixing their defects. Avery uses his average of 8 hours system-test time to find and fix a defect. Here's the table with Avery's more realistic estimate of system test cost:
|
Company (for a specific release) |
Number of people |
Cost per person-day |
Number of days |
Number of system test fixes |
person-days |
Average days to fix during system test |
System test cost to fix a defect |
|
Dan |
5 |
$500 |
40 |
125 |
200 |
1.6 |
$800 |
|
Avery |
10 |
$500 |
20 |
30 |
200 |
1.0 |
$200 |
Using the updated data, here's a clearer picture of the cost to Dan and Avery's projects to fix a defect:
|
Implementation Costs |
||||||
|
Project |
Number of people on the project |
Cost per person-day |
Average time to fix for implementation |
Implementation cost to fix |
Number of Implementation fixes |
Implementation total cost before System test |
|
Dan |
5 |
$500 |
not tracked |
not tracked |
not tracked |
not tracked |
|
Avery |
10 |
$500 |
2 hours |
$125 |
250 |
$31,250 |
|
System Costs |
|||
|
Average time to fix during system test |
System test cost to fix a defect |
Number of System test fixes |
System test total cost |
|
1.6 |
$800 |
125 |
$100,000 |
|
1.0 |
$200 |
30 |
$6,000 |
|
Post-Release Costs |
|||||
|
Pre-release total cost |
Average time to fix post-release |
Post release cost to fix |
Post-release number of fixes |
Post release total cost |
Total pre and post release cost |
|
$100,000 |
15 person-days |
$7,500 |
23 |
$172,500 |
$272,500 |
|
$103,850 |
5 person-days |
$2,500 |
2 |
$5,000 |
$108,850 |
Avery has high system test costs, because his project spends more time looking for defects than fixing defects However, Avery's total defect fix cost for Avery's larger project is lower than Dan's smaller project. And Avery's post-release fix cost is substantially lower.
Each project will have its own cost to fix a defect, because the cost depends on the activities undertaken in the project and when you start tracking defects, as well as cost to fix. Use your fix cost to decide how you want to proceed with this project or the next one. If your cost is too high, and you're not yet in system test, you could try some defect detection and prevention techniques. Just make sure that if everyone is associated with finding and fixing defects that you don't only count the fix time, that you count the detection time also.
If your find-and-fix cost is high in system test, what's the risk of releasing early? Avery might have used his find-and-fix cost of $3333 to chose to end system test early and release early, knowing that his post-release cost would rise. Only Avery and his management could estimate the risk of releasing early.
Use the pre-release fix costs to see if you and your staff are being cost-effective in your pre-release activities. I've found that each organization has a typical post-release cost,
About the author
Johanna Rothman, known as the “Pragmatic Manager,” helps organizational leaders see problems and risks in their product development. She helps them recognize potential “gotchas,” seize opportunities, and remove impediments. Johanna was the Agile 2009 conference chair. She is the technical editor for Agile Connection and the author of these books:
- Manage Your Job Search
- Hiring Geeks That Fit
- Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects
- The 2008 Jolt Productivity award-winning Manage It! Your Guide to Modern, Pragmatic Project Management
- Behind Closed Doors: Secrets of Great Management
- Hiring the Best Knowledge Workers, Techies & Nerds: The Secrets and Science of Hiring Technical People
Johanna is working on a book about agile program management. She writes columns for Stickyminds.com and projectmanagementcom and blogs on her website, jrothman.com, as well on createadaptablelife.com.
