What Testers Can Do About Technical Debt (Part 1)

[article]

Taking a Closer Look
Are you seeing signs of technical debt? If you suspect you're floundering in technical debt, take a few measurements to see if you can make sense of the data. The following table shows data I gathered from one project. I looked at the state of the product on a quarterly basis, counting the total number of lines of code. (We used the Unix command, wc -l recursively over all the code in the branch on the specific date.) When taking gross measures, LOC is appropriate because it gave us insight into what was happening in the project. For other measurements, LOC is not appropriate.

Aside from LOC, measure the Fault Feedback Ratio, the FFR. To measure FFR, we took the defect data from the bug-tracking system, and for that date, we measured the number of bad fixes to the total number of fixes. We also determined the cost to fix a defect pre-release.

To measure cost to fix a defect, we tracked all the defects for the release, and took the average cost to fix. See my previous column "What Does It Cost to Fix a Defect?" for more information on how you would calculate the cost to fix a defect.

Date

Size in LOC

FFR (this week only)

Average cost to fix a defect, pre-release

1/1

425,000

14%

1 person-days

4/1

800,000

16%

3 person-days

7/1

1,500,000

12%

2 person-days

10/1

2,000,000

18%

 3 person-days

It's always tempting to jump to conclusions from looking at the data. Don't. Probe and ask questions to make sure you know what the data really means. For example, the code size increased almost 50 percent each quarter. Do I think the developers are capable of writing that much high-quality code? What did the developers do, to generate that amount of code? Did they leave experiments (rejected designs) in the code? Did they hire more people? Did they use a code generator? If you'd like more guidance on what you should expect for increases in code size, see Capers Jones' book, Software Assessments, Benchmarks, and Best Practices.

In this case, it turns out that instead of re-architecting the code to account for new knowledge and changes in requirements, the developers were cutting and pasting code all over the system. The system was bloated, and every time the developers had to make a fix in one place, they had to remember the forty-seven other places to fix.

Then I look at the FFR. Does it make sense? What makes the FFR go up or down? In this case, the FFR started high, and aside from the third quarter, continued to go up. What happened in the third quarter to make the FFR go down? In the third quarter, one development team re-architected one small module that was originally a huge source of defects. With the new version, the total number of bad defects went down and the cost to fix a defect went down. They had paid off some of the "debt," so now they were paying less "interest" on the debt. You may also want to look at FFR by subsystem or module, to see if one subsystem or module is causing much of the defect-fixing pain.

The cost to fix a defect for this organization is extremely high. If the developers and testers find twenty defects a day, then they generate about sixty days' worth of work for every day of testing.

Let me know your thoughts on technical debt. My next column will provide pointers for diagnosing and decreasing the debt.

Acknowledgements
I thank Esther Derby, Dale Emery, Dave Smith, and Jerry Weinberg for their review on this column.

About the author

Johanna Rothman's picture Johanna Rothman

Johanna Rothman, known as the “Pragmatic Manager,” helps organizational leaders see problems and risks in their product development. She helps them recognize potential “gotchas,” seize opportunities, and remove impediments. Johanna was the Agile 2009 conference chair. She is the technical editor for Agile Connection and the author of these books:

  • Manage Your Job Search
  • Hiring Geeks That Fit
  • Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects
  • The 2008 Jolt Productivity award-winning Manage It! Your Guide to Modern, Pragmatic Project Management
  • Behind Closed Doors: Secrets of Great Management
  • Hiring the Best Knowledge Workers, Techies & Nerds: The Secrets and Science of Hiring Technical People

Johanna is working on a book about agile program management. She writes columns for Stickyminds.com and projectmanagementcom and blogs on her website, jrothman.com, as well on createadaptablelife.com.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!