Estimating Testing Time

pre-release. If I knew that the first cycle in the last release found 200 problems, that it took the developers half a person-day each to fix the problems, and I have ten developers, I estimate ten working days to fix problems. That's a long time. And yes, I was on a project where that's what it took. It was agonizing-we thought we'd never finish fixing problems.

Now that you know how long a cycle should take, estimate the duration for fixes after the first cycle. How many cycles of testing do you plan? When I set up projects, I tend to use some proactive defect detection and prevention techniques, so I generally plan on three cycles of testing. In a casual conversation with Cem Kaner a number of years ago, he mentioned a product for which he planned on eight cycles of testing. For one project, for which I was a contract test manager, we had almost thirty cycles of testing. I can't tell you how many cycles of testing you'll need because that depends on the product's complexity, how good your tests are, and the initial count of product defects before this project started.

Here's what I have noticed from my work with multiple organizations. The groups who want to decrease testing time tend to perform the least proactive work reducing the overall number of defects, and they typically perform primarily manual black box testing at the end of a project. I understand their desires, but they've set up their lifecycle and processes to produce exactly the wrong result.

The best guess I have is to estimate the number of cycles you'll need for testing, the duration of one cycle, and the time it takes for developers to fix problems between cycles. Add up the testing plus fixing/cycle and multiply by the number of cycles you think you'll need, and you'll have an estimate of the testing time needed. By the way, when I do this, I never give a single number; I always give time per cycle ("It will take us six days per cycle"), my estimate of the number of cycles ("We'll need four cycles"), and my estimate of defect-fixing and verification time ("Plus three days between test cycles for developers to fix problems"). That way I can show the costs of not performing proactive defect prevention in a nice way. And I can show my uncertainty in my estimation ("That's a minimum of thirty-six working days, longer if the defects take longer to fix and verify").

If you want to reduce testing time and create a low-defect product, test all the way through the project with a variety of test techniques. Use test iterations so you know at the end of one test cycle where you stand with defects. The lower the number of defects and the more sophisticated your tests, the faster your testing time.