In Search of Defect Tracking Systems

Defect tracking systems are one of a software organization's "family jewels." They frequently control how work gets assigned to engineers and provide the data to support how managers decide when to ship. They also determine which defects to ship and which to defer. These are clearly business-critical decisions.

Unfortunately, senior decision-makers often seriously undervalue the problems these systems solve. So when it's time to invest resources in a new system, they allocate the resources as an afterthought. After all, it isn't a "real" project (the kind that generates "real" revenue for the company and ships to "real" customers). Money, time, and resources are all underestimated, commonly assuming it might take two engineers a couple of months to build the system. But unless you set these expectations correctly up front, no matter how hard you work to implement your system your chances of success will be severely limited. Better to take a broader view and treat this mission as a real project right from the start.

Understanding the Scope of the Project
Even before the project starts, you need to discover how much the new defect tracking system is really worth. Then you can get a better idea of how much time and money are reasonable to invest to get it. One way I do this is to gather some interested and important parties together, such as development managers, testers, and test managers, for a kickoff session. The very first question I ask in this session is:

"What is a highly successful solution really worth to this company?"

In many software development organizations, the defect tracking system is the primary means of communication between workers. So what's that worth? For most IT businesses, the answer is "quite a lot." Look at it this way: You'll end up paying for the system one way or another. You can pay for the system by under-investing and then getting poor productivity and poor decisions for months or years to come. Or you can put up the cash (in organizational time and effort as well as money) and get a system that really supports your work. Defect tracking systems, especially for larger organizations, are expensive.

And if they work well, they're well worth it.

Once you start this project, one of your first tasks will be to estimate the time required to put together the new system, and what the price range for new tool purchases will be. It is important to get this information into the approval process early. "Sticker shock" quickly cancels many much-needed improvement projects, when decision-makers are suddenly confronted with good systems costing hundreds of thousands, and sometimes millions, of dollars.

The bottom line is "you get what you pay for." Problems do not shrink to fit the amount of resources you're willing to expend to solve them. A defect tracking team must take it upon themselves to treat the project as if it is as real as any other project, and vow to deliver a product (albeit an entirely internal product) which is up to the quality standards of the rest of the company's products. The central lesson to be learned is that the clients-the ones who are paying for the solution-need to have a clear understanding about how much the solution is really worth so they can intelligently decide what to do to solve the problem.

Work with Management to understand what funds and approvals will be required. Determine the cost in lost time to market, customer-discovered defects, and the development rework you'll have to do without the system.

Once you determine and secure adequate funding, then you