Here's a puzzle: If one defect has a severity rating of 3 and a priority rating of 2, and another defect has a severity rating of 2 and a priority rating of 3, which one do you fix first? In this column, Johanna Rothman tells why she thinks severity/priority combinations can be confusing, and she offers her own simpler, three-tiered rating system.
I was discussing release criteria with a client recently. The development manager proudly arose and said, "We have no defects at level 0!" I asked what that meant, and he smirked, saying, "No bugs we absolutely have to fix before we ship." "Great!" I said. "How many defects do you have at the next level?" He sat down and said, "I don't actually know." "Oh?" "Well, the developers and testers are fighting it out down the hall."
These folks were tying themselves into knots, trying to deal with the risk of releasing the product with risky defects. They had four levels of priority and five levels of severity. They used all combinations, but they really only had two levels: the ones they fixed and the ones they didn't fix. The problems they didn't fix during a given release, they didn't normally fix in a later release either. They weren't using the information in their problem reports to their benefit.
Instead of priority and severity, I use risk as a way to deal with problem reports, and how to know how to fix them. Here are the levels I choose:
- Critical: We have to fix this before we release. We will lose substantial customers or money if we don't.
- Important: We'd like to fix this before we release. It might perturb some customers, but we don't think they'll throw out the product or move to our competitors. If we don't fix it before we release, we either have to do something to that module or fix it in the next release.
- Minor: We'd like to fix this before we throw out this product.
Three simple levels. But it's not simple to transition to them. When I suggested these levels instead of their twenty combinations, the managers said, "But the developers won't know what to do first. And the testers won't know what to verify first." I asked if we could talk to a couple of developers and a couple of testers.
One developer thought the levels were a good idea. He couldn't tell the difference between a priority 2/severity 3 problem and a priority 3/severity 2 problem. To him, they were all the same. He thought if he could see all his critical problems, he could manage his time and approach his fixing better.