Arguing Apples and Oranges

[article]

Priority levels:

  • Now: drop everything and take care of it as soon as you see this (usually for blocking bugs)
  • P1: fix before next build to test
  • P2: fix before final release
  • P3: we probably won't get to these, but we want to track them anyway

And now you can see why Tim was so adamant that the issue was a high. From his perspective, it was a P1 matter. They're both right. It's of medium severity, but P1 to fix.

Priority Is Relative; Severity Is Absolute
Further, the priority might change over time. Perhaps a bug initially deemed P1 becomes rated as P2 or even a P3 as the schedule draws closer to the release and as the test team finds even more heinous errors. Priority is a subjective evaluation of how important an issue is, given other tasks in the queue and the current schedule. It's relative. It shifts over time. And it's a business decision. By contrast, severity is an absolute: it's an assessment of the impact of the bug without regard to other work in the queue or the current schedule. The only reason severity should change is if we have new information that causes us to re-evaluate our assessment. If it was a high severity issue when I entered it, it's still a high severity issue when it's deferred to the next release. The severity hasn't changed just because we've run out of time. The priority changed.

Priority and Severity Don't Mix
In response to Johanna's column last week, some people suggested using both severity and priority to come up with a composite risk number. While this intuitively sounds like a way to resolve the priority-severity divide, I suggest using such an approach with extreme caution. It's multiplying apples by oranges in an attempt to quantify bananas. Risk is yet a third type of information. The risk associated with any bug depends on the severity of the issue, certainly. But it also depends on the likelihood that the user will run into it as well as the possible losses that might occur. I don't attempt to quantify all this when assessing the severity of an issue. In fact, I think that in most cases assessing the risk of a single issue takes more time than it's worth. Only for potentially poisonous bugs involving dangerous fixes do I really want to weigh the risk of fixing it against the risk of not fixing it.

Establish Work Precedence
The best way to avoid confusion about what comes first is to ensure everyone in the organization takes their cues for work precedence from priority and nowhere else. Developers fix P1 defects first. Testers verify P1 fixes first. Technical writers document P1 issues first. Everyone works in priority order: the priority reflects importance to the business. Saying, "This bug is more severe than that one so I'll work on it first" is as bad as saying, "I like this bug more, so I'll work on it first." The severity rating is technical information used by managers as a piece of the formula in determining the priority rating. The priority rating is the final word on the order in which the work is done by programmers, testers, and everyone else. The ultimate lesson here, regardless of the terms or levels you use to categorize your bugs, is that any classification scheme will only be effective if everyone agrees on definitions. So perhaps that's the very first question to ask when an argument is brewing about severity, priority, or risk: "Help me understand exactly what information you're using from each defect record and how you're using it?"

Reference
Clarify Your Ranking for System Problem Reports, By Johanna Rothman

Tags: 

About the author

Elisabeth Hendrickson's picture Elisabeth Hendrickson

The founder and president of Quality Tree Software, Inc., Elisabeth Hendrickson wrote her first line of code in 1980. Moments later, she found her first bug. Since then Elisabeth has held positions as a tester, developer, manager, and quality engineering director in companies ranging from small startups to multi-national enterprises. A member of the agile community since 2003, Elisabeth has served on the board of directors of the Agile Alliance and is a co-organizer of the Agile Alliance Functional Testing Tools program. She now splits her time between teaching, speaking, writing, and working on agile teams with test-infected programmers who value her obsession with testing. Elisabeth blogs at testobsessed.com and can be found on Twitter as @testobsessed.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!