Last week Johanna Rothman proposed that tracking the priority and severity on bugs can lead to confusion about what's really important. She suggested adopting a simplified classification scheme. I think that severity and priority are two different kinds of information and that it's important to track both. Where there's confusion, the problem is usually a lack of agreement about definitions and confusion about who is calling the shots on the project. A simplified classification scheme may limit the confusion, but it won't solve the underlying problem. It may actually increase tensions as project team members feel others aren't listening to their concerns.
Let's peek in on a discussion in a bug triage meeting.
Tim, the marketing manager, is shaking his head. "That's a high on the severity scale. It's really bad, guys. You have to make it a high."
Jordan, the development manager, is barely containing her frustration. Her eye is starting to twitch as she replies, "No, Tim. That's not all that bad. It's an inconvenience, I agree, but there's an easy workaround."
"Inconvenience?!?" Tim says a bit more loudly than he intended. "You call not being able to print an inconvenience?!? That's a disaster!"
"Yes, I call not being able to print from one particular type of printer without installing an upgraded driver from the vendor's website an inconvenience. The user just needs…"
"I know what the user needs," Tim cut in. "The user needs to be able to print out of the box! You can fix this in our code, right?"
Jordan nods, "Yes, but we'd just be working around the vendor's…"
"Then fix it." Tim stood over Jordan, glaring.
"But it's a medium at best!" Jordan objected. "The user isn't losing any data, doesn't have to reboot, isn't crashing. They just have to update a driver."
This argument could continue forever. I've seen many arguments like this go on and on. What's really happening here? Why are Tim and Jordan about to be at each other's throats?
Priority is Business; Severity is Technical
Tim is looking at business priority: "How important is it to the business that we fix the bug?" Jordan is looking at technical severity: "How nasty is the bug from a technical perspective?" These two questions sometimes arrive at the same answer: a high severity bug is often also high priority, but not always. Allow me to suggest some definitions.
Severity is levels:
- Critical: the software will not run
- High: unexpected fatal errors (includes crashes and data corruption)
- Medium: a feature is malfunctioning
- Low: a cosmetic issue
Now you see why Jordan was arguing that the Print bug was a medium: a feature was malfunctioning.
Priority levels:
- Now: drop everything and take care of it as soon as you see this (usually for blocking bugs)
- P1: fix before next build to test
- P2: fix before final release
- P3: we probably won't get to these, but we want to track them anyway
And now you can see why Tim was so adamant that the issue was a high. From his perspective, it was a P1 matter.
They're both right. It's of medium severity, but P1 to fix.
Priority is Relative; Severity is Absolute
Further, the priority might change over time. Perhaps a bug initially deemed P1 becomes rated as P2 or even a P3 as the schedule draws closer to the release and as the test team finds even more heinous errors. Priority is a subjective evaluation of how important an issue is, given other tasks in the queue and the current schedule. It's relative. It shifts over time. And it's a business decision.
By contrast, severity is an absolute: it's an assessment of the impact of the bug without regard to other work in the queue or the current schedule. The only reason severity should change is if we have new information that causes us to re-evaluate our assessment. If it was a high severity issue when I entered it, it's still a high severity issue when it's deferred to the next release. The severity hasn't changed just because we've run out of time. The priority changed.
Priority and Severity Don't Mix
In response to Johanna's column last week, some people suggested using both severity
About the author
The founder and president of Quality Tree Software, Inc., Elisabeth Hendrickson wrote her first line of code in 1980. Moments later, she found her first bug. Since then Elisabeth has held positions as a tester, developer, manager, and quality engineering director in companies ranging from small startups to multi-national enterprises. A member of the agile community since 2003, Elisabeth has served on the board of directors of the Agile Alliance and is a co-organizer of the Agile Alliance Functional Testing Tools program. She now splits her time between teaching, speaking, writing, and working on agile teams with test-infected programmers who value her obsession with testing. Elisabeth blogs at testobsessed.com and can be found on Twitter as @testobsessed.
