How Do We Compare?

problems more often than a non-regulated industry.

But, what about the fact that the defect escaped system test? Or, that the defect escaped integration test or unit test? Do you want to do some root-cause analysis and see where it would have been more reasonable to find the defect, given the way you develop and test your product? I think so, and I think it’s reasonable to seek alternatives. No matter how you look at it, any cascading defect that causes you to lose millions or to not care how many people you assign to fixing it fast is a huge problem. If you might have even one of those, you ought to consider how to eliminate it and significantly reduce the risk of those kinds of defects altogether.

Look for the Causes and the Context
When people ask me about comparisons, I often discover they want to compare apples and frogs. The response to “How agile are we?” is often “Not very,” which is not helpful to the person asking. She is doing her best, given her context.

A better approach is to ask other questions, such as “What is preventing you from shorter iterations, and why do you need hardening iterations at the end?” The other person may describe the project context and why the organization came to have long iterations in the first place—reasons that may have been applicable at one time but no longer are.

When I asked one colleague these questions, she said, “We didn’t think we could fit enough into shorter iterations, and our testers are twelve hours away. So, we have the too-much-to-do problem on both the development and testing ends of the project.” Lengthening iterations isn’t the answer, though new agile teams often think it is. I used the Five Whys to walk my colleague through the effect of her thinking, and she learned why longer iterations aren’t such a good idea. A value stream map would have shown her the same thing.

Compared to What?
When someone asks you how she is doing compared to others, the best rejoinder is to ask, “Compared to what?” The only relevant comparison is how well you are doing with respect to yourself. Your value stream map will show you if you are wasting time somewhere. Your data can tell you if your defects escaped release. Look for causes of outlier data and the context for it. You don’t need to compare yourself to the industry. You are unique. Revel in your uniqueness, and understand why.