The 3 Ps of Evaluation
Evaluation is difficult. Honest evaluation might reveal we're building the wrong thing or not moving fast enough. It may result in the realization that we're not being rigorous enough with the process we're following or that we're mistaking process compliance for process effectiveness. The stuff we evaluate falls into three categories: pace, product, and process.
Pace is where we measure how much we've done. We may have planned on building five features in the last sprint but only completed four. So, what does that say about how fast we're moving? How does that effect the scheduled delivery date? Do we fool ourselves, say, "We'll get faster next time," and make excuses for poor performance, or do we face facts and adjust the schedule to reflect how fast we're really moving. It's OK to say, "Let's go one more sprint. If our velocity stays the same, we'll adjust the schedule." It's not OK for managers to say, "You guys committed to getting this stuff done. You'd better figure out a way to make up the lost time."
We'll need to inspect what we've built. Not the parts, but the whole thing. I see many teams do a product demonstration, which is a good idea. Knowing you'll have to show product to your peers and stakeholders is strong motivation to finish and do a good job, but don't stop there. A common concern of folks adopting agile development is that in the rush to build more functionality faster the software will become riddled with bugs or turn into an unmaintainable ball of mud. If you don't pay attention to quality, you will indeed end up with bug-filled behemoth.
When evaluating your product, it's not sufficient just to look at it. We need to dig a bit deeper. This digging deeper part is bit of a blind spot for many agile teams. Before you wrap up your product review, do the following:
- Ask testers to speak to functional quality
Is the product full of bugs, or is the functional quality good and improving?
- Ask developers to speak to code quality
Is the code in good shape? Well designed? Easy to change and to add new functionality? Or is the rush to finish features compelling us to take shortcuts?
- Ask UI designers to speak to user experience
The product may look OK, but how is the user experience? Is it easy for users to accomplish what they need to in the system? Does the product have the look and feel and general quality of experience in line with your company's standards? Or, are we just adding many crappy features into the product?
- Ask the product manager to speak to benefit and releasability
Since we don't get value from software until we release and people use it, it's important to speak to how close we are to being ready to release. As anyone seasoned in software development can tell you, finishing all the features we planned on doesn't necessarily mean we're arriving at a product people will use and value. The product owner should talk about what is necessary to build towards a releasable product. This may involve adding more features, changing some features already built, or removing features that aren't really important.
- Use simple, subjective assessment
Don't get too rigorous about your measurement of quality here. Adjectives and subjective evaluations are pretty good ways to describe where you believe you are.