How to Preview User Satisfaction before Your Release


Why wait to discover how your users will react to your system when there are ways to measure such things during development? This column describes a simple tool to develop visibility into customer satisfaction. Learn how you can begin to manage expectations so that neither you nor the customer has an unpleasant surprise on release day.

It's 10 a.m. You're about to ship to five beta sites. You've met the date, you're within budget, and the defect counts have been steadily declining for the last four weeks. Still, you're a little nervous. How will the customers react to this new release? 

Carrie was unhappy to learn during beta testing that the product her team had been building for insurance underwriting reps had drifted from what the users had wanted.

"It's too slow," the beta test group said. "It takes forever to pull up the records when I have a customer on the phone!"

"But in test it performed within the parameters in the requirements," Carrie protested. "What changed?"

Just a small design tradeoff that affected the default display order of the records. It had seemed like a reasonable decision at the time.

If you're a project manager, you know how important it is to have visibility into the product and the project. It's bad news to find out in a beta test that you have built a product that doesn't meet your customer's expectations. How can you tell if you're still on track to meet customer expectations?

Agile methods are one option. The agile school advocates close customer contact and frequent deliverables that are directly tied to business value. Still, many projects follow more traditional models: the customers (or their surrogates) are involved up front in defining features and then come back in during testing. A lot can happen during the time in between. If you're not using an agile method and don't have the luxury of having a customer assigned full-time to your team, how can you stay in touch with what the customer expects? One option is to measure customer satisfaction as you design and build the product.

Take a page from marketing and use a survey to catch a glimpse of customer reactions before the product ships. During development, ask the people who are closest to the product—sponsors, designers, and customers (or surrogates)—how they feel about the project.

You probably identified a set of system attributes during initial project definition. Suppose your customers identified these four qualities that were most important to them: easy to use, fast, always available, and secure. Of course, these descriptions are too vague to be helpful in actually building a system. You probably negotiated more specific targets for the product as a whole and for specific functions within the product. But for the purpose of a satisfaction survey, these more general attributes are about the right level.

For each attribute, create a bipolar scale, with the desired attribute on one end of the scale and its opposite at the other. For example: the opposite of "fast" is "slow"; the opposite of "easy to use" is "difficult to use."

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.