The buzz around the software industry is "quality matters." Pick up any technical magazine or newspaper and you'll read about the cost of bugs, the price of poor quality, and the need for improvements throughout the development lifecycle. But do we agree on just what "quality" is?
To get to the heart of the matter, we asked a number of people, "What is quality, anyway?" We chose to question not just industry experts, but also software vendors, the people down in the trenches, and the people running the show. Throughout 2004, we'll explore what it means to "care about quality." To kick things off, let's see what James Bach, renowned exploratory tester and quality guru, and Lisa Skett, a quality assurance analyst, had to say.
Quality Is a Relationship
James Bach believes that Jerry Weinberg has the best definition of quality: value to some person. So who are those "persons," what do they value, and how do you decide whose opinion matters? "Those are questions I find useful to ponder, every day of every project," explains James.
"Quality is not an intrinsic characteristic of software. It's a relationship among the product, the people who have expectations about the product, and the world around them." He gives an example of a Nokia cell phone he once owned. The phone was ideal, reliable, and had good features. "But," says James, "I no longer use it because it's not compatible with my new cell service." The cell phone did everything it was intended to do, but still ended up in the garbage can because the world around it changed. Similarly, if you send a perfectly bug-free product out into the world, and it's more expensive than the customer expected or doesn't do what the user wants, it's probably not going to be perceived as a quality product.
James cautions against trying to satisfy these relationships by conforming to written requirements. "Conforming to written requirements can be a good thing, or it can be irrelevant, but it's never the whole story. The whole story of quality simply can't be recorded on paper. We have to understand what's implied by the written requirements, not just what is explicitly stated. To me, it's more important to stay awake throughout the project and stay in close contact with those who matter."
Often the challenging part is deciding just who matters. James remarks, "It's up to the tester to try to identify all of the people whose opinions matter. A good tester should detect hints of important people not at the table who should be, and pass that information along. In other words, speak up if you think that the final product will be seen as a failure by someone who may be ignorable now, but will be vocal and influential later on."
James also says that testers should focus on providing information. "To think you are the gatekeeper for quality is a self-destructive idea. That way lies madness. Testers don't have the power—or the perspective—to make those kinds of determinations. It's my job to provide the best information I can, both in the ways they want and in the ways I think they should want." He gives an example of a buffer overflow error. Historically, developers may have dismissed it—"Our users won't be entering extraordinarily large numbers into a field. That bug doesn't matter." But that kind of bug can be a boon to hackers. "The hacker sure turns out to matter when CNN is reporting a new virus that exploits the buffer overflow," James adds with a laugh.
"Ultimately, though, your quality goals should be aligned with the goals of the company," concludes James. "Headlights should point in the direction you're going; otherwise, they aren't going to be of much use to you."
Quality Is About Meeting Needs
Lisa Skett, an SQA analyst from a large NY financial company, has approached software from a variety of angles, so it's no surprise that she believes quality is largely a matter of perception. But if you press her, she’ll tell you that it's less about perfect code and more about user and business needs.
"I once heard of a diver whose Rolex gave out on him underwater," she explains. "For many, the Rolex is the epitome of quality, but for this guy, a Timex turned out to be the quality choice because it better served his needs at hand."
"As a tester,” Lisa says, "I am focused on how well the software integrates and whether all the defined requirements are met." But when she sits at her computer to use a program she only cares about one thing. "Does it do what I want it to do?" If not, she says, "it doesn't matter how wonderful the code is."
Perhaps this focus on user results comes from her evolution into testing from the business side of the company. Lisa started 22 years ago settling annuity benefits, where her main focus was to get the job done well and quickly. Software was only a tool she used to make things happen. It just needed to function and be easy enough to use so as not to slow her down.
Eventually, though, Lisa realized that participating in how the system functioned might help her and others make software improvements that would ultimately increase their productivity. Soon she became involved in testing from a user's perspective. “During this time," she recalls, "I focused mostly on whether the software was easy to use. Did it do what I expected? I didn't give much thought to what was processing behind the screens."
Now that she’s doing full-fledged testing, she readily gets under the hood to see if the software is processing requests correctly, if code changes in one area are affecting other components, if the whole system is integrated properly. "I've learned to take nothing for granted," she explains with a laugh. Lisa believes that this kind of nitty-gritty analysis is needed to ferret out problems large and small. When she talks about creating reusable test cases and improving quality processes, you can hear the fervor in her voice.
Yet, she cautions, “You must always be mindful of the end point: doing business. QA people are not just a part of IT. We are employees of the company and must ultimately satisfy business needs." In Lisa's view, it's all about needs. The users need the software to do what they expect, the tester needs to prove the software is processing correctly, and ultimately everyone needs the business to succeed, so that they can stay employed. She continues, "It's not just ‘Does this field work?’ It's 'How are we helping this software make the company more successful?'"