On Beauty, Quality, and Relativity

said about the impact of biases on our noticing—how it is easy to miss things. If we fail to recognize genius in a metro station, how many other things are we missing through incongruent contexts?

As I was first reading about the Joshua Bell experiment, I couldn’t help but seeing the similarities with software testing. After all, testers are often asked to evaluate the quality of a piece of software. But, just like beauty, quality is highly subjective. Quality characteristics are all factors that contribute to the overall quality, which in the end is the sum of all the quality scores of all persons involved. Such a thing is hard to gauge.

If we take a look at Jerry Weinberg’s definition of quality as “value to someone,” shouldn’t the context in which we are assessing that quality be taken into account as well? After all, there are a lot of factors that are capable of influencing the quality of our judgments:

• Feelings: Isn’t quality perceived differently by testers when experienced in different circumstances—for instance, when they are in a hurry, when they don't want to be bothered, or when they're not in the mood?
• Expectations: What about a tester’s view on quality when he already "expects it to be mediocre" (like one might expect from most street musicians)?
• Words: Seemingly trivial conversations can have quite an impact. Testers are often primed by simple phrases from everyday tester life: “Oh, you don’t need to test that; I didn’t change anything there”; “Can you perform this little test for me? Don’t sweat it, it will probably just take an hour or so ”; “Any news on those test results? Boss wants them by noon— just saying.”
• Distractions: Your daily life and work environment is a tasty smorgasbord of distractions: kids, desk, mails, colleagues, and your computer screen with those social media that need scouting 24/7. All of these interactions may provide much-needed defocusing, but they are not creating an optimal setting for our quality-related efforts.

Quality has a lot in common with beauty. To properly assess the characteristics of a particular product (or work), the testing (or viewing) conditions need to be optimal. If quality is “value to someone,” maybe we should extend that with “in optimal conditions” or “in an ideal context."

This leaves us with another question: Is there such a thing as “optimal testing conditions”? What are those, really? People are people; we will always be biased and fooled by our own minds. Neither can we deny that we are living in the age of distractions. So, how can we set up for a totally unbiased, unspoiled look at quality?

We probably can’t, but what we can do is try to make those conditions as good as possible. How? By creating an environment for testing in which testers get enough freedom in their work, or by being more aware of possibly disturbing factors, taking them into account, and acknowledging them when testing. We can try to eliminate as many distractions (interpersonal, technical, and environmental) as possible. That way, our judgments on quality will be of high quality.