Most people tend to rate art on a “beauty scale.” But beauty is, and always will be, in the eye, ears, nose, mouth, hands, and brains of the beholder. A decision about beauty or lack of beauty in a particular work of art is instinctive and natural. In fact, you probably won’t even have to make that decision; you’ll either be captivated by a piece of art or you won’t.

Art critics, however, operate on a different level. They are not interested in mere beauty but also in characteristics like shapes, patterns, symmetry, colors, textures, composition, proportions, presentation, framing, inherent meaning, uniqueness, fulfilled intent, and skill of the artist. These are attributes that contribute to the overall “beauty” of a work of art, but there is more to the evaluation of art than meets the eye. The appreciation of a work (this applies to non-visual art forms, as well) is also influenced by the setting in which it is presented to an audience.

This became clear through an experiment conducted by the Washington Post in 2007. On a cold January morning, in a Washington, DC metro station, a man with a violin played six pieces by Bach (Johann Sebastian, not James Marcus). This lasted for about forty-five minutes. During that time, approximately 2,000 people went by, most of them on their way to work. The man played continuously. Six people stopped and listened for a short while. Twenty people gave money but continued to walk at their normal pace. Only one person recognized the musician, and she couldn’t believe her eyes and ears.

Here’s the catch: That violin player wasn’t just an anonymous street busker. He was Joshua Bell, one of the greatest violin players in the world. That morning, he was playing the “Chaconne,” one of the most intricate pieces of music ever written, on a 300-year-old Stradivarius violin with an estimated worth of $3.5 million. He collected a total of $32 dollars. The night prior, he earned considerably more playing the same repertoire at the Boston Symphony Hall, where regular seats went for $100.

This experiment addressed some interesting questions:

• If we’re confronted with beauty in a commonplace environment and at an inappropriate hour, do we perceive it? Do we stop to appreciate it? Do we recognize talent in an unexpected context?
• What is beauty, really? Is it a measurable fact or merely an opinion—or is it a little of each, colored by the immediate state of mind of the observer?
• If a great musician plays great music that no one is hearing, is he any good? This is an old epistemological debate that continues to engage philosophers.

Each passerby had a quick choice to make: Do I stop and listen? Do I hurry past with a blend of guilt and irritation, heading for that first office coffee? Do I cough up some cash, just to be polite? Does it make a difference if the player is really bad? What if he's really good?

Lots of people didn’t bother to listen that January morning. Does that mean we should label all those commuters as unsophisticated yahoos? No, because when it comes to beauty, context matters.

The 18th-century German philosopher Immanuel Kant, who took beauty seriously, said the same thing in his Critique of Aesthetic Judgment (1790). He argued that one's ability to appreciate beauty is related to one's ability to make moral judgments. Furthermore, to properly appreciate beauty, the viewing conditions must be optimal—and "optimal" doesn't mean heading to work, focusing on that report to the boss, or fetching the kids from school in time.

Much has been said about the impact of biases on our noticing—how it is easy to miss things. If we fail to recognize genius in a metro station, how many other things are we missing through incongruent contexts?

As I was first reading about the Joshua Bell experiment, I couldn’t help but seeing the similarities with software testing. After all, testers are often asked to evaluate the quality of a piece of software. But, just like beauty, quality is highly subjective. Quality characteristics are all factors that contribute to the overall quality, which in the end is the sum of all the quality scores of all persons involved. Such a thing is hard to gauge.

If we take a look at Jerry Weinberg’s definition of quality as “value to someone,” shouldn’t the context in which we are assessing that quality be taken into account as well? After all, there are a lot of factors that are capable of influencing the quality of our judgments:

• Feelings: Isn’t quality perceived differently by testers when experienced in different circumstances—for instance, when they are in a hurry, when they don't want to be bothered, or when they're not in the mood?
• Expectations: What about a tester’s view on quality when he already "expects it to be mediocre" (like one might expect from most street musicians)?
• Words: Seemingly trivial conversations can have quite an impact. Testers are often primed by simple phrases from everyday tester life: “Oh, you don’t need to test that; I didn’t change anything there”; “Can you perform this little test for me? Don’t sweat it, it will probably just take an hour or so”; “Any news on those test results? Boss wants them by noon—just saying.”
• Distractions: Your daily life and work environment is a tasty smorgasbord of distractions: kids, desk, mails, colleagues, and your computer screen with those social media that need scouting 24/7. All of these interactions may provide much-needed defocusing, but they are not creating an optimal setting for our quality-related efforts.

Quality has a lot in common with beauty. To properly assess the characteristics of a particular product (or work), the testing (or viewing) conditions need to be optimal. If quality is “value to someone,” maybe we should extend that with “in optimal conditions” or “in an ideal context."

This leaves us with another question: Is there such a thing as “optimal testing conditions”? What are those, really? People are people; we will always be biased and fooled by our own minds. Neither can we deny that we are living in the age of distractions. So, how can we set up for a totally unbiased, unspoiled look at quality?

We probably can’t, but what we can do is try to make those conditions as good as possible. How? By creating an environment for testing in which testers get enough freedom in their work, or by being more aware of possibly disturbing factors, taking them into account, and acknowledging them when testing. We can try to eliminate as many distractions (interpersonal, technical, and environmental) as possible. That way, our judgments on quality will be of high quality.