On Beauty, Quality, and Relativity


said about the impact of biases on our noticing—how it is easy to miss things. If we fail to recognize genius in a metro station, how many other things are we missing through incongruent contexts?

As I was first reading about the Joshua Bell experiment, I couldn’t help but seeing the similarities with software testing. After all, testers are often asked to evaluate the quality of a piece of software. But, just like beauty, quality is highly subjective. Quality characteristics are all factors that contribute to the overall quality, which in the end is the sum of all the quality scores of all persons involved. Such a thing is hard to gauge.

If we take a look at Jerry Weinberg’s definition of quality as “value to someone,” shouldn’t the context in which we are assessing that quality be taken into account as well? After all, there are a lot of factors that are capable of influencing the quality of our judgments:

    • Feelings: Isn’t quality perceived differently by testers when experienced in different circumstances—for instance, when they are in a hurry, when they don't want to be bothered, or when they're not in the mood?
    • Expectations: What about a tester’s view on quality when he already "expects it to be mediocre" (like one might expect from most street musicians)?
    • Words: Seemingly trivial conversations can have quite an impact. Testers are often primed by simple phrases from everyday tester life: “Oh, you don’t need to test that; I didn’t change anything there”; “Can you perform this little test for me? Don’t sweat it, it will probably just take an hour or so ”; “Any news on those test results? Boss wants them by noon— just saying.”
    • Distractions: Your daily life and work environment is a tasty smorgasbord of distractions: kids, desk, mails, colleagues, and your computer screen with those social media that need scouting 24/7. All of these interactions may provide much-needed defocusing, but they are not creating an optimal setting for our quality-related efforts.

Quality has a lot in common with beauty. To properly assess the characteristics of a particular product (or work), the testing (or viewing) conditions need to be optimal. If quality is “value to someone,” maybe we should extend that with “in optimal conditions” or “in an ideal context."

This leaves us with another question: Is there such a thing as “optimal testing conditions”? What are those, really? People are people; we will always be biased and fooled by our own minds. Neither can we deny that we are living in the age of distractions. So, how can we set up for a totally unbiased, unspoiled look at quality?

We probably can’t, but what we can do is try to make those conditions as good as possible. How? By creating an environment for testing in which testers get enough freedom in their work, or by being more aware of possibly disturbing factors, taking them into account, and acknowledging them when testing. We can try to eliminate as many distractions (interpersonal, technical, and environmental) as possible. That way, our judgments on quality will be of high quality.

User Comments

Linda Rising's picture
Linda Rising

I have heard the Joshua Bell story many times and I love it! I think the message is a good one. We see and hear what makes it through our unique set of filters. I'm sure you've read the recent study that showed professional violinists couldn't tell the difference between superb modern violins and priceless Strads :-)! Thanks for this great comment on beauty!!

March 6, 2012 - 9:39am
Ajay Balamurugadas's picture
Ajay Balamurugadas

Excellent article Zeger. I liked the way you highlighted the similarities between beauty and software testing. The points about "optimal testing conditions" are interesting. If we wait for the optimal testing conditions, we might never test. At the same time, good testing might require good conditions. :)

Thanks for the article.

March 6, 2012 - 11:37am
Tim Thompson's picture

I think that establishing a process and sticking to it might help keeping quality assessments consistent. The process needs improvements as needed, but those improvements can be made when there is sufficient time for this.
What I mean by process is defining the testing tasks upfront, especially the creation of detailed test plans. And with detailed I mean detailed to the nth degree. All these plans have to be reviewed, ideally by other testers and business analysts (or whoever created the requirements). Once there is broad consensus that the test plans are on point and sufficient then testing can begin. The benefit is that the time testers need to 'think' is removed for the most part from the time when tests are executed. That allows for focusing on many other things that the tests failed to cover, that is typically things that are not related to the items to be tested, but that are still incorrect.
That translates to the Joshua Bell story. The people in the subway might have stopped and listened more willingly if their minds weren't occupied by other tasks such as catching the correct subway to be at work on time and which things need to be done today.
The Joshua Bell example is like exploratory testing. Testers might find the most obvious flaws in software (or violin playing) by simple consumption, but it is impossible to cover all aspects and document what was done and which results were achieved by the test. I agree that it is faster in finding the big ticket items, but for the nitty gritty stuff you need to sit in the concert hall and focus on the one thing.

November 22, 2014 - 10:09am

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.