What is quality? What makes some people notice bugs and other people not notice them? Are defects always bad? Software test engineer Bernie Berger has spent time considering the nature and essence of testing. This article takes an introspective approach by posing a list of philosophical questions for testers to consider.
I'm not a philosopher. I am, however, an experienced software tester who likes to think about what he's doing and why he's doing it. On second thought, this might make me a philosopher.
At a recent conference, James Bach, a well-respected software testing consultant, described software testing as "organized skepticism" toward software. Along those lines, if we are skeptical about the specific behavior of software, we could also have a more general skepticism about our personal functions and relationships within the software quality organization.
For those of you who have taken time to ponder the meaning of life or "QA," I present this list of philosophical questions about software quality. Some of these questions have obvious answers, some have multiple answers, and some others are rhetorical. There’s not a single correct answer to any particular question; in fact, there might be many or there might be none. Perhaps you'll gain new insight as you ponder.
This list is not a completed work. It is intended to be a first step in gaining a deeper appreciation for what we claim to do as software quality professionals. You will probably get the most out of it by reading through the entire list, then returning to the top to consider your answers as you re-read each question.
The Nature of Quality
· What is quality?
· What is perfection?
· Is perfection binary? Something is either perfect, or it is not perfect?
· Is quality binary, or are there levels of quality?
· Are humans perfect?
· Are humans capable of producing perfection?
· Are humans capable of producing perfect software?
· Are humans capable of producing quality?
· Are humans capable of producing quality software?
· What defines high and low quality?
· Why are there bugs?
· Why doesn't everything work right the first time, all the time?
· What was the first defect ever created in the history of the world?
· What was the first defect you personally created?
· What was the first defect you ever found?
· What can we learn from defects?
· Are defects always bad?
· Bad occurrences sometimes have good outcomes, such as when things are "a blessing in disguise." Could there ever be a positive outcome from not finding a bug?
· If so, under what condition would not finding a bug constitute a good thing?
· Could we say that all missed bugs are blessings in disguise?
· If so, then should the goal of testing be to miss as many bugs as possible, to get the most blessing? (obviously not)
· What are accidents?
· What causes them?
· How can they be avoided?
· Should they be avoided?
Free Will / Predetermination
· Do people have free will to make choices in life?
· If so, can a tester choose to find a bug?
· Can a tester choose not to find bugs?
· Can developers choose to create or not to create bugs?
· Is software testing an active or passive activity?
· Do people's moods or emotions play a role in software quality? If so, how?
· Can a tester control his actions so that he is more prone to finding bugs?
· Is life predetermined, and if so, is software quality predetermined? By whom?
· Is software quality predetermined by Process?
· Is software quality predetermined by Management?
· If everything, including software quality, is predetermined, what's the use of testing?
· If well-meaning developers have total free will over their actions, then why do they make mistakes?
· Are testing results predetermined?
· There are “x” number of bugs in an application, some will be discovered and some will not. Are we meant to discover the bugs that we did find? Are we meant not to discover the ones that we did not find?
Existence of Defects
· Do bugs exist if they are not found?
· Do bugs exist if they are not reproducible?
· How much effort is necessary to reproduce a bug?
· How much effort is necessary to prove that a bug exists, was fixed, or is still open?
Goal of Testing
· What are the similarities and differences between the concepts of "comparing" and "contrasting"?
· Do we compare expected results with actual, or do we contrast them?
· If we compare, then the goal of testing is to verify that software works. If we contrast, then the goal is to prove that it doesn't. Is there a practical difference?
· What is the real purpose of testing?
· Who does it ultimately benefit?
· How does testing software benefit humanity/society?
· Is there ever a benefit to humanity by releasing defective software?
Improving Tester Quality
· What is skill? What is talent?
· Why are some testers better than others at finding bugs?
· What happens to a tester biologically/mentally/(spiritually?) when he is totally focused and completely concentrating on the application that he's testing?
· Can that be replicated?
· Can that be measured?
· Can it be taught?
· Can it be invoked at will?
· Is there anything predictable behind it, or does it just happen?
· Can you get "psyched up" to find bugs, like athletes do before events? How?
· Is there a way to mentally prepare for testing software?
· Can you use visualization techniques to become a successful bug finder?
· Can people's emotional state at a given time contribute to defective software?
· What nontechnical elements-emotions, beliefs, psychology, timing, stress, anxiety, depression, substance abuse-can contribute to defective software?
· What makes you notice bugs?
· What makes some people notice bugs and other people not notice them?
Releasing Buggy Software
· Is there a difference between accountability and blame?
· What causes accountability?
· What causes blame?
· Should testers be held accountable for defective software?
· Should testers be blamed?
· Should anyone ever be blamed?
· Should everyone always be blamed?
· Do users have a right to have a zero tolerance for defects?
· When they pay, does the price include the defects contained in the software?
· Who should set due dates for projects?
· When should software be released?
· Who decides when software is released?
· What role does QA or Testing have in the release process?
· What role should it have?
· Other quandaries
· Is "process" the answer to everything?
· Does process always work?
· What about the U.S. national election in 2000? Did that process work?
· Do you remember Ford Motor Company's slogan "Quality is job 1"? Was it really job 1?
· Why are there so many automotive recalls?
· On 15 August 1965 the Beatles played the first concert ever to take place in an outdoor arena at Shea Stadium in New York in front of 55,000 screaming fans. However, they showed up with a sound system fit for a high school gym. As a result, not only could the fans not hear anything, but the band members couldn't even hear themselves play. I would call this a defective concert, yet it is one of the most famous rock concerts in history. Is there anything we can learn from this as software testers? It would be interesting to learn whether the sound engineers reviewed their sound system requirements, and if so, what went wrong?
Obviously, I believe in asking questions. Asking questions can serve multiple purposes. It can be a request for information (such as, “How should I test this software?”), or it can activate professional self-awareness (“Why should I test this software?”). I find "why" questions intriguing because thinking about their answers usually reveals a deeper level of understanding.
Please enjoy this list. It might even trigger some of your own questions. I'd be delighted to read your comments about this list of questions as a whole, your answer to any particular question listed here, or any question that you may come up with as a result of thinking about one of mine. I can be reached at firstname.lastname@example.org.