How Did I Miss That Bug? Managing Cognitive Bias in Testing: An Interview with Gerie Owen and Peter Varhol

[interview]

CP: Peter, you are a TestStudio evangelist at Telerik. You're ...

PV: I've actually left Telerik. I'm starting, actually at the beginning of March, at FrogLogic, which is another testing automation practice.

CP: OK. You're also a writer and speaker on software development and quality topics. You've authored dozens of articles on software tools and techniques for building applications. You have given conference presentations, including ours, and webcasts on a variety of topics, user-centered design, integrating testing and agile development, and building the right software in an era of changing requirements.

PV: That's all correct.

CP: OK. Anything to add?

PV: No, I don't think so.

CP: OK. All right. Gerie, you are with Northeast Utilities Inc., correct?

GO: Yup.

CP: OK. You are QA consultant. You specialize in developing and managing off-shore test teams. You have implemented the offshore model, developing training and mentoring new teams from their inception. You also manage large, complex projects involving multiple applications, coordinating test teams across multiple time zones, and delivering high-quality products.

GO: Yes, and I'm also a speaker and writer. One of my new roles is consulting with a lot of the large projects and working on test methodology and test architecture.

CP: All right. Fantastic. Let's go ahead and dive into your session here. It talks about cognitive biases. How are testers' biases affecting their ability to test effectively?

GO: I think biases affect testing in that testing and finding, and specifically with "How Did I Miss That Bug?," is about judgment. If you look at a bug, if you look at testing as judgment about software quality, then you would look at bugs as judgments as to how the software performs.

If we miss a bug, it's really missed in judgment. If you look at it that way, you also look at biases. Cognitive biases, basically, impact judgment. That's how testing it and cognitive bias, how bias, relates. Your biases affect your judgment. Your judgment affects your testing.

CP: OK. Are there a lot of common biases that testers tend to have?

PV: Inattentional blindness, representative bias, and curse of knowledge—I think those are three very common ones. I'll let Gerie explain them just a little bit more.

CP: All right.

GO: Well, inattentional blindness, that refers to Chabris and Simons's gorilla experiment. It's basically something that you missed that's in plain sight. In the gorilla experiment, they, Chabris and Simon, had their subjects watch a short basketball game, a clip of a basketball. They had to count the number of passes that one of the teams made. At the end of it, they came up with various numbers.

Then they were asked if they saw anything else. During the middle of it, a person in a gorilla suit had walked across the basketball court and done a little dance. Only 50 percent of the subjects noticed that. So inattentional blindness is when you're—the way that would apply to testing is that you're so concentrating on validating your application to spec that you can miss some potentially big things, like the name of the company is misspelled on the website.

PV: We heard that one at a conference recently. One of the, yeah, attendees actually stood up and said, "I understand completely what you're talking about. But what none of us noticed was that, when this application ran, my company was spelled wrong."

GO: Yeah.

CP: That's usually a pretty important aspect.

PV: One would think so. The whole inattentional blindness thing is all about you're so focused on looking at one thing, or one particular class of things, that you can miss some very significant other things that are going on.

About the author

Cameron Philipp-Edmonds's picture Cameron Philipp-Edmonds

When not working on his theory of time travel, Cameron T. Philipp-Edmonds is writing for TechWell, StickyMinds, and AgileConnection. With a background in advertising and marketing, Cameron is partial to the ways that technology can enhance a company's brand equity. In his personal life, Cameron enjoys long walks on the beach, romantic dinners by candlelight, and playing practical jokes on his coworkers.

Upcoming Events

Mar 27
Apr 13
May 03
Jun 01