Deception and Self-deception in Software Testing

they report upwards, putting strong pressure on testers to misreport their findings-to show testing progress or product quality as better than it is:

"We have to get creative about these numbers." [And paint a false picture.]

"We can call more of these tests passed." [Although they have failed, or perhaps not even been run.]

"The rest of the project is status green. Testing needs to be green, too." [Though it clearly isn't.]

These are some of the things testers will hear from managers who want them to lie.

Deceptions Practiced by Testers
Testers are no more immune than anyone else to temptations to fool ourselves and others about testing. We can easily succumb to the belief that we are doing the right thing-or the only possible thing-when we have blinkered ourselves to other possibilities. And we too like to believe that:

"We're going to make it!" [Whatever "it" is-often an impossible date with impossible scope.]

Have you ever exaggerated the testing risks to get the time you thought you'd need for testing? Or padded your estimates because you "knew" you'd need contingency you couldn't get into the schedule otherwise? How about overplaying test coverage or inflating the efficacy of your team's ability to find bugs with your testing process?

Or maybe you downplayed a material fact. At a workshop I conducted recently on testing deceptions, testers said they frequently ignored bugs that were "out of [their] scope," saving time in reporting by convincing themselves that these bugs weren't important enough to bite somebody sometime.

Testers' deceptions are no less damaging-to the software and themselves-than any other kinds of deception.

What Can We Do about Testing Deceptions?
In every case, the answer is the same: Ascertain the facts, and tell the truth. That's simple to say, but of course it isn't always easy to do. Combating the self-deceptions of others can be particularly difficult. And people, especially managers, can have a lot invested in what they've said. But we have to try.

Here are some steps to help avoid deceptions:

• Find out the facts, preferably before a deception occurs . Those facts might be about the software quality, the pros and cons of test automation and the various options available, or why the product quality is holding up the testing-and therefore holding up the project.
• Present the facts straightforwardly and unemotionally .
• Never give in to pressure to lie .
• Be scrupulous in our own thinking and communications about testing . That means being open about our thinking and planning, and reporting the risks and problems we find.

Testers are paid to provide information about software quality. It's our job to tell the truth about testing and to see that misconceptions aren't proliferated.

What's your experience with deceptions or self-deceptions in testing? Have you ever fooled yourself-or maybe even consciously deceived? Have you had to address deceptions proffered by others? How would you advise your fellow-testers to deal with deceptions?