What You Don't Know May Help You

[article]
Summary:

Some testers take it upon themselves to learn as much as possible about the inner workings of the system under test. This type of "gray box" testing is valuable, and most testers have the technical wherewithal to grasp much of what's going on behind the scenes. But it's important to recognize that sometimes "ignorance is strength" when it comes to finding problems that users will encounter.

When it comes to software testing, ignorance can actually be a source of strength. In particular, software testers who are not familiar with how the system has been put together can be in a better position to find certain bugs than testers who may be more familiar with the system internals.

Testers are often treated as second-class citizens, and this makes many of them eager to learn as much as they can about the software they are testing in order to prove their technical competence. I don't wish to discourage this learning, but I do wish to recognize the maximum benefit of the inherent unfamiliarity that testers have when they are new to a project.

It is well known that the value of independent testing partly comes from having a set of fresh eyes on the software product. They are more apt to try things that might fail or notice problems that others have overlooked. Testers who are new to a project—with the least knowledge about the mechanics—bring the freshest sets of eyes. Let's look at how to get the maximum benefit from these fresh sets of eyes.

I'm a believer that greater emphasis must be placed on making our systems more forgiving and less enigmatic. Ultimately, our software products will not be successful if we expect our users to have to understand the inner workings of our software. Therefore, testers without this understanding can teach us a valuable lesson about how our software will be used. I find that there are several areas of testing in which "ignorant" testers can be helpful.

Usability
Testers unfamiliar with the inner workings of a product can often be very helpful in identifying usability problems. They might notice that the program flow is confusing or that terms used are hard to understand. Smart companies regularly put new employees through usability tests. This gives the company the benefit of the new employees' fresh perspectives, gives the employees insight into how the products they are going to help develop will be used, and puts everyone on notice that design is important. Even if you don't have such a program, you can still make sure that new testers are given a chance to express their observations regarding product usability.

Installation and Setup
Software installation and configuration are areas that are often handled late in the development process. As a consequence, early testers often must learn how to manually install and configure the software. Workarounds are common, perhaps requiring the manual copying of certain files or the manual creation of particular accounts or data sets. Testers who come to a project later won't have been trained to avoid these problems and are thus more likely to stumble across installation problems that had been deferred.

Error Handling
New testers are less likely to know how the software is "supposed" to be used and therefore are more likely to stumble across errors. It is important for software to handle errors appropriately (give appropriate notice of invalid input, provide options for recovery, and ensure that no data is lost). Error handling code is always a good place to look for defects. Informed testers will want to plan to exercise all error conditions, but the most important ones to check are the ones that the programmers didn't anticipate. "Uninformed testing" is one good strategy for finding them.

About the author

Bret Pettichord's picture Bret Pettichord

Bret Pettichord is an independent consultant specializing in software testing and test automation. He co-authored Lessons Learned in Software Testing with Cem Kaner and James Bach and edits TestingHotlist.com. He is currently researching practices for agile testing. Contact him at www.pettichord.com or bret@pettichord.com

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!