The Importance of the Using Right Test Techniques

[article]
Investing in Software Testing, Part 4

called dynamic tests. Structural tests are one major example of dynamic tests.

Structural tests are based on how the system is built. Some people refer to structural tests as "white box" or—perhaps more appropriately—"glass box", because the tester looks inside the "box" (the system under test) and derives tests from this knowledge. Structural tests are most typically applied to individual components and interfaces, being particularly effective at discovering localized errors in control and data flows.

One elegant and powerful form of structural testing involves reusable, automated test harnesses for the system under test. Using this harness, programmers can create structural test cases for components as they code them. Programmers can then check these tests into the source code repository along with the component. A cleverly-constructed test harness will execute this ever-expanding, automated regression test suite each time the source code is built into a new release. A colleague of mine, Greg Kubaczkowski, and I described a specific example of this kind of structural test harness in the article "Mission Made Possible," available on www.stickyminds.com.

Another form of structural testing involves the creation of custom test data. Testers creating such data often must understand the underlying database schemas or designs, especially when working on carefully crafted data sets that probe the boundaries of data validity in one or more ways. In addition to testing individual components and data flows, with careful design we can reuse such test data for other kinds of tests, a topic we’ll revisit later in this series of articles.

Because structural testing requires an understanding of the internals of the system under test, programmers and other development engineers can do an excellent job of structural testing. In enlightened organizations, you may find the programmers charged with structural testing, but ably assisted by a test team that includes programming experts and test toolsmiths, helping to set up test harnesses. Testers with experience in automation also tend to know tricks for developing reusable and sharable test cases and data. This can be especially useful when it comes time to test the interactions between components.

No matter who does the structural testing, they will need to understand some fundamental test design techniques to do a good job. For example, one of my client's programmers were talking about structural tests that "tested everything that could break" when we started working with them. In addition to building the test harness described in "Mission Made Possible," my associates and I taught these programmers effective techniques for designing structural tests that find bugs, such as boundary analysis, condition coverage, basis tests, and the like. Books such as Boris Beizer's Software Test Techniques and Bill Hetzel's The Complete Guide to Software Testing discuss these and other structural testing techniques in more detail. Those charged with structural testing should understand and apply these methods.

Behavioral ("Black Box") Testing
Another form of dynamic testing is perhaps most familiar to test professionals, the behavioral or "black box" test. Behavioral tests, as the name "black box" implies, focus on what the system does, not how it does it. Some test professionals believe that inside knowledge of how the system works can actually make a behavioral tester less effective, because this understanding can interfere with the ability to test the system in ways that the user would actually use it. However, I've found that I can minimize this problem through careful test design and execution, and insight into how the system works can help behavioral testers find more bugs and write more perceptive bug reports.

Behavioral testing most typically focuses on global issues of workflows, configurations, performance, and so

About the author

Rex Black's picture Rex Black

Rex Black is President and Principal Consultant of RBCS, Inc., a consultancy that provides testing experts worldwide, serving clients such as Bank One, Cisco, Hitachi, IMG, and Schlumberger in consulting, training, and hands-on implementation. He has written Managing the Testing ProcessCritical Testing Processes, and numerous articles, along with presenting papers and keynote speeches at international conferences.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!