Are there elements of glass box testing and black box testing that overlap and can be helpful to either type of tester? One developer looks at the gray area between black box testers and glass box testers and comes up with some surprising results.
A short time ago I had to prepare a testing course for software developers who were responsible for both unit testing and system testing and programmed in a relatively unpopular programming language based loosely on Pascal. The information I found when searching for course materials pointed back to Java and C++, but the development staff with which I was working did not know or use those languages. In fact, the best material I could find was from a Cem Kaner course called "Black Box Software Testing," or BBST.
I posted on a software discussion list, actively asking people what they thought of using BBST materials for software developers and soliciting recommendations for open source materials. The answers I received were consistent: "No, no, no. You can't teach a course in black box testing; get some material on glass box testing," and then they would recommend a course that was in Java or C++. Occasionally someone would recommend a book or class and say something like, "I have heard good things about this but haven't read it"—and it would wind up being in Java or C++. The existing material simply would not be helpful to the development staff, and I would keep coming back to BBST.
I next asked why black box testing was inappropriate. The typical response was "It's black box, genius. That's not the right kind of testing." Others would say, "Well, black box testing is for people who can't see the code—it's not really for the developers."
Both answers were unsatisfying to me. They were definitions of the term, not reasons to reject the course material. One or two people mentioned that black box testing doesn't include statement or path coverage metrics, which is true. But it turned out that this particular customer had no requirements for coverage to be measured, and the BBST course actually offered just enough information about coverage and coverage metrics to meet my needs.
So I looked at the basic skills that the black box course covered: critical thinking, general systems thinking, identifying patterns of software failure, issue isolation, and investigation skills. These are skills that any tester might want to grow, and I found a huge overlap, regardless of box color (see Figure 1).
There were areas of the BBST that I wanted to skip. Bug advocacy and how to write a good bug report become less important if the person who logs the bug is the one who is going to fix it, and bugs found in unit testing might not get logged at all. Still, it seemed foolish to throw out the entire course when I could just toss out a couple of chapters.
This got me thinking about what it takes to be a good black box tester and what it takes to be a good glass box tester.
The developers who would take my course already have the skills on the left. Because equivalence classes work just as well on functions as they do on GUIs, the developers could stand to benefit from any black box material that would cover the skills in the middle.
So, if I can screen things from the far right out of the training materials and the developers already have the skills on the left, then the differences between black box and glass box testing (or the value of distinguishing the difference) greatly decreases.
If the difference between the two is not valuable, then why are we splitting them up? On this project, splitting hairs over the color of the box was actually harmful. It caused me to shut out potentially valuable