Testing Error Code

There are two types of code that developers write-functional code, and error code. This column is about understanding the unique challenge presented in testing error code. We'll begin by discussing error code from a developer's perspective. Then we'll explore the best ways to attack this challenge as testers.

There are two types of code that developers write. First, there is the code that gets the job done-which we'll call this type of code functional code because it supplies the functionality that satisfies user requirements. Second, there is code that keeps the functional code from failing because of erroneous input (or some other unexpected environmental condition). We'll call this type of code error code because it handles errors. For many programmers, this is the code that they are forced to write out of necessity, not because it is particularly enjoyable.

Writing both types of code simultaneously is problematic because there are context switches that must be made inside the head of a software developer between the two types of code. These context shifts are problematic; they require the developer to stop thinking about one type of code and start thinking about the other.

Consider Johnny, a hard working hypothetical developer, writing a new application. Johnny begins by writing the functional code, maybe even going so far as using something like UML to fully understand the various user scenarios that Johnny must code. Good, Johnny. Indeed, good programmers like Johnny can find a wealth of information out there to help them write good functional code. The books all address it, the tutorials address it, and there are many useful published examples to work from.

But, what happens when Johnny realizes the need for error code? Perhaps he is in the middle of writing or specifying some code object when he decides that, say, an input needs to be bounds-checked. What does Johnny do? One choice for Johnny is to stop writing the functional code and write the error code instead. This requires a context shift inside Johnny's developer-head. He must stop thinking about the user scenarios and the functional code that he is implementing, and start thinking about how to handle the error. Since handling errors can be complicated, this may take him some time.

Now, when Johnny returns to the task of writing the functional code, his mind has to recall what he was thinking about when he last put it down. This context shift is harder than the first, given the myriad design-decision details and minute technical details that go into writing any nontrivial program. You see the problem: poor Johnny has had to endure two context switches to handle a single error. Imagine how many such context switches happen writing even a small application.

Another choice for Johnny would be to postpone writing the error code in order to avoid the context shift. Assuming Johnny remembers to eventually get around to writing the error code, he's probably going to have to spend some time recalling the nature of the error event he's trying to write a handler for. So now Johnny is writing the error code without the benefit of context. Writing error code is problematic no matter how you face it. And therefore a ripe place for guys like me to look for bugs. So now let's look at the testing perspective, how do we approach testing error code?

Forcing error messages to occur is the best way to get error code to execute. Software should either appropriately respond to bad input or it should successfully prevent the input from ever getting to the software in the first place. The only way to know for sure is to test the application with a battery of bad inputs. There are many factors to consider when testing error code. Perhaps the most important is to understand how the application responds to erroneous input. I try to identify three

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.