Testing Error Code


different types of error handlers:

Input filters can be used to prevent bad input from ever getting to the software under test. In effect, bad inputs are filtered by, for example, a graphical user interface, and only legal inputs are allowed past the interface.

Input checking can be performed to ensure that the software will not execute using bad input. The simplest case is that every time an input enters the system, the developer inserts an IF statement to ensure that the input is legal before it is processed; that is, IF the input is legal, THEN process it, ELSE display an error message. During this first attack, it is our goal to ensure that we see all such error messages.

Exception handlers are a last resort and are used to clean up after the software has failed as a result of processing bad input. In other words, bad inputs are allowed into the system, used in processing, and the system is allowed to fail. The exception handler is a routine that is called when the software fails. It usually contains code that resets internal variables, closes files, and restores the ability of the software to interact with its users. In general, some error message is also displayed.

Testers must consider each input that the software under test accepts and focus on erroneous values. The idea here is to enter values that are too big, too small, too long, too short-which values that are out of the acceptable range or values of the wrong data type. The major defect one will find with this approach is missing error cases-input data that the developer did not know was erroneous or individual cases that were overlooked. Missing cases almost always cause the software to hang or crash. One should also be on the lookout for misplaced error messages. Sometimes the developer gets the error message right but assigns it to the wrong input values. Thus, the message seems like nonsense for the particular input values submitted.

Finally, of pure nuisance value are uninformative error messages. Although such messages cause no direct harm to the user, they are sloppy and will cast doubt in a user's mind on the credibility of the software producer. "Error 5-Unknown Data" might have seemed a good idea to some developer, but will cause frustration in the mind of the user who will have no idea what they did wrong. Whether one is testing an input field in a GUI panel or a parameter in an API call, one must consider properties of an input when conducting this attack. Some general properties to consider are:

Input type : Entering invalid types will often cause an error message. For example, if the input in question is an integer, then enter a real number or a character.

Input length : For character (alphanumeric) inputs, entering a few too many characters will often elicit an error message.

Boundary values : Every numeric data type has boundary values and sometimes these values represent special cases. The integer zero for example is the boundary between positive and negative numbers.

Be prepared to find some spectacular bugs!

For more information and examples, see Chapter 2, How to Break Software (Addison-Wesley, 2002).

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.