The Value of Requirements-Based Testing

[article]

of what the user wants or needs and to document this need in some fashion that will help the builders and testers do their job. This document, or requirements specification, does not always have to be perfect because it will change as the design and code are created. Still, we should make every attempt to clarify ambiguities and omissions to avoid building in defects from the start.

Testers can play an important role in testing the requirement specification before the design and coding begin. Some people use word pictures, use cases, inspections, or some other technique. The method that we find effective for many organizations is called "preventive testing." We admit, this is a bit of a misnomer since preventive testing does not really succeed in preventing bugs, but helps find them very early in the development process.

The concept is deceivingly simple, and no doubt already in use by many of the readers of this article. High level test cases are written to "test" the requirement spec prior to the commencement of design and coding. These test cases will probably never be executed, their only role is to discover ambiguities, errors, and omissions in the specification that will ultimately be used to create the system. Some of the test cases will identify flaws in the requirements specification which will cause it to be updated, rendering the original test case obsolete. Yea, those test cases have already done their job.

Here's an example:

Take a typical (inaccurate) requirement specification-this one is for an ATM:

R1: A valid user must be able to withdraw up to $200 or the maximum amount in the account.

This is obviously a weak attempt at writing a requirement specification, but this is certainly not atypical. Developers try to solve problems. Testers are also problem solvers, but we also try to look for things that can go wrong. What can go wrong with the spec above? What is a valid user? Hopefully that is documented in another requirement, but if not we could create some test cases to challenge the term "valid" user. What does the word "or" mean? If I have one million dollars in my account can I withdraw it all? How many withdrawals can I make in a day? Can I withdraw an odd amount, say $158.27?

A few sample test cases:

TC01 Attempt to withdraw $800 from an account with $800 in it (i.e. the max amount in the account)

TC02 Attempt to withdraw $200 10 times in one day.

TC03 Attempt to withdraw $158.27

Notice that these "test cases" are not elaborate enough. They don't even have expected results since they will probably never be executed. Their purpose is merely to challenge the voracity of the requirements spec which will ultimately form the basis for the development effort.

If you had written the above test cases, you would have discovered defects in the requirements specifications that no doubt would have resulted in a clarification of the requirement specification. Oh, I know what you're thinking: Do we really need to do this? Surely no one would ever try to create code to withdraw pennies, nickels, dimes, quarters, etc. from an ATM. Unfortunately, things like that happen more often than we would like to think.

So far, we've talked about test cases based upon the requirement specifications that are used to validate the accuracy of that particular document. Do we also need to create test cases based upon the requirements specification that will be executed? I think so. I believe that it is important to create tests that demonstrate that the user can

About the author

Rick Craig's picture Rick Craig

Rick Craig is a consultant, lecturer, author, and test manager, who has led numerous teams of testers on both large and small projects. In his twenty-five years of consulting worldwide, Rick has advised and supported a diverse group of organizations on many testing and test management issues. From large insurance providers and telecommunications companies to smaller software services companies, he has mentored senior software managers and helped test teams improve their effectiveness. Rick is co-author of Systematic Software Testing and a frequent speaker at testing conferences, including every STAR conference since its inception. 

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

May 04
May 04
May 04
Jun 01