Managing the Testing Process

[article]
Opening the Black Box

necessary to run multiple tests from several areas to completely test software components. Following this process can reveal problems in areas of the software that have been passed over in earlier tests. This is also why it is important to see testing from the code perspective.

When considering functionality in testing, it is important to think also about how the functionality is being tested. It is common to develop a relatively "quick-and-dirty" set of tests that exercise the functionality of the code in a "nominal" way, to make sure that all of the pieces are in place and a given version of the software is worth the added effort of doing a complete set of tests on. The complete testing suite should be targeted at testing the functionality to a more exacting standard, in essence deliberately trying to push the software to its limits. This type of functional testing includes load, performance, and endurance testing where appropriate.

Understanding how tests and problems map to the code is necessary to understand how far along testing has progressed, what remains to be tested, and how complete the testing has been. A detailed knowledge of the code is not required for the entire project team. However, management does need to understand how much of the code has been tested, how much remains to be tested, and how many modules are affected by problems discovered during testing. With this information, informed decisions can be made about the consequences of fixing or leaving problems alone. Armed with such information about test coverage, it is possible to understand how thoroughly the software is being tested, and how much of the software is being exercised by the tests. Often this comes as an unpleasant but enlightening surprise the first time it is measured.

When considering code coverage as a measure of the completeness of testing, it is often important to decide what a reasonable level of coverage is for the software being tested. There are many different ways to measure completeness of code coverage, some of which are extremely difficult to achieve for a typical piece of software. These higher levels of coverage are commonly used for software with a correspondingly high level of criticality-either software that is widely used, such as operating systems or Web servers, or software that is used in very critical functions, such as medical devices or aircraft controls. Regardless of the project, however, teams need to decide on a level of coverage that is most appropriate to the particular software under test.

Measuring the impact and consequences of problems that arise during testing is a critical step in the process. This should include how much of the software is affected by a given problem, at what point during testing a problem was found, and what kinds of problems regression tests are attempting to uncover. This information is needed to monitor the overall progress of the software through testing and to make informed decisions about software release. So by combining all of the different perspectives of schedule, functionality, code, and problem resolution, it is possible to understand and manage software testing, rather than treating it as a black box.

The catch to this is that while these elements of information are usually present in a software project, it is often difficult to find them. Also, typically such information is presented at a level of detail and terminology understandable only by testers and developers. Recently, test management tools have been developed to collect and present this information in the different forms that are better targeted to different members of the software development team. For

About the author

TechWell Contributor's picture TechWell Contributor

The opinions and positions expressed within these guest posts are those of the author alone and do not represent those of the TechWell Community Sites. Guest authors represent that they have the right to distribute this content and that such content is not violating the legal rights of others. If you would like to contribute content to a TechWell Community Site, email editors@techwell.com.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Sep 22
Sep 24
Oct 12
Nov 09