How to Jump-Start Inspection by Outsourcing

[article]
  • very good at finding memory leaks and null pointer dereferences, these defects ought to be found by these tools. However, these tools will not find these defects unless there is a test case that exposes the defects, and if you can find the defect earlier at less cost (e.g., through inspection), you're better off.
  • Misunderstood responsibilities. Many programmers operate under the ssumption that it is their responsibility to implement functionality, and it is the responsibility of the testing/QA department to find defects. To them, doing inspections would be doing the tester's job.

Outsourced Software Inspection Services
Outsourced inspection services, a new approach that has emerged over the past few years, may be a way to address many of the specific issues mentioned above. A typical outsourced inspection service consists of the following steps:

Step 1 . Collect source code . In this step you decide which code you want inspected, and you make a package that includes the source files and any header/include files that may be needed to compile the source files. The service provider may provide tools to assist in this process.

Step 2. Complete an application survey that gives the service provider the technical details needed to inspect your application. This includes data such as the target operating system, the language the application is written in, version number of the compiler, and so on. There may also be questions regarding preferences, such as "do you want to receive reports of null pointer dereferences in out-of-memory condition?"

Step 3. Submit the application files plus the survey to the service provider via secure FTP, through a Web site protected with SSL, or on a CD-ROM using a secure courier service. For extra security, the package may be encrypted.

Step 4. The service provider inventories the code, and supplies you with an inventory and possible cost adjustment.

Step 5. The service provider applies static analysis technology (see, e.g., "Value Lattice Static Analysis" by Bill Brew and Maggie Johnson in Dr. Dobb's Journal, March 2001) to inspect the code and produce a set of inspection points. The static analysis is parameterized by the answers to the questions on the survey.

Step 6. Trained engineers at the service provider remove false positives , trying to ensure that only real defects will be reported to you.

Step 7. You receive an inspection report containing detailed descriptions of the defects found, and a management report containing metrics on the inspection (and trend information if you've sent the same application before).

The first thing I'd like to clarify is that by nature, outsourced inspection services are complementary to other quality initiatives. Outsourced inspections are generally poor at identifying functional defects, which require knowledge of the domain the code is dealing with.

Nonetheless, it is an effective way of finding structural defects such as NULL pointer dereferences, memory and other resource leaks, uninitialized variables, bad deallocations, etc.

Second, since it is impossible to simulate every single execution path with every possible value, and because tradeoffs have to be made to reach acceptable false positive and false negative rates, outsourced inspections don't find every instance of these structural defects (just like manual inspections won't find every instance).

Jump-Starting Inspection by Removing the Barriers
Outsourced inspection services have a number of unique features, many of which address objections that are often raised in relation to the general concept of inspection.

No use of in-house resources. This is the point of outsourcing: apart from a little bit of effort to package the source code and fill in the survey, these services do not require any in-house resources. This addresses

About the author

Jasper Kamperman's picture Jasper Kamperman

Jasper Kamperman, Ph.D., is Senior Software Inspection Consultant at Reasoning, Inc. At Reasoning, Jasper has been a leading developer of the technology powering Reasoning's InstantQASM service. Jasper has a master's degree in Physics from the University of Utrecht, and holds a Ph.D. in Computer Science from the University of Amsterdam. He has published in ACM TOPLAS and several trade journals.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

May 04
May 04
May 04
Jun 01