Mission Made Possible

Your mission, should you choose to accept it: A client needs to implement an integrated, automated unit, component, and integration testing process. The system development teams are spread across three continents. They use multiple platforms. Their system architecture is interdependent and complex. Some tests need to be run on an application server. Both static and dynamic testing are required. The process you design must be simple to use, not an additional burden for busy developers. It should be automated and easily integrated. By the way, only three designers have been assigned to this mission. You have six weeks. Good luck. This message will self-destruct in twenty seconds...

Ever feel like you've been thrown into an episode of Mission: Impossible ? Even with test development tools at our disposal, designing and executing early test processes is never easy. Throw in short timelines and complex systems, and it can seem truly impossible. But this vital mission must be performed. Though this is only one unique situation, the problem-solving tools and lessons learned have universal applications.

Challenge 1: Diverse Group, Multiple Platforms
The geographical distribution of the project arose in part from the fact that the company was growing through mergers and acquisitions. Testing, especially integration testing, was critical because the diverse and still-coalescing team would be learning each other’s working and programming styles during the project. This created opportunities for miscommunication—and bugs. The team also had to contend with three different platforms: Linux, Solaris, and Windows. The tools we chose had to be flexible across these multiple platforms.

Challenge 2: Complex Architecture
Architecturally, the system under construction was composed of six layers (see Figure 1). Each layer added additional services and capabilities used by the layers above it. In turn, each layer used services and capabilities provided by any layers below it. The increasing functionality and interdependence culminated at the GUI layer. The layers were built from components that provided one distinct service or capability, often by using services and capabilities provided by components at their own layer or below. The components, in turn, were built from units. A unit was the smallest distinct object that could be checked into the source code repository, written by a single programmer.

This system design, while elegant, not only constrained the definition of the testing tasks, but also the objectives and sequencing of those tasks. The technical challenge, then, was for the test tool to be able to support the following three distinct categories of testing tasks (shown in Figure 1):

• Unit testing The testing of each unit, performed by interacting directly with its public and private interfaces. This testing would reveal logic and data flow errors in the basic building blocks of the system itself.
• Component testing The testing of each component, performed by interacting with the services and behaviors of the component under test, which might in turn interact with other components. This testing would reveal problems in the services and behaviors of the constituent system components.
• Integration testing The testing of two or more components, performed by interacting with the interfaces and behaviors of these collected components. The collected components could all reside in the same layer or in different layers. This testing would reveal incompatibilities and bottlenecks between related and dependent components.

Because of the three testing tasks the tool was to accomplish, we referred to it as the "UCI Test Harness."

Challenge 3: Interdependency of Test Sequencing
The constraints of the system design also influenced test task sequencing. The UCI Test Harness could test units independently of any other unit or component. However, before a component could be tested, three