CM The Next Generation: Test Case Management

is especially the case if the granularity of a test case is small.  A good CM system helps to manage aggregates of small test cases so that individual tests can be represented as individual test cases while still allowing the management of less
granular clusters.

A fine granularity has the side effect for test cases that changes to the test case repository are generally of the form of adding in new test cases, rather than testing existing
ones.  This obviously depends on the test specification complexity. But generally, the user interface must be optimized for adding and tagging/organizing test cases.  This is not to say that full change and version control will not be necessary.  But often, the original test case is as important as the revised and must be maintained as a separate test case.  Thus, there is less revisioning and more management of test cases.

Automated Test Cases
Test case automation is essential to quality. But CM tools can help by allowing easy packaging of an automated test suite based on the build and the features and options contained within it or targeted for testing. The CM tools can also help to automate the collection of test results, especially when it can inject the test case identification
into test suites.  This is highly dependent on both the types of tests and the test engine technology.

The Test Environment
The CM tool can track not only the test case but the conditions for running the test.  More advanced systems should allow specific environments to be defined that can be shared by multiple test cases.  When test environments and data setups can be tracked separate from the test case itself, there is an economy of specification, and the resulting economy  of management effort for changes to the environment.  At the same time, the tools should allow a complete test case description to be presented entirely as a single set of instructions.  This is especially important when addressing failed test cases individually.  This occurs when the failed test case is fed to the development team for diagnosis, starting with reproduction of the test case that caused the failure.

Scalability
When the granularity of the test cases is fine, the number of test cases can easily grow into the tens of thousands.  It is important that CM tools be able to scale to these requirements - especially considering that it is desireable to track multiple products in the same repository to facilitate sharing of test cases and requirements across products.

Test Case Management - An Integrated Solution
A next generation CM tool will consider test case management as part of a full ALM solution, and not as an isolating test case management function.  Role management, generation of problem reports, coverage of requirements, traceability to builds, and hence to the changes in each build - these are all essential components of test case management.  A solution that is designed in the context of an end-to-end lifecycle  will yield real advantages.

The user interface is critical, but not just from a verification team perspective.  The developers need to be able to navigate test results.  Management needs to navigate the
data but also correlate it with changes, problem report arrival rates, audit requirements, builds, etc.  A combination of object-oriented capabilities, management dashboards and traceability navigation aids should be an important component of the UI.  Object-oriented operations should be applicable to hierarchical displays to facilitate reporting, metrics, test case extraction, etc. on hierarchical subtrees.