Building a Configuration Management (CM) Capability for Test

Test is a very valuable discipline in the engineering field. For years, there has been a need for test plans to ensure that systematic testing occurs on projects, test designs to ensure the test approach is reflective of the project under development, and test procedures to ensure testing is being performed in a repeatable manner. More recently, the test function must be able to support more and more complex application development. With this comes the need for test automation and more sophisticated test tools for more efficient testing. This implies more test artifacts that must be managed which include numerous test scripts, test cases, test data, test results.

As more test items exist, there is a tendency for them to evolve due to the changes to requirements and code and, therefore, must be managed effectively. When there are an increasing number of test items, this increases the risk of failing to accurately track all test items, particularly when this is done manually. This also increases the risk of over-writing changes to test scripts due to a lack of control and increases the risk of missing important test results on how software is tested compared with past or future testing results. As test teams invest more time in effort in the test process, test tools, and test artifacts, there should also be an investment in the control provided on these items. This is where Configuration Management (CM) can help.

Where CM can help a Test Team

CM benefits Test by ensuring the code that is migrated to test is properly version controlled and is clearly identified and baselined. However, there is another way that CM may benefit Test. This is by establishing Configuration Management (CM) for Test items. Today, this is as important as establishing CM for Development. As requirements are changed (added, modified, or deleted) throughout an application lifecycle, this constitutes the creation of new test scripts, the modification of existing test scripts, or removal of test scripts within a baseline. Therefore, effectively managing test items becomes very important since the application and the associated test items continue to change.

Also, as more automated test tools like Mercury’s Quick Test Pro, LoadRunner and Rational’s JTest get used, CM would be useful to help recognizes all of the artifacts created. This is particularly true since each of these tools create multiple files in multiple directories for a single test. CM can help mitigate and reduce the risk associated with change by helping a Test team baseline and manage their test items and changes to them. It can also improve communications within a Test team when everyone knows where to look for test items and changes can be quickly identified. What are some ways CM can help in the test area?

• CM helps align the correct version of a test script with the correct version of code. Often requirements will change (e.g., get added, modified, or deleted) and this initiates a change in the code. This then may require a change to test scripts that trace back to the requirements they test. This helps ensure quality is systematically improved. Using the incorrect version of a test script will produce test results that can be misleading and lead to unneeded rework.
• CM helps track the test results so that improvements or regression can be identified. Either a missing or an incorrect copy of the test results can result in incorrect comparisons and lead to additional work. This is particularly true with capture and playback tools that record the events performed during a test session into a file that can be replayed against the