One of the simplest ways to enhance QA's ability to manage the test process is to put the QA group's Test Cases under CM control. Besides being an industry best practice, putting Test Cases under CM control will also help to qualify your organization for CMM Level 2. Depending on the features and functions of your firm's CM tool, putting Test Cases under CM control can provide the following benefits:
- A central repository for all QA Test Cases
- Ensure that only authorized users can modify Test Cases (e.g. Testers, QA Manager)
- Checkout/checkin control of Test Case documents
- Versioning of Test Case documents
- Provide a life cycle, approval methodology, and audit trail for the modification of Test Cases
- Email notifications when Test Cases are modified
- Allow for the linking of Test Cases to Requirements
- Allow for the linking of Test Cases to Bug Reports
It's usually a good idea to define terms, so let's define a Test Case for purposes of this discussion. At my last employer, the test engineers had accumulated an inventory of nearly 500 pre-formatted Word documents that they used to document the test scenario for each business requirement. These documents contained the Test ID, Test Name, Description, Test Set-up, and step-by-step test instructions (i.e. Test Script) for each test scenario. These Test Cases would be executed in their entirety (as a regression test) or in part (as a final functional/acceptance test) prior to distributing a release to production.
Initially, the Test Cases contained the test results as well. The testers would copy the Test Case documents en masse as templates for each release, conduct their tests, and document their results in the release-specific copies of the Test Case documents. As part of the conversion to CM control, we decided to simplify the process by removing the test results from the Test Case documents. The test results were migrated to a separate Test Results Matrix (one matrix for each release). This step allowed us to migrate the Test Cases as a single set of templates into the CM repository.
With an Automated Test Tool (ATT), the Test Script or even the entire Test Case might be stored and managed within the ATT itself. We did not enjoy the luxury of an ATT, so the Test Cases were maintained manually. If your shop has an ATT that stores the Test Scripts internally, then there may still be a need for a Test Case document to serve as a "wrapper" for each Test Script. Otherwise, the entire Test Case may end up in the ATT, and CM would not need to manage the Test Cases at all.
As a Configuration Manager, I worked with the testers to design and implement the Test Case repository and maintenance process. The steps to convert Test Cases to CM control are roughly identical to the steps one would follow to convert a software application to CM control. Assuming your CM tool can manage Word documents just like any other configuration item, the steps are:
- Identify and resolve conversion issues (as noted above)
- Design and build an appropriate CM repository structure (i.e. folders/subfolders)
- Load the Test Case documents into the CM repository
- Implement native OS security rules to restrict external access to the repository
- Implement internal CM security rules to restrict functions by group/role
- Implement process rules, development areas, and other features (e.g. email notifications, approvals) to establish a life cycle for maintenance of the Test Cases
- Provide some quick, informal training to the QA staff.
The process was simple and straightforward. Each tester had a development area on their own workstation, and they would checkout Test Cases as the need arose (e.g. a Requirements change), make their changes, and then checkin the modified Test Cases. Our CM tool had a nice interface to MS Word that allowed the testers to modify the Test Cases without leaving the CM client, and it used the MS Word "Track Changes" feature to highlight differences. After an approval by the QA Manager, the CM tool would automatically complete