The Journey Through Traceability to Process Control

you write a book for each requirement). And the test matrix won't tell you if you've covered your requirements. You'll get a large number of test cases against each requirement, with no indication as to whether or not they completely cover the requirement. Keep requirements simple so that the coverage of the corresponding test case(s) can be readily seen.

In turn, keep your test cases specific. Don't have a test case that says "Test the User Interface" or even one that says "Test the File menu". It might be ok to have one for a particular menu button, or you may have a number to test the various significant options. Either way, coverage would be identified a lot more easily. If you have a single test case covering several options on a button, remember that the test case will be marked "Failed" if any of the options fail. So it might be better to have specific test cases. This is most easily facilitated by a hierarchical test case definition, where you can decompose a test case into sub-cases if necessary - much like decomposing an activity into several tasks.

The second pre-requisite is to have your test process feeding it's data into your test run repository. Ideally, your test run repository will track a test case against either a specific build or a sequence of closely related builds. If the process doesn't feed the data properly or the repository doesn't track it properly, your traceability will be compromised.

I would recommend that you clearly segregate your test cases into specific areas (see my May, 2005 CM Journal article). In particular, there are different levels of requirements - Customer requirements, Product Requirements, and Software/System Requirements, that need to be dealt with separately. Customer requirements are tested to verify that you meet the needs of the customer. Product requirements are tested to ensure that your product meets its specification. And Software/System Requirements are tested to ensure that your design was properly implemented.

Configuring Your Process

Your SCM tool must allow you a good level of customization so that you may configure your state and task-based workflows. Your process is going to evolve. If you can't embody it in your CM tool, you'll never reach levels 4 and 5 of your capability maturity model. You need to be able to change it while maintaining control and improving traceability.

Process configuration means configuration of your object workflows, your data schema, your roles, your user interface, your rules and triggers, your data access permissions. If your SCM tool allows incremental changes to these without any end-user down time, all the better. You can continuously improve your process. You can define your process loosely and then tighten up the rules. Or you can over-tighten the rules and then relax them.

Avoid an SCM tool that requires you to "stand down" while you script a new process.

Hard Rules vs Policy vs Convention

I like to look at process at various levels: rules, policy and convention. All are needed.

Let me give you an example of a hard rule versus a policy. The CM tool forces you to use a change package to check-out/check-in files. This is a hard rule. You won't get files in the system without a level of traceability. But if change packages are just a policy (e.g. they are optional database lists that link revisions together), be careful. You're likely to find some revisions not included in a change package at some point or other and your basic assumptions underlying all of your queries may yield incorrect results.

I like the following loose