This paper discusses the fundamental approaches to establishing a test strategy and the varying levels of the test plan.
Test planning is one of the keys to successful software testing, yet it is frequently omitted because of time constraints, lack of training or cultural bias. Indeed, a survey taken at a recent STAR testing conference showed that only 81% of the companies participating in the survey completed test plans (and our experience has shown that many of those 81% are calling the testing schedule a plan). Testing without a plan is analogous to developing software without a project plan and generally occurs for the same reason: pressure to begin coding (or in this case testing) as soon as possible (i.e. why isn't Sam coding yet?) Many organizations measure progress in development by lines of code delivered and in testing by the number of test cases run. While these can be valuable measures, they don't recognize planning as a worthwhile activity.
Test planning can and should occur at several levels (For our European readers, the word level is used rather than stage). The first plan to consider is the Master Test Plan. The purpose of the Master Test Plan is to orchestrate testing at all levels (unit, integration, system, acceptance, beta, etc.). The Master Test Plan is to testing what the Project Plan is to the entire development effort. In fact, the Master Test Plan can be a separate document or could be considered part of the project plan. Test managers should think of the Master Test Plan as their major communication and control device. Test planning is a process (that ultimately leads to a document) that allows all parties involved in the testing process to proactively decide what the important issues are in testing and how to best deal with them. The goal of test planning is not to create a long list of test cases, but rather to deal with the important issues of testing strategy, resource utilization, responsibilities, risks, and priorities. In fact, test cases don't even belong in the test plan, but rather should be placed in test design specifications.
In test planning, the process is ultimately more important than the product. Discussing issues of what and how to test early in the project lifecycle can save a lot of time, money, and disagreement later. I once had a consulting assignment at a major American company where I was supposed to help them create their first ever Master Test Plan. Following up with the client a few months later, the project manager told me that the creation of the Master Test Plan had contributed significantly to the success of the project, but unfortunately they hadn't really followed the plan or kept it up to date. "Let me get this straight," I responded. "You didn't use the plan, but you felt that it was a major contributor to your success…can you explain that?" The project manager told me that when they began to fall behind they began to dispense with much of the project documentation, including the test plan (sound familiar?). But because they had created the plan early in the project lifecycle, many testing issues were raised that normally were not considered until too late to take action. The planning process had also heightened the awareness of the importance of testing to all of the project participants. Now I believe that keeping test plans up to date is important, so that is not the purpose of telling you this story. Rather, I'm trying to stress the importance of the testing process not just the document.
In addition to the Master Test Plan, it may be necessary to create detailed or Level-specific Test Plans. On a larger