- (given equal testing), and in extreme cases may need refactoring or rewriting.
We find it helpful to present this data in graphical form, as shown in Figure 5. Sponsors and upper management will get an overview of the bigger picture, whereas team members will be able to compare their results with those of their fellow team members.
Figure 5: A typical graph showing the number of issues found by deliverable
It's also important to keep track of test results over time. This gives you a historical view of how fast deliverables are being delivered and stabilized. We record total test results (passed, failed, not tested) on a weekly basis (see Figure 6), and present the results in graphical form (see Figure 7).
Figure 6: Historical data: test completion over time
Figure 7: Historical data: test completion over time
What this approach does not do
This approach can complete--but certainly does not replace--traditional project-progress tracking and reporting. In particular, this approach is purely deliverable-oriented in that it blissfully ignores all notions of delay, costs, resource consumption, critical paths, and so on. These notions can and should be managed using other techniques such as Gantt diagrams, PERT diagrams, earned-value, and project management tools such as MS Project. Indeed, it is important to give upper management, team members, and the project sponsor as complete a view of project progress as possible. Test-based deliverable status is an important and easily understandable facet of project reporting, but delay, cost, and task-oriented views are equally as important.
Often, project progress is uniquely measured using a project plan based on estimations of the remaining work for each unfinished task. In our latest project, we found that test results at a relatively low level could be used to give another, more objective view of project progress. This type of reporting also helps developers focus on tangible results and deliverable-oriented work.