positive. Data driven automation is the preferred method of automation whenever it is practical.
The cost reductions are no longer projections; it has been possible to reduce the size of the automation team by eliminating contractor resources and re-assigning team members to test staffs. This reduced the team’s overhead and added highly skilled test resources to several test teams.
Over the ten-month period from September 2001 to July 2002 the maintenance percentage for the team was 17 percent. A tribute to the smaller scripts and the methods employed when creating automation.
In addition to these changes in the method of automation, changes to components of the process were implemented. A Project Status DB was created which replaced the Excel spreadsheet as a tracking tool. The DB can instantly provide current team assignments; projects in progress, projects expected, projects completed, and offers the ability for the user to do Ad Hoc queries.
A Script Result Reporter DB was created that accepts the automated test results from the automation tool. The tool writes to the DB and those results are available to the user in the form of an Excel spreadsheet or a Word document.
Sizing and Priority algorithms and tools were improved. A project estimation tool was built to use during the Test Assessment phase of planning.
It is projected that by the end of this calendar year, three members of the former nine-member automation team will remain as a center of expertise and that the majority of the maintenance will be shifted to the test staff. The first training session has been conducted for a group of test analysts in accepted methods of script repair. A standards document was provided as a guide.
Measuring the Results
Several analysis efforts over the past eighteen months have revealed significant contributions towards improving the test process and moving the product to market more quickly.
Other efforts have analyzed the cost of automation as compared to the benefits derived and those results were positive. As the process grew and was refined, the efficiency grew and the size of the staff was reduced, the time required to complete the task was reduced. And of course, costs were reduced.
These charts are a depiction of the cost of testing of one group of products over a span of a year, it displays the cost of testing with and without automation, shows increased test scope, and identifies the perceived payback percentages during the test process.