How to allocate testing resources is a difficult issue. Sometimes it is hard to justify testing in projects, especially small ones. As quality control professionals, we know that testing is essential to guarantee product quality. This article shows a new approach to the testing methodology, which is applied at the organization I work for. The new approach intends to answer some of the questions in order to ensure that testing resources are appropriately allocated in a project. The main goals are to reduce testing costs, keep the product quality high, and anticipate the delivery date.
Allocating an independent testing team to test a software product is often a controversial issue, mainly because development managers are frequently not interested in paying for these services due to budget and time constraints. On the other hand, senior management generally is interested in these services-senior managers know the importance of testing and how the tests protect the organization and their products from embarrassing failures.
One of the most common excuses from the development managers is that independent testing is too expensive and lengthy-the project schedule has to fit the customer's schedule. Thus, there is no time left for independent testing activities. The organization I work for faces this constantly. As a systems test manager, I have been asked how I can reduce the timeline for testing a software product. I am faced with this dilemma largely because the projects developed at my organization have changed a lot recently. In our first years, we were asked to develop commercial, off-the-shelf software (COTS). Adding systems testing to a COTS project was not difficult and no one questioned it. All groups involved in developing the project were interested in testing. The reason was that the project schedules were long (eighteen to twenty-four months), they had many (six to ten) engineers working on them, and several artifacts were produced during development time.
Under those conditions, we applied a complete set of extensive tests and techniques, following our testing methodology fully. We worked together and provided the development teams with suggestions and comments for the projects during the test cycles. There were no customers to specify details for the COTS products. The testers were simply another team with experience and another perspective.
Then the Internet arrived. Everything changed, including the customers, and several tools have become available to help the development teams build powerful solutions faster and cheaper.
My organization has followed the wave. Nowadays, we have been developing Internet solutions with Documentum, BEA, and FileNet Panagon tools for telecom, chemical, and energy supplier companies, locally and offshore. We have also added CMM Level 3 to our skills. It has been one of the greatest challenges for the organization to add the new technologies and skills since 1998.
Everything has had a large impact on the way we run testing activities. The effort and time for developing the products have dropped dramatically. The development team size was reduced to three to four engineers. The effort usually ranges from eight to twenty personal months (PMs), and the project length is shorter, from three to six months. Now, some new questions have arisen to join the old ones:
- How can the customer pay 50 percent of the project costs for qualifying a software product?
- How can a systems test be added to a project in this new scenario?
The new questions presented new challenges for the systems test group. Functioning independently from the rest of the organization, we had to define how we would work in this increasingly cost-sensitive scenario and not be pressured.
Our task was to propose a new approach to our testing methodology to the organization. The proposal would guide us in defining how the systems test group should work on future projects. In addition, product quality would not suffer through reduced testing costs, and the project delivery date would be anticipated (the famous QCD!).
The Resource Allocation Process
First we properly defined the new scenario and problems we faced. We noticed that the differences among the projects were related to the effort, not to the technology or to any other project characteristic. And the major problem was estimating