When the music changes, so does the dance.
Most of you will agree that software testing has changed from many perspectives over the period of time. It has changed in the way testing is being carried out; also, the importance of the phase itself has increased.
Most importantly, software testing has emerged as an area with a huge business potential. A few years ago, software testing was considered just as a phase of software development, which it exits to be at present also. But, as the software community has matured with time, it has discovered the importance of leasing software testing to third-party vendors. Now, we see that many companies get their software developed, and tested by different vendors. As a result, we see a lot many Software Testing Projects in the market.
With this development, it becomes important to understand the life cycle of a testing project.
This paper is an attempt towards
- Understanding that testing projects are different from development projects, and finding out the major differences between them
- Understanding the life cycle of a testing project.
- And identifying key areas needing change for better execution of testing projects.
Note: The paper does not aim at explaining technical matters related to testing.
The need to understand STPLC
We know that testing is a phase of Software Development Life Cycle (SDLC). But, when testing is leased to a third-party vendor, it becomes a project in itself for that vendor. And, it is very natural that a testing project is treated like most of the other development /maintenance projects. Most of the companies would be following a certain customized set of standards for executing projects, and would tend to enforce the same set of proven standards on the testing projects too.
But, at times they may find that the standards do not seem to work as well as they worked for other kind of projects.
Here arises the need to understand how testing projects are different from other projects, and then, going further, how they should be tackled for better results.
The most evident difference between a development project or a maintenance project, and a testing project is the phases of the projects. The phases of a testing project are completely different from the phases of a development or a maintenance project. This is the reason why methodologies, and standards need changes for better execution of testing projects.
The life cycle of a testing project is completely different from the life cycle of other projects. So, let us first understand Software Testing Project Life Cycle (STPLC).
STPLC
Understanding Software Testing Project Life Cycle is not as difficult as the term itself. Most of you, who have worked on testing projects, know it very well either consciously or unconsciously.
For understanding STPLC, we just need to divide a testing project into logical phases. The phases of a general testing project are:
- Requirement /Spec Understanding
- Test case creation
- Test execution, and defect reporting
- Defect fixing by development-vendor
- Retesting fixed defects, and reporting
Let us understand each phase in detail:
1. Requirement /Spec Understanding
The phase
The first phase of any software-testing project is "requirement understanding" or "specification understanding". This phase involves going through the requirements document/functional specification document, and understanding the expected functioning of the software to be tested.
Adapting to the changes:
The requirements /specifications should be well documented. The client or developing vendor might have prepared these documents. If these documents are not available or not up-to-date, some mechanism should be devised to clarify what you really understand about the project you are going to test, and what and how you are really going to test it.
It is important to note that this phase becomes more important in case of testing projects as compared to testing phase because of the following two reasons:
- As a different company is carrying out software testing, testers may find that query resolution, and requirement understanding takes comparatively more time than if testing is being carried out by the same company who developed the software.
- All of us, who have worked on development projects, know the importance of "Requirement Analysis" in SDLC. If a requirement is analyzed incorrectly, and is detected while development/testing, the cost to be paid is enormous. Similarly, while testing, if a requirement is understood incorrectly, the tester may create incorrect test cases, execute them, and may even end up reporting incorrect defects, resulting in wastage of efforts.
The bottom line is that, this sub-phase is one of the major difference between a testing phase, and a testing project, and from tester's perspective, it is the most important phase of STPLC.
2. Test case creation
The phase
Once the testers have a clear understanding of requirements, they can start creating test cases, specifying test steps, and relevant expected results.
Adapting to the changes:
After the test cases /test scripts /test data are created, they should be verified for quality, and adequacy. Test cases should also be created for covering random flows for unexpected end-user behavior. Although it is nearly impossible to cover each and every aspect in test cases, an attempt should be made that the test cases/test scripts /test data cover most of the flows.
This phase may also require some communication with client and /or development vendor if the requirement/specification documents are missing or not up-to-date. If possible, a sign-off can also be taken from client for quality, adequacy, and completeness of the test cases /test data/test scripts.
3. Test execution, and defect reporting
The phase
After completion of test case creation, the task of running the test scripts and executing the test cases can be performed.
Adapting to the changes:
The defects found during test execution have to be reported to the client/development vendor. Here, the format/tool used for reporting defects becomes important because testing, and development vendors are different. The defects need to be precise, and clear to avoid any sort of disconnects, and to minimize average defect lifetime.
One more issue to be taken care of while executing test cases is setting of test environment. Setting up the test environment can be an issue, as testing-vendor may be testing from a different geographical location. It should be ensured that test environment is properly set before the actual execution begins.
It should also be ensured that test environment is different from the development environment, and any code push to the test environment by the development-vendor is reported to the client. If test environment, and development environment is same, developers may fix any reported defect, and reject it saying that the defect never existed.
4. Defect fixing by development-vendor
The phase
Once the defects are detected, and reported, they will be fixed by the development-vendor(s).
Adapting to the changes:
Development-vendor /client may even reject some of the defects, or even ask for clarification on some of the defects. The efficiency, and effectiveness of the communication depends on the format /tool used for reporting defects.
Note: This phase is not an action item on testing-vendor's part, and the time take by development-vendor during this phase is the major factor for average defect lifetime.
5. Retesting fixed defects, and reporting
The phase
The fixed defects have to be retested to see if the defects are fixed as per specs or not.
Adapting to the changes:
If demanded by client/development-vendor, clarifications should be given on defects reported earlier. Appropriate methodology should be devised to keep a log of all the communication done. Here I will once more emphasize the importance of format /tool used for reporting defects, and communication done.
Adapting to STPLC
After knowing the phases of STPLC, let us try to analyze these. It should be noted that phase 1, and 2 are one time activity at the start of any testing project. While, steps 3 through 5 are repetitive tasks, to be carried out on daily basis. More importantly, these steps are the most important from client's perspective. The client might be more interested in the number, and kind of defects a testing-vendor finds, or skips.
Phase 1 may not even consist of any deliverable to the client. Phase 2 does consist of deliverables to the client. But, as this phase does not consist of daily deliverables, the testers may have some time to work on their deliverables, and polish them well before they are delivered to the client. The test cases/ test scripts may be thoroughly reviewed for quality, and completeness before final delivery.
In contrast to phase 1, and 2, phases 3 through 5 are daily activities, and consist of daily deliverables. Daily delivery may not allow testers to conduct thorough reviews of their deliverables. This increases the importance of being first time right, every time. Your client sees your work through your deliverables. Majority of the deliverables in these phases will require good communication skills, apart from testing skills. Even if your work is very good, if your deliverables are not really professional and flawless, you might be in soup. Hence, the testers need to be very clear, and precise in communication.
One more important aspect is the processes set for phases 3 through 5, or the tool used while these phases. Before setting the processes, a thorough analysis should be carried out to understand the kind of reports, and analysis needed from the results of execution of these phases. And, the tools should be selected in such a way that generation of required reports, and analysis is possible, and easy. Testers will have to generate these reports on daily basis. So, reporting tools should be selected such that generation of these reports /analysis take minimum time, and testers can devote more time on productive work (testing).
Conclusion
To summarize, executing a testing project is not exactly the same as executing a testing phase of any other project. As testing projects are projects in themselves, they carry with them all overheads carried by any other kind of project.
We know that if one developer can do a work in 10 day, 2 developers cannot do the same work in 5 days. They will take some more time. Say 6 days. The reason is the overheads of increased team. As the team size increases, communication, training, and many other overheads increase.
Similarly, the overheads increase when testing is taken as a project. To minimize the effect of these overheads on project schedules, testers need to visualize the problem areas in advance, and devise appropriate methodologies. This calls for changes in various aspects.
