This article is about getting users involved early in the iterative test cycles, and it also discusses exploratory testing. The author also delves into the benefits of an incentive-driven, team competition approach to testing.
Getting users involved early in the iterative test cycles is discussed. Exploratory testing is discussed. The benefits of an incentive driven, team competition approach to testing is presented
Experienced project managers are familiar with the benefits of several best practices:
- Early User involvement in the SDLC provides valuable feedback to the requirements and design, as these issues are much more costly to repair if not found until the later project stages. Figure 1 illustrates the relative cost to repair defects as a function of the lifecycle phase in which the defect was found.
- Iterative Development cycles provide a productivity improvement in identifying and repairing development issues, as developers can be fixing issues found in a previous test cycle while the testers are verifying the next cycle. The cycles can be sized for optimum productivity, for example based on subsystem functionality or on the size of the teams.
- Exploratory testing, if completed by subject matter experts, can accomplish a good portion of the verification goals without spending significant schedule time on extensive test case planning and writing. The important aspect to keep in mind is that exploratory testing is not ad hoc. To be truly effective, you must use subject matter experts, and they must be provided a test charter such that:
- Detailed test outlines or scenarios are created and used as a guide during test execution. These must trace all functional requirements to verification activities, and must identify the test flow or schedule of tasks necessary to achieve verification.
- Test results logs must be kept so that issues can be recreated if necessary
- Defects must be adequately documented and reported
- Boundary values, functional paths, and associated parameters must be verified during multiple user iterations and variations of the test outline.
Note: Exploratory testing does not replace system verification, it complements it. Your system verification may require complete testing of system functions, performance, interfaces, etc, which may require engineering your test environments, test tools, and formal peer-reviewed test scripts written in advance. Some system requirements can only be verified by measurement, analysis, or inspection, and these may require additional non-exploratory test engineering approaches.
What is Competitive Exploratory Testing?
All three of the practices identified above can be brought together in what I call Competitive Exploratory Testing. The concept is simple. When provided incentives, people work hard. When workers are part of a team, they feel compelled to help the team succeed. If there are multiple teams competing with each other, with incentives to win or do well in a contest, the team productivity increases. The contest really heats up (and the fun begins) if the progress for each team is plainly visible for all the teams, as I will show in the example below.
The SSALSA Project
In 2002 I joined a large scale, client-server project in New Mexico. The objective of the project was to replace a mainframe financial system with a web based server system. The program is supported by over one thousand case workers in a dozen state field offices, where low income residents may apply for and receive various types of financial support, work programs, and training. The project was supported by a contracted development team and by 16 state employees (system Users). The state team was collocated in a large test lab with a workspace and workstation for each member. The team was provided instruction in the processes for creating test scenarios, running the tests, and documenting test results and defects. The project management team had set up an iterative development approach, and the state test team was receiving a build to test about every three