The Truth About Exploratory Testing

Exploratory testing is an approach that all true test engineers use, but its exploitation by those who wish to escape accountability and forgo up-front planning is disturbing. If these misguided individuals would more closely study the detailed works of exploratory testing experts (those in the industry who have led the effort to perform and teach exploratory testing as an official approach), they would discover what exploratory testing really is—a way to enhance scripted testing.

Only a surface-level view of the approach is obtained from abbreviated write-ups. In some cases even the experts incorrectly suggest that scripted testing conflicts with meaningful exploratory testing and hurts the quality of applications. This sends the wrong message about scripted testing and about testing processes in general.

The term "scripted testing" often encompasses test case and procedure design, documentation, and execution. Test design is the mental conception or planning of the test. Documentation is formally writing or printing the test design information, and execution is the actual test case implementation. Test scripting creates traceability, provides information about expected application functionality, identifies test scenarios more completely than could often be done during real-time testing, and provides information for properly setting up the environment and establishing test data.

Exploratory testing—simultaneous learning, test design, and test execution—allows the tester to control the design of tests as he performs them and to use information gained while testing to design new, more effective tests. Experts indicate that the term "exploratory testing" refers to a thought process that exists on a continuum between purely scripted and purely exploratory. Purely scripted means the tester only performs tests based on instructions provided by the script. Purely exploratory means none of the tester's activities are pre-specified, and the tester is not required to generate any test documentation beyond bug reports. Using this definition, we can clearly see that it's not really a question of whether you perform exploratory testing or scripted testing, but rather to what degree you perform each. Exploratory testing can be executed during up-front script design and test execution whether or not a script is used. But the ability to effectively implement the analytical behavior associated with exploratory testing separates skilled software test engineers from "test executors," who merely follow steps created by someone else.

When experts speak of a conflict between scripted testing and exploratory testing, they’re referring to people who follow pre-documented information to avoid having to make real-time decisions for better testing of the application. This is a problem, but not due to the development of scripts or a conflict between scripted and exploratory testing. The problem lies with the test implementation, and this may be better understood by comparing test scripts to the test plan as a whole. The test plan is a guide for managing the entire testing effort, but during implementation there are things in the plan that must change based on what's happening in real time. The fact that you need to make changes, however, doesn’t mean that writing and using the test plan is bad. It only means that the test manager must be analytical and flexible enough to make reality based adjustments to the plan. The plan is still necessary, because without it testing would be terribly inefficient. The plan is not meant to limit you, but rather to aid you, and the same applies to scripts.

Test scripts provide a springboard for executing tests in the application. In some respects, the concept of a continuum of scripted and exploratory testing oversimplifies the relationship, because it suggests that the more scripted the test approach, the less exploratory it is. This is not