Exploratory testing--questioning and learning about the product as you design and execute tests rather than slavishly following predefined scripts--makes sense for many projects. But does it make sense for agile projects? In this week's column, Johanna Rothman examines how exploratory testing might work on an agile project.
It's easy to see how exploratory testing works on waterfall projects. In fact, it makes a ton of sense on waterfall projects to do at least a little exploratory testing before you decide where to focus your efforts, design your tests, or consider any automation. That's because on a waterfall project testers are much more likely to be crunched for time at the end of the project. (For ideas on how to use exploratory testing on a waterfall project where you don't have enough time see my previous column, " So Many Tests, So Little Time .")
But does exploratory testing make sense on an agile project? Of course. Yet agile exploratory testing doesn't look the same as it does on a waterfall project. There is less of a need for exploratory testing after the product is created and a greater role for exploratory testing as the team defines and develops the product.
Michael Bolton says, "When people talk about exploratory testing, they often mean the process of test execution. But the three pillars of exploratory testing are test design, test execution, and learning." In waterfall projects, the execution is most visible. But on agile projects, the feedback loop between test design, test execution, and learning is quite short--and agile projects require all three pieces.
On agile teams, there is little role for a manual black box tester who waits for the requirements, the design, or the code and finally tests according to a predefined script. That's because there's no waiting for requirements, design, or code. The product owner "defines" a feature or chunk of a feature, but the definition is more like a promise to keep discussing that feature rather than a firm commitment--"This is what the feature will do."
If the developers are using test-driven development, the tests help define the design. Even if the developers aren't using test-driven development, they tend to work in small chunks--no more than a day or so long--which decreases the waiting time for code.
There is still a role for testers who can turn their keen observations into ways to quickly test an area in depth. But those testers are even more valuable if they can develop tests and testing ideas--both manual and automated--before the feature is completely coded.
Roles for Agile Testers
- Testers can help the whole team work in a test-driven way by:
- Assisting with defining requirements in the form of user stories and generating system-level tests from those requirements. Think of this as a questioning, test-driven approach to development.
- Assisting with modeling the requirements (user stories)--by asking questions--and generating system-level tests from these stories.
- Testers provide information about each chunk, as the developer delivers it. Sometimes that information is confirmation that the code does what the user story says, does not have side effects, and is release-able at the end of each iteration. Sometimes that information is extra information about the product and other paths the testing could take.
- Finally, testers can perform system-level regression testing during an iteration. (To be honest, on well-established agile teams, the developers typically run the low-level regression tests, allowing the testers to do what they do best: explore, question, and learn.) If the organization is new to agile, the testers might run regression system-level testing at the end of the project.
Where Does Exploratory Testing Fit During an Agile Project?
Exploratory testing is helpful when working with the product owner and the project team, trying to define user stories. Imagine you're working on a banking system and want to allow between-account transfers. The developers have a scheme to use checksums