When I first started coaching agile teams I found it a challenge to coach testers. This was understandable because the agile movement was really developer-centric and much of the resource material reflected that. I spent time talking with skilled agile testers and watching them in action. A key thing I found was that they pose questions that no one else thinks to ask. In this article we will explore those questions and the context in which they are asked.
First of all, what do I mean by testing? I like James Bach's definition below because it suggests that testing is much more than a suite of test cases:
"Testing: Questioning a product in order to evaluate it ."
I find that good testers don't just question the final product. They question the product before work has even started and while it is being developed. And they also challenge the team itself. While this is true of many traditional testers, I have found these lines of questioning to be more prevalent on agile teams because a "whole team" mentality is encouraged and quality is a team responsibility.
The Agile Testing Matrix
|There are a lot of questions a tester can ask. We will use Brian Marick's agile testing matrix to help frame them.
I have added some typical test activities that could take place in each quadrant.
|I will take a slight liberty with Brian's matrix by using "Develop Product" instead of "Support Programming" as an axis.
Now we can frame the meta questions that drive effective agile testing.
It should be apparent that the activities and questions in the right-most quadrants are equally valid on traditional and agile projects. So there is not a lot new here for testers on agile teams. The main difference is that these activities run throughout the product life-cycle rather than being phased as post-development work.
Are we building the right thing?
Programmers are often inwardly focused asking the question "How can I make this work?" Testers balance this perspective with a more customer-centric view by asking outward facing questions to the product owner during iteration planning:
|What benefit would users get from this story?||This helps ensure that the story is delivering tangible value. The benefit can be captured using the template first suggested by Rachel Davies:
I want to
So that I can .
|How would we know this story is working?||Asking this aligns team as to how the new capability is intended to work. This in turn assists the team in identifying and estimating tasks. Examples of successful completion can be captured in story tests as "happy paths".|
|What are alternate ways this story could work?||Considering alternate ways that a story can unfold keeps the team alert to alternate solutions and prevents them from tunneling in on solutions too early. This question may also identify hidden complexity which will help scope the work more accurately. Examples of alternate paths can be captured in story tests as "alternate paths".|
|In what ways could this story fail?||Helps the team consider failure scenarios which they may not have considered. This helps bound the problem and eliminate confusion as to what is in and out of scope. Examples of failure scenarios can also be captured in story tests as "sad paths".|
Are we building it right?
The intent of questions here is to help instill a quality mindset in the team by finding and preventing defects while work is underway. The focus is on helping the team to get stories "done, done" meaning that that