Twenty years ago you would have people on a team but they wouldn’t talk. They wouldn’t break bread. I think agile teams truly have to realize that it’s not an individual sport, but it’s a team sport, and you have to play well in that environment, and truly look beyond your domains. I want testers to help developers be inquisitive. Even if you can’t spell PHP—at iContact we were a PHP shop—sit down with the developer and help them write a unit test. Be inquisitive. Show that you have a constructive interest in what they are doing, and expect the same in return because you’re on the same team.
JV: What type of testing techniques are appropriate for agile projects?
BG: There’s a couple core things that I try to amplify. One is this: traditional test automation, which was always approached from the UI down with very monolithic non-developer tools, from HP and Mercury to other tool vendors. It’s not that the tools were bad, but it was this top-down approach. Those approach and strategies were very susceptible to maintenance and challenges from the UI. There’s this muli-tiered approach in agile that’s emerged where you approach some tests from the UI, but very few. You approach some from the middle-tier side, via tools like Cucumber or FitNesse, and you also engage developers to write lots and lots of unit tests in whatever framework you’re using: N unit, X unit, PHP unit—whatever. It’s sort of this multi-tiered approach. I think that’s different, and it’s much more effective. That’s one thing we spend a lot of time on: The disruptive nature and the adjustments you have to make in test automation. It’s a whole team view, not just a testing view of automation.
Another area I’ve gotten great traction in is exploratory testing. I spend a lot of time yakking about exploratory testing in an agile context—exploratory testing and also session-based testing.
JV: Can you briefly explain what is session-based exploratory testing?
BG: You define charters. Let’s say we decompose Skype. You and I are interacting on Skype, so we decompose Skype into a set of charters. We would test the administrative system, the user system, the email system, and the naming system and we would create charters that say “We are going to test this area of the application.” We would decompose the charter and say “We can explore for about ninety minutes in that area.” You might come up with thirty to sixty charters depending on how you decompose any application. The charters are sort of your guardrails that control where you are going to explore and where you are not.
When you are going to attack an application and if we’ve just made changes, the team will decide which charters are relevant for the changes we’ve just made. So they might slice off twenty charters and those charters fit into ninety minutes—a typical time box is ninety minutes. I like to set up pairs. One of the nice benefits that I’ve seen is by doing exploratory testing in pairs you sort of reinforce this teamwork notion in agile. We’ll have developers pair with developers and testers pair with developers. I’ve had Salesforce customer support folks that we pair with testers and developers. It’s an all hands on deck sort of approach where you pair people up, they explore and are guided by the charter, and then you keep notes of your travels. You keep notes on the bumps you find. You keep notes for how to reproduce a bug; you don’t really log bugs.
The focus of an exploratory testing session is to explore as much as you can; to explore the most intriguing areas of your charter and then run them again. You then have a debrief at the end where the team gets together and they talk about what they’ve found. You compare notes and you decide which charters you are going to run next time, not just from what we’ve planned, but from what we’ve found. It’s an incredibly effective way of diving and conquering. It’s very collaborative; it’s very paired.
One of the things we noticed at iContact was that the best testers were actually customer support folks because they had this wonderful view of the customer and their challenges. We had developers and testers fight over who could pair with the customer support agent so they could learn how the app was being used and that would ripple back to the team and into our development activity. So it’s a testing activity, if you think about it, but it has all these side effects where you discover bugs and new test cases. Exploratory testing is a testing activity and an exploration activity. You find juicy bugs, but you also look for ideas for new tests.