Tester, Know Your Product

Should you diligently produce multiple big documents before testing begins? Consultant Fiona Charles argues that you should do that only if you believe that documentation is your product as a tester. If your product is information, you should instead minimize test documentation and engage with the software to build the product your stakeholders are paying for.

What's your product as a tester?

It's an important question. Your product should determine your means of production—the essential activities and tasks, how you apportion effort, and the priorities you assign to various activities.

Your product is the principal thing that your stakeholders are paying for you to produce. For a tester, that product is information. It's what you produce that actually furthers achievement of your stakeholders' goals for the project or business. Yet, judging by effort, it seems many testers believe their product is a large set of documents that describe what they think they're going to do when they finally get to engage with a software system.

You could argue that you have more than one product—that information is primary, but you must still produce both evidence that your primary product is sound and guidance for your team or for people who might have to tackle this job at another time.

And indeed, that's true for many testers and test organizations. So I'll ask another question: Do you really believe that all the documents you currently produce represent the best use of your time in developing your primary and secondary products?

If you are in the testing mainstream doing what is sometimes called "structured testing" (by people who don't understand that most exploratory testing is structured) and if you are not working on a genuinely agile project, you are probably producing some or all of the following:

  • A test project schedule
  • A master test plan
  • A test strategy
  • Test plans (one for each major type of test)
  • Test cases
  • Scripts
  • A test execution plan

As a consultant, I work with many organizations where all these documents are required deliverables of the standard test process before actual testing begins. With few exceptions, I find that testers and test managers expend excessive amounts of time on heavyweight documents that do little or nothing to move their testing forward, and that frequently even impede testing by diverting effort that would be better spent thinking about and working with the software.

Project schedules are important, but how much time have you spent messing about with a tool like Microsoft Project, vainly trying to produce a credible schedule that reflects your real intentions yet satisfies a project manager who demands an unmaintainable level of detail? Compare that to the time you've spent actually planning—thinking about what to do and how and when to do it. On some projects, the test manager no sooner produces a schedule than she has to turn around and rejig it because someone else has slipped his schedule or the project manager wants to "compress" some of the testing. I'm with US general (and subsequently president) Dwight D. Eisenhower, who said, "In preparing for battle I have always found that plans are useless, but planning is indispensable."

Testing is research. It's far more like battle than it is like manufacturing. We go into it with a strategy and plan in mind, but we don't know exactly what we're going to encounter that could completely derail our plans and force us to rethink. It's the thinking that's significant, not the probably-obsolete-tomorrow artifact that encapsulates it. Let's not spend more time churning out that artifact than it's worth.

Some form of schedule is likely essential, but can you say the same of a master test plan (MTP), particularly if you're also going to write a test strategy and one or more test plans? Often, an MTP is demanded far too early. How can you develop any plan before you understand the testing problem and your strategy for addressing it? The "solution" in most organizations that produce one is a boilerplate MTP copied and pasted from previous projects. Is copying and pasting useless text a cost-effective use of your time?

User Comments

James Christie's picture
James Christie

I agree with every word. I've just been glancing through an "exemplar" test strategy document for a very large company, and it's full of utterly pointless, time-wasting boilerplate explanations of the V Model, and the difference between validation and verification.

It's only after that sort of rubbish that you get interesting, but vague, stuff like: "User Acceptance Testing is expected to focus on Documentation resulting from the application, and how the system fits with known business processes/procedures. The functionality will be tested against the Business Requirements. Usability will also be a key consideration at this stage, when the end to end structure to be delivered into Production is in place."

Well, it might be interesting, but only in the sense that it reveals the blinkered attitude that produces such "strategies" and which is expected of the testers. Testing will be constrained by what is already known, rather than seeking to discover what is not yet known. The bit about usability is pure window dressing, put in to give the impression that they took it seriously. There is no mention of usability for any of the earlier stages. Waiting till UAT to discover usability defects is a surefire way to get them dismissed as cosmetic. It's far, far too late.

Remember this is an example of how to do a strategy properly at that company. People who produce such woeful exemplars, and those who are expected to work with them need to read articles like yours, Fiona. They need to be constantly challenged. What are they trying to do? Produce verbose documents of atrocious quality? Or test?

November 22, 2011 - 6:03am
Clarissa Lovett's picture
Clarissa Lovett

I would love to be a part of a team that didn't kill obscene amounts of trees! I agree with every word, Fiona! Thank you for writing what I've been thinking for quite some time. :-)

November 22, 2011 - 12:06pm
Scott Nickell's picture
Scott Nickell

Nicely put: "Testing is research. It's far more like battle than it is like manufacturing. We go into it with a strategy and plan in mind, but we don't know exactly what we're going to encounter that could completely derail our plans and force us to rethink."

Like many, I started out years ago on a team where gigantic, yet still woefully inadequate, test strategy/plan/case documents were the norm. And, like many, have since found more effective ways to spend my time. The big strategy/plan docs tended to exist in WOM (Write-Only Memory) - no one ever referred back to them after they were created. And in my experience, detailed, step-by-step manual test cases can be useful for bringing in new people and getting them familiar with the product, but not much else. If you're going to all that trouble to write test cases that don't require thinking to run, then why not just write them as automated scripts?

November 23, 2011 - 2:21pm

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.