Lean Test Documentation

In traditional test design, testers are forced to create scores of documents. A lot of time is wasted documenting detailed tests that are neither needed nor useful with the exception of mission-critical applications or unless you are testing for compliances and regulations in which heavy documentation is necessary.

Lean Test Documentation
When I say lean test documentation, I am referring to any test documentation that is optimal and capable of conveying the same information using fewer details. It is less verbose, less bulky in terms of number of pages and “lean.”

Traditional test documents are heavy in size, shape, and magnitude. They are detailed, consume a lot of time to write, and yet testers don’t have enough patience to gobble it up during testing.

As James Bach wrote in one of his blog posts on Test Documentation, “A lot of people I teach seem to be under pressure to create more documents to describe their test process. But documenting is not testing. It is one of the chief distractions to testing.”

Heavy documentation can be reduced by using smart techniques like mindmapping, outlines, checklists, and matrices. I have compared and contrasted these techniques below.

Mindmapping
A mindmap is a visual representation of ideas, concepts, or tasks encompassed within a central theme. It can be used for visual tracking of test coverage, testing sessions, feature priorities, and status reports.

Mindmaps prevent extensive and useless documentation by displaying content that is succinct and brief. For example, a tester can write many pages about product exploration and learning that can be transformed into a simple mindmap below.

Figure 1. Example Mindmap

Mindmaps can simplify test approaches by crunching scores of pages into a worksheet, as this mindmap on Claims Testing shows.

This other example of a mindmap posted on the Moolya blog summarizes a list of heuristics used for mobile application testing. This demonstrates how even lengthy documents can take the form of self-explanatory visuals.

Pros:

• Flexible and easy to use
• Triggers creativity
• Facilitates Quick Updates in the world of changing requirements
• Reduces test documentation time saving time for real testing

Cons:

• Mindmaps cannot be converted to direct excel reports
• Unsuitable on metrics-crazy projects
• Unfit for detailed documentation

Checklists
A checklist is a master list of items that needs to be checked against before, during, or after performing some action. For example, when a pilot refers to a pre-flight checklist to confirm if everything is in place.

Atul Gawande, author of  The Checklist Manifesto says, “Checklists provide a cognitive net. They catch mental flaws inherent in all of us—flaws of memory and attention and thoroughness.”

Using checklists and one-liner test ideas instead of writing fifty-step test scenarios reduces wasteful documentation. Checklists need to be good enough to understand and brief enough not to turn people's brains off.

Pros:

• Aids memory
• Maintain consistency in tasks and activities

Cons:

• Cannot be used as a learning tool

Tabular Representations
Some data is easier to represent using tables arranged in rows and columns. The same data if described verbally make take several paragraphs to describe.

Platform coverage can be displayed in a table as indicated below:

In a traditional test plan document, platform coverage details would eat up at least three to four  pages in the test environment section of the test plan. Here it’s reduced to a 2x5 table.

Pros:

• Pervasive method of communication
• Visually appealing
• Saves time and effort by avoiding verbose information

Cons:

• Not all data can be represented using tables

Outlines
Outlines are traditional pictorial representations used for visual depiction. They includes drawings, sketches, graphics, and others.

Features in a product can be displayed as an outline. Outlines give quick introduction into all sub-features listed in each feature. Such an outline saves several pages of feature descriptions.

Figure 2. Feature Outline

Pros:

• Inspirational
• Easy to understand
• May be “linkable” to detailed requirements

Cons:

• Doesn’t actually tell the reader what to do
• May be redundant with system documentation

Why
If a document-centric culture, documentation may be the only element of testing that those outside the team actually see—creating that the documents become more important that the testing. Many non-testers and management folks mistake testing for documenting tests and demanding more money from customers. Pradeep Soundararajan speaks here about how customers are fooled by heavy test documentation.

What do customers care about?

Customers don’t care about how many documents are produced. They care about how the product works when it gets to them. Forcing testers to slog with document creation is a foolish thing to do for any organization, especially those that believe that keeping customers happy is their foremost goal. Adopting lean test documentation saves lot of time and energy for the tester, which can be used for Brainual testing.

Have you used any kind of lean documentation? I invite you to share your experience in the comments section.