What to Do with Too Much Test Documentation: An Interview with Fiona Charles


JV: That's so interesting. I wonder if it's just a different view, say, of what management has on testers than what they might have with programmers. Are there any certain products or industries in which there's mountains of test documentation that are just unable to be avoided, or is it sort of the same across all industries?

FC: I think that the amount of documentation, the kind of documentation you need to produce for testing, is entirely context dependent. The one overriding principle is that the artifacts have to serve the work, i.e., they have to serve the testing work, not the work serve the artifacts. So if you're in a highly regulated industry, if you're in an industry where it's extremely important that there be evidence of the kind of testing you've done, then think about what is appropriate evidence for that.

It's not necessarily huge numbers of plans and documents done beforehand. It may simply be that you do a video of your testing or you keep notes about your testing. You demonstrate the results of your testing in various ways. I think some of this comes, as I say, out of people who want to control what testing is done, but I think it also comes out of a misunderstanding about what kind of documentation you need necessarily.

JV: How do you know whether or not you're spending too much time on test documentation to begin with? I mean, is it just you come out and you're like, "Wow, we haven't done anything, but we have filled out fifty pages?"

FC: I wrote a StickyMind's article about this a while ago. The product, as a tester, is information for your stakeholders. If you're spending more time writing a whole lot of documentation, which is not your product, and not spending the time on testing, then clearly the balance is not there.

JV: Are there any easier ways for testers to notify stakeholders of these testing efforts without having to provide so much paperwork?

FC: Well, they can talk to them.

JV: They can talk to them.

FC: That's a really good start. I use the word artifacts deliberately because a prose document, and often we're looking at acres and acres of continuous prose, a prose document is not always the best artifact. It could be a diagram that's annotated to demonstrate what you're going to do. I've used that often for test strategies. You could use mind maps for certain kinds of test documentation. You can do all kinds of things. You can use a wiki.

It really depends on what the requirements are of your situation. If you look at the standards, again, going back to the IEEE standards, typically some of those requirements come out of a need to demonstrate due diligence in testing. Well, you don't always need to demonstrate due diligence. If you are in a context where you might be called into court or where your software is particularly high risk, then sure, you do need to demonstrate that you've exercised your best professional judgment in how you test and that you have, in fact, carried through on what you said you would do, but again, you don't have to have acres and acres of continuous prose to do that. There are lots of options.

JV: Got you. We're winding down a little bit on our time, but I want to get a couple questions in on how to deliver bad news because it's another area that you talk about, and this sort of segues nicely into this. Well, let's say that you have to deliver some bad news to some of these stakeholders. Can you describe a situation of how delivering bad news in a careless way can lead to a really negative outcome?

FC: I could give you an example ...

JV: I'd love to hear it.

FC: In fact, I talked about this. I did a webinar on this topic this morning before your’s started, but a tester in a large organization that does software for the consuming public was at a product launch and a product-launch party. So here's this big celebration in this big company with the vice-president, who was the executive sponsor for this particular product, plus a whole lot of other vice-presidents ...

JV: A big deal.

FC: ... and making the speech and everybody's happy. An innocent and terribly honest tester blurts out, "But the thing's full of holes."

JV: Oh, man (laughs).

FC: Well, yes, true, but this is not the right forum for that.

JV: Right. I can tell that. That's a careless and awkward manner to express that sentiment in that way.

FC: I mean, "careless" is a bit harsh, but it is, in fact. It's not knowing better.

JV: Not knowing better. Yeah, there’s a better way of saying it. What are some tips you can give our readers and listeners when they're preparing to deliver some bad news? Obviously, don't do it during ... when all the vice-presidents ... when everything, when all that's around.

FC: Well, pick where and when you're going to have the conversation. Pick a quiet place and don't ambush. If you have an unwelcome message to deliver, don't deliver it in a way that ambushes the other person. Don't corner the other person. Don't embarrass them or unnecessarily annoy them.

About the author

Upcoming Events

Nov 05
Nov 14
Dec 05
Jun 03