What to Do with Too Much Test Documentation: An Interview with Fiona Charles


Fiona Charles is a Toronto-based test consultant and manager with thirty years of experience in software development and integration projects. In this interview, Fiona discusses excess test documentation and what to do about it as well as the art of delivering bad news.

Fiona Charles is a Toronto-based test consultant and manager with thirty years of experience in software development and integration projects. In this interview, Fiona discusses excess test documentation and what to do about it as well as the art of delivering bad news.

Jonathan Vanian: This is Fiona Charles. Fiona, thank you so much for being here with us today.

Fiona Charles: You're welcome. Thank you.

JV: Why don't you just talk a little bit about yourself, your career so far, and just explain your expertise to some of our listeners and readers?

FC: I'm a consultant and sometimes contract test manager. I've been in the industry since 1978, so I've been around for a while. Mostly in the last few years, I've either been consulting on testing and test management or I've been managing testing, as I said. Typically, I manage testing on large multi-project programs.

JV: Ok.

FC: And I do rescues of testing efforts that have gone astray. I look for problems to solve.

JV: You look for problems to solve. Is that a good definition of a tester?

FC: Oh, I think so. Having been doing it for a long time and I’ve been around testing for a long time, I make sure that that's what I do, so I do boring stuff.

JV: (Laughs). Let's talk a little bit about test documentation and how that can sometimes boggle down testers. Can you talk about an example where a team might be overwhelmed with documentation?

FC: I certainly can. I have a lot of examples, but I'll talk about one. I was brought into a project to "fix the testing," and when I got there, what I found was that the testers had been ... There were actually two leads and a couple of testers. They hadn't had very much to test, and they'd been sitting around for about a year, and they'd been churning out documents because that was what their management wanted them to do and it was what the standards set by their testing practice wanted them to do.

JV: Right.

FC: I can't remember how many documents now, but there was a strategy. There was a master test brand. There were a bunch of plans because there was a plan for this and a plan for that and a plan for what happens when we get this. Just about every single one of those plans was twenty-five pages at least, and I started keeping count because it became quite consistent. Around about page twelve or thirteen, you got this tiny, tiny little bit of content about the project, but up to then, it was boilerplate. It was definitions. It was a whole lot of nonsense that had nothing whatever to do with the project. Then you'd get past that little bit of content, and there'd be more stuff.

JV: A lot of busy work.

FC: It is really busy work, and it's a terrible waste of time. Not only is it a waste of the tester's time and expertise, it's a waste of everybody else's time. You ask people to read this stuff because you want them to buy in. You want them to understand what it is that you're doing. They can't read it, and they don't read it. In this particular case, all of this stuff was completely ignored.

They had a strategy document. It was fifty-pages long, and it had no strategy whatsoever in it, and it wasn't the tester's fault. It was because of what they were being asked to do, and I find that's very common. It's a result, I think, of certain kinds of standards. The IEEE standards are the classic ones that are typically adopted in organizations. That stuff is not appropriate really for a lot of what people are doing, and those standards tend to focus on documentation, not on testing.

JV: How did this happen that those are no longer appropriate? Was there a time when it was appropriate or is this a different era that we're in now?

FC: Oh, it's partly that, but I think it's also that people want to control testing. They want to control testing in a way that they don't particularly want to control programming. You don't say to a programmer, "Sit down and document before you do it everything that you're going to do."

JV: Right.

FC: We ask people for designs, but we don't expect them to document every point and click. Somehow, we expect that of testers. Well, I don't, and I say we. Management expects that of testers, and that's absurd. I was in an organization where, when I said we're doing fairly lightweight documentation and we're not running down the expected results because the testers know what to expect and they'll interpret and they'll make a call based on whether what they see is reasonable and what they learn about the system and so on, and the manager said to me, "You're putting too much on the testers," to which the only answer has to be, "Is it putting too much on the programmers to expect them to program without those explicit directions?" The testers are just as bright. They're just as skilled, and if they're not, why not? Let's make sure they are.

About the author

Jonathan Vanian's picture Jonathan Vanian

Jonathan Vanian is an online editor who edits, writes, interviews, and helps turn the many cranks at StickyMinds, TechWell, AgileConnection, and CMCrossroads. He has worked for newspapers, websites, and a magazine, and is not as scared of the demise of the written word as others may appear to be. Software and high technology never cease to amaze him.

Upcoming Events

Apr 13
Apr 27
May 03
Jun 07