Documentation Metrics

What Do You Really Want to Measure?

The number of pages that Writer X produced last month depends on many factors, including the writer's workload for that month. What does that have to do with the current project? You'd have to track many variables to see if the writer was truly being as productive, on a page-by-page basis, as he or she was last month. (Tracking these variables would be as much work as writing the manuals!) But more to the point, measuring documentation rates can lead people to believe that "more pages = better documentation." We know that’s not true; isn't it time we dropped metrics that support this idea?

Software Dependent Metrics
If your publications team reports to engineering or development, management may have a tendency to relate documentation standards to software development (X lines of code per day = Z pages of documentation per day). But remember that you are writing for the user audience, not the developers. Why should the users care about the ratio of X to Z? They don't. They'll judge your product on how easy it is to use, and nothing more.

Metrics That Add Value
Again, a quality metric is a metric intended to be a measure of document quality. You can't really make this measurement in the abstract—you need to consult with the user. Whether you do it before release of the product (through controlled usability testing) or after release (through customer feedback), you need to hear from the people who are using the product. They're the final arbiters of quality. 

Below are some quality metrics I believe have value.

Web-based Feedback
Develop a Web-based documentation feedback utility or survey that offers some reward (a drawing for a palm-held computer, for example) in exchange for the user's suggestions on a technical information product. When designing Webbased feedback forms, be sure to ask for information that will help you make accurate measurements. Don't limit the feedback to "1-to-5" responses, where a 1 represents "unacceptable” and a 5 represents "exceeds expectations." Allow users to comment on the information they need (or don’t need) in future revisions of your product. Then publish those results throughout executive levels on a monthly basis. Provide cumulative and current month/previous month evaluations of the responses you receive. It's a great way to elevate the visibility of the technical publications professionals.

 Usability Testing/Prerelease Feedback
Conduct some small-scale usability testing of beta versions or final drafts of user documentation with

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.