But It's a Feature!

Better Software Magazine
Volume-Issue: 
2003-02
Summary:

When you file a bug report, you become a bug advocate. It's your job to follow up to see that the bug is addressed one way or another. The way you write your report influences whether the application's behavior is changed. In this issue's "Bug Report," Danny Faught gives tips on writing successful bug reports.

"WHEN I CLICK File→open, the application crashes, erases my hard drive, and kicks my cat."

That's a compelling bug report. You probably don’t need to refer to the manual that explains what the File→open function does to make the point that harassing your pets is not one of the software's intended features. However, the everyday run-of-the-mill, low-severity bugs we usually deal with aren't this clear-cut. And the action the organization should take as a result of the bug reports we file isn’t as clear-cut either.

Becoming a Bug Advocate
When you file a bug report, you become a bug advocate—it's your job to follow up to see that the bug is addressed one way or another. The way you write your report influences whether the application's behavior is changed, the documentation is changed to match the current behavior, or the bug is ignored. Many bug reports don't advocate a particular course of action and, as such, are easier to ignore. Consider this brief bug report:

"When I start the application from the command line with no options, I get the error 'bad filename' and the application aborts."

Does the tester who wrote that report want there to be a default action when no command-line options are specified? Or does the tester want the documentation to better explain why at least one command-line option must be given? Or does the tester just want a better error message? Maybe she doesn't care what the fix looks like, as long as the results are clear to the user and the behavior matches the documentation? Let's look at four ways the bug report above could be focused to improve its chances of being acted upon.

Please Change the Behavior. "When I start the application from the command line with no options, I get the error 'bad filename' and the application aborts. Page 6 of the User Guide states that the application will use the default filename 'foo.dat,' so I expected the command to succeed with no options."

Please Change the Documentation. "When I start the application from the command line with no options, I get the error ‘bad filename’ and the application aborts. The User Guide does not clearly explain the command-line options: namely that the filename must be specified."

Please Change the Error Message. "When I start the application from the command line with no options, I get the error 'bad filename' and the application aborts. An error message is expected, but it should be improved to better explain the situation, e.g., 'no filename specified.'"

Please Change Either the Documentation or the Behavior. ?When I start the application from the command line with no options, I get the error 'bad filename' and the application aborts. This scenario is not documented in the User Guide, so it could be frustrating to the user."

Leveraging Documentation
So what happens if they choose to ignore a bug that you feel needs to be fixed? I have a trick...er...I mean technique to convince developers to reconsider a bug. The idea is to focus on the documentation and the error messages.

For example, I recently filed a bug describing an error message from two different tools on an operating system that I’m testing for one of my clients: ps and sweep (see the sidebar "Getting Your Fix" for more details). The error didn't seem to indicate any malfunction that would affect the operation of the system. It also didn't affect mainstream users of the system, who wouldn't have access to the errant programs, though it would cause confusion for both internal and third-party developers who were using

File: 
AttachmentSize
But It's a Feature!88.79 KB

About the author

Danny R. Faught's picture
Danny R. Faught

Danny R. Faught is the maintainer of testingfaqs.org, a source of information about tools and other resources for software testing. He is proprietor of Tejas Software Consulting, an independent consulting practice focusing on helping clients manage the quality of the software they produce.

Upcoming Events