Understanding Both Sides of the Test Tool Fence

[article]

it becomes more difficult to walk away, especially if the tool is an expensive investment such as a high-end load testing tool. High-volume tool users tend to value open architecture and ease of use.

On the developer's side, the higher fence of high-volume tools allows development to follow standardized testing practices because the tool is designed to solve an industry-wide problem and be amortized across many customers. Flexibility is of utmost priority in order to maximize the general usefulness of the tool without extensive customization. The developer therefore is able to determine the shape and function of the tool according to standard practices, rather than accounting for numerous specific variations desired by individual customers.

With low-volume tools, where the customer is empowered, the user's needs are explicitly expressed in a specification and set of acceptance criteria. The developer responds to these needs, creating the required capabilities in the tool. In the end, however, the customer pays for this empowerment with a higher price tag. Low-volume tool users typically place the highest value on accommodation of their specific situation.

Developers of low-volume tools enjoy greater latitude in using innovative methods to fulfill the customer's needs. In this arena, tools are created to fill a specific need, with generalization as a low-priority goal. Customization is the key to success.

View From the Other Side
Understanding the view from one's own side of the fence is important, but it is equally important to know the view from the opposite side. The grass may or may not be greener on the other side-but potential customers should try to understand the mindset of the tool's development staff by looking at several areas, including

  • Documentation: Should be adequate to explain the tool's usage and installation, as well as illustrating the testing philosophy implied by the tool
  • Intuitive use: Given that the customer understands the testing being automated, the steps performed by both user and tool should seem natural and progressive

In order to understand the developer's point of view, a potential customer may wish to find out the background of the tool's development staff, including

  • Has the development staff personally performed the type of testing being automated?
  • Do the developers know and understand any applicable standards or government regulations?
  • Does the team understand the purpose of the testing that the tool will perform?
  • Can the development or support staff help educate a customer in any or all of these areas?

As a tool developer, understanding the customer is critical to success. With all tools, the documentation should address all of the areas in the above questions. For high-volume tools, this may be enough information to ensure that the tool adequately performs its assigned tasks.

In low-volume tools, though, more information is needed. There are several additional areas that the developer should examine regarding the potential customer, including

  • Does the customer appreciate the tradeoffs between flexibility and capability?
  • Can the customer adequately express their requirements?
  • Can the custom requirements be translated into an effective algorithm?

The developer's job, when the customer doesn't understand something, is to educate.

A potential user must peer over the fence to evaluate the methodology and flexibility built into the tool, plus the development team's knowledge about the type of testing being automated. These areas will help establish the potential value of the tool beyond the glossy-printed brochures used to market it.

The developer, on the other hand, must understand the industry being served, know the type of testing being automated, and understand how this type of testing fits into the customer's overall needs. The developer must also understand the limitations of the

About the author

Steve Morton's picture Steve Morton

Steve Morton is an automated test tool developer by trade, operating in an arena of low volume and high expectations for the past eight-plus years. He has primarily worked on an automated, low-level structural analysis and test definition tool tailored for safety and mission-critical software development markets such as the aerospace and medical devices industries. His ongoing efforts to bridge his developer’s viewpoint into the viewpoint of a tool’s customers has led him to acutely examine the nature of the relationship between developers and users of automated software testing tools. In addition, Steve has used both commercial and homegrown testing tools throughout his career, enabling him to stand on both sides of the automated software test tool fence.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Nov 09
Nov 09
Apr 13
May 03