Working With Automated Testing Tools

A Pragmatic Approach
Member Submitted

Unfortunately many corporations make blunders with their automation strategy even before a single test script has been automated with a testing tool. In this article, Jose Fajardo will discuss some considerations that should be examined before purchasing expensive automated testing tools and working with them. The author also suggests some robust automated testing approaches and dispels some common testing misconceptions based on his actual hands-on automated testing experience.

Automated testing tool selection and acquisition
Many companies first learn about automated testing tools through conferences, trade shows, internal contractors, consulting companies, or from employees that have previous experience with one or more automated testing tool. Usually after companies learn about automated testing tools and decide to acquire an automated software solution they get product demos from the vendors, then the vendors provide the companies a copy of their software copy with an evaluation license. After a company's evaluation period of the software expires the company may wind up spending hundreds of thousands of dollars on automated testing software solutions that may not even meet a partial listing of the company’s testing needs.

The need to select and purchase automated software solutions should be compelling and critical while meeting a vast majority of the company's testing efforts. The reader should note that it might not be possible to find an automated testing software solution from a single vendor to meet all of the company's automated testing needs. Companies should have a test manager or a testing champion on-site who is thoroughly familiar with automated testing tools software in general and can provide substantial feedback to the company researching test tools for possible acquisition during the evaluation period. The test manager should articulate clearly and have a vision as to how the automated software solution will be used, how compatible the automated software is with the existing IT environment, who will use the software, and develop a realistic schedule of how many test scripts can actually be developed with the automated testing tool given the company’s already established deadlines for a given software release or deployment.

Test managers should also recognize that the mere fact that a software vendor provides a satisfactory product demonstration for its automated testing tool is not sufficient to guarantee that the automated testing tool will be compatible with the project's application or with the testing team's on-going testing objectives. The test manager should perform due diligence in asking the automated software vendor for a demonstration of its automated software against the company's IT environment. Furthermore, the test manager or the project's testing champion with previous knowledge of an automated testing tool should compose a "wish-list of automated test scripts" from the project's current test scenarios ranging from low complexity to medium complexity that they would like to have the vendor fully automate in their environment before acquiring a specific automated tool from a vendor. The onus and burden should fall squarely on the vendors to demonstrate how their automated testing tools will fit the needs and be compatible with the company’s software application.

Test managers should be cognizant of the fact that many automated testing vendors will make many promises about their tools during their product demonstration or “dog and pony show” to make a sale for their tools when in fact these promises may not be consistent with the company’s testing goals or may go unfulfilled. The test manager should seek software demonstrations from at least 3 different vendors during the tool selection phase and should rely on various readily available automated tool selection matrices before recommending making a purchase for an automated testing tool. Armed with the knowledge of another automated testing tool the appointed test manager or test champion should inquire and verify that the product that the vendor is demonstrating has features and capabilities that exceed or are similar to automated testing products that they have worked with at previous projects.

Minimum Criteria Factors to consider before tools acquisition
At the very least the acquired automated testing software should meet the criteria


About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.