You Make the Call

Navigating the Tool Selection Process

free alternative product if it can get the job done. This is where your test tool guru will come in handy. Utilize their expertise, and get their knowledge on their tool of choice. Also, investigate other divisions of your company. Your company may have already purchased a tool (or several tools) in another division for their project. It would be wise to get their feedback and experiences with various test automation software.

Create a Proof-of-Concept (POC) Checklist. This document will allow you to define what the tools will need to accomplish in order for your organization to successfully use the product. This document should detail your application, technologies, platforms, and what tasks need to be accomplished with the software before a purchase is considered. For example, you may have several test cases you would like to execute, which show the tool's ability to work with your application and test the business rules. Strong object recognition may also be a criterion. The POC checklist should contain everything you feel you need to see in order to make an informed buying decision. However, this list should be reasonable for all vendors and participants involved. Extensive POCs (evaluations in which the vendor participates over several weeks) may cause the vendor to charge you for time spent using their tool with your application. This is a very important step in the overall strategy. Sometimes the vendor will provide you with a template of a POC checklist. This should be used as a starting point, and not the official document. If you don't set your criteria, the vendor will do it for you. The tool needs to meet your requirements, not the other way around.

Narrow your search and contact potential vendors. You have done your research, and now you can begin to contact the vendors who seem to meet your criteria. This is where the steps above save you a ton of time. You must realize that when you call vendors, you will be in contact with their sales representatives. Sales reps are trying to make money. When a potential customer calls in stating that they are looking to implement a test automation solution, they will immediately want to provide you with a software solution. They will constantly call you trying to set up an onsite demo and/or POC. In order to save you and your organization the headache, proper planning is the key. Put yourself and your organization in a position in which you will be prepared to listen. Without proper research, all the tools will start to look alike after the onsite demos.

Bring 'em on site. The next step is to have the vendors come on site and complete the POC on your application based on your criteria. Make sure that you give the vendor a pristine workstation to install their application. This machine should have the proper amount of virtual memory and hard disk space. Also, make sure to have only one test tool on the workstation at a time. During the POC, some vendors will complain that a competitor's application is interfering with theirs. I have seen this happen many times, and most concerns are well founded. Each tool is doing something similar, chances are there is a conflict. To limit quirky behavior, provide a "clean" workstation for each vendor.

Establish selection criteria. What if two or more vendors meet your criteria? What will be your deciding factor(s)? Will it be price, ease-of-use, ease-of-maintenance, quality of service, licensing, company background, etc., or a combination of each? What if they are dead-even on price? Establish what you consider the differentiators

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.