process. But that's only a small piece of the overall selection process.
Trial By Fire
The next method employed by some companies, is to have a novice group of business analyst/testers who currently do manual testing, dictate the evaluation. This is usually the group that heard through the grapevine that these tools will "make their job easier" and "reduce their testing time" with "no programming required." This can be very detrimental to the entire tool selection process. The tool that seems the least intimidating usually wins. Unfortunately, all of the testing tools require some sort of programming expertise. This does not mean that the tester must be a programming guru, but must have a basic understanding of programming concepts in order to develop and maintain their test code.
Trial By Committee
This is when a company uses several members, involved either directly or indirectly in the tool selection process. This may consist of employees from different departments (development, QA, test team, operations, support, business development, technical writing, etc.). Each committee member has a vote to decide which tool meets the organization's needs. This segment of the evaluation process has a distinct advantage because you have the expertise of several groups. But this also means that members involved in the selection process have a vote, but the tool chosen has no direct impact on them. For example, Operations may not care about test automation, but may have a need for production-monitoring software. Technical support may not care about performance testing, but may have some interest in defect tracking software. The committee should consist of a group of individuals who truly have a vested interest in the implementation of the test automation process. Bringing in members with whom you interface, but have no direct impact in your test automation strategy, could be detrimental to getting the right software.
How does one go about selecting the right automation strategy? Here is a process that I recommend:
Know your needs. Before you get on the phone and start calling vendors for evaluation copies, you should first take a close look at your organization and your testing needs. Is your organization truly ready to begin test automation? Who on the team is capable of using these tools? What are their skill sets? Are you looking for test automation, test management, performance testing, defect tracking, or all of the above? Do I need a committee? If so, who should be involved? How much do we have in our budget? Does our budget allow us to start a useful test automation strategy? What are our testing goals? These and several other questions will need to be addressed before the search begins.
Establish testing criteria. Knowing what technologies need to be tested is extremely important. What is my application platform (Java, .NET, HTML, ASP, JSP, Siebel, Oracle Forms, SAP, etc.)? If performance testing, what protocols are involved? What is the overall architecture of the system? What other projects and technologies will I have to support in the future? Make a list of the critical (and noncritical) technologies involved in the development of your application. This will help narrow the search if a particular vendor does not work (or work well) within your environment.
Do your research. Now it is time to research the various vendors and products that will best serve your testing needs. Find tool lists at sites like StickyMinds.com and qaforums.com. Check out various vendor websites that specialize in testing. In addition, use Google and other search engines to find test tools, and to search for open source alternatives. Again, never rule out a