Working With Automated Testing Tools

A Pragmatic Approach
Member Submitted

specified below. The reader should note that it might be necessary to purchase more than one software solution from a vendor or vendors to meet the various testing needs of the project. The criteria is enumerated below.

  1. Automated testing tools should have version control capability
  2. Automated testing tools should have workflow capability for the reporting and closing of defects.
  3. The recording testing tool should recognize the custom controls, objects, GUI, and generic controls (i.e Active-x controls) for the application under test to allow playback of the recorded scripts.
  4. Automated testing tools should allow for the sequencing of test sets with dependencies for the execution of the automated test scripts. For instance execute test script B only after test script C has successfully completed.
  5. The recording testing tool should have a scripting language that is widely accepted, robust and recognized (i.e. Visual Basic).
  6. The recording testing tool should produce reports to verify the execution of the scripts and provide a means to store the execution reports.
  7. The automated testing tool should be compatible and integrated with standard word processors and spreadsheets
  8. The vendor of the automated testing tool should offer online support to allow the customer to report identified problems or bugs with the automated testing software.
  9. The recording testing tool should work with external/internal data sheets, to allow for the creation of parameterized data driven scripts.
  10. The automated testing tool should have the capability for report generation to track and collect metrics for the number of scripts that passed, failed, number of defects open/closes, number of test cases that have been developed, etc.
  11. The automated testing tool should have email notification for the reporting and closing of defects.
  12. The automated testing tool should have an open architecture that makes it flexible enough to modify it to extend the tool’s functionality.
  13. The automated testing tool should have the capability to allow storage of automated and manual scripts and to serve as a repository for test artifacts.

Again the reader should be aware that the delineated criteria 1-14 above is by no means an all exhaustive list of attributes that the automated testing tools should possess before acquisition but rather the aforementioned criteria should serve as a baseline for the minimum features that the automated testing tools should possess.

1. Straightjacket effect:
Companies that fall under this category purchase all their automated testing tools from a single vendor and make a significant commitment of time, money and resources to the automated testing solutions from a single vendor. The company is highly dependent on a single vendor for all their automated testing needs. Although, this approach may be cost effective if a single vendor is capable of meeting all of a company’s testing needs, in actuality this is highly unlikely due to a company's bevy of heterogeneous IT applications. The test manager and the company may instead opt for a hybrid solution of automated testing tools where automated solutions are procured from 2 vendors or more to avoid the straightjacket effect of working with a single vendor and its potential limitations. Below is an example from an actual project where the straightjacket effect hampered one of my client's ability to automate test cases.

Example: My client, a consumer products company for which I did consulting for had spent over 500,000 dollars in automated testing software from a single vendor. The company bought a bolt on to their existing ERP system running in production and wanted to test the ERP bolt-on with their automated testing tools. The vendor offered an add-in program at a cost of an extra $10,000 dollars that supposedly could recognize

About the author

Jose Fajardo's picture Jose Fajardo

Jose Fajardo (PMP, M.S., and SAP certified) has worked as a test manager for various companies utilizing automated testing tools. He has written and published numerous articles on testing SAP and authored the book titled Testing SAP R/3: A Manager's Step by Step Guide. Throughout his career Jose has helped to create testing standards and test plans, mentor junior programmers, audit testing results, implement automated testing strategies, and managed test teams. Jose can be contacted at

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!