Working With Automated Testing Tools

A Pragmatic Approach
Member Submitted


The return on investment on automated testing tools is best qualified as a series of intangible benefits since demonstrating how a particular automated testing tool saves money over manual testing might be an intractable challenge. Many vendors drum up their automated testing tools with a plethora of benefits that few projects ever actually manage to gain.

The main benefits of purchasing automated testing tools and using them to automate test scripts are: 1. Consistency, 2.Under certain conditions (i.e. regression testing, and benchmark testing) a previously recorded test script is repeatable, 3. Automated test scripts may help save time during the test execution phase, 4. Electronic storage of test results after test execution, 5. Some automated test tools produce graphs, results that would be inconvenient to produce by hand, 6. Execution of test scenarios with multiple data sets for parameterized data-driven test scripts, 7. Integration testing of various business processes for correlated test scripts, 8. Automatic sequencing of test sets with dependencies, 9. Automatic validation and verification of business objects for the application under test and 10. Batch scheduling of test scripts for execution. There are various other benefits gained from using automated testing tools such as: the workflow activities during the reporting and closing of defects, the automatic collection of test metrics from test management tools, the mapping of requirements to test cases, a central repository to store all test scripts and automated test scripts, etc. Again this is not an all-inclusive list of benefits of working with automated testing tools but just some of the salient benefits of working with automated testing tools.

A company may save money by not hiring as many testers to perform test execution during the test execution phase after the company purchased an automated testing solution and thereby lowering the overheard costs associated with test execution and this may be perceived as a return on investment from the purchased test tools. But what about all the costs associated with maintaining the automated test tools, and the costs of hiring technical testers to produce and maintain automated test scripts? These costs may easily outweigh any savings incurred by not hiring as many testers to execute test scripts by hands and thus the return on investment from the automated test tool may prove difficult to quantify.

However I do think that over time although the automated test tools might not generate any tangible ROI the test tools do bring the project a level of convenience and assurance that may not otherwise be available in the absence of an automated testing tool.

As an example of the convenience that the automated tools may bring to a project I will relate my experience from a previous project. I was at a project where a tester executed test scripts by hand during the middle of the night and the tester had to verify the test results before the morning shift started. I created an automated test script for the same process that this tester executed by hand and embedded programming error handling logic within the script to send an alarm to a pager if the application under test encountered a given set of errors during testing. This tester no longer had to work the night shift to manually test the application since I had automated the process for him and I embedded test script logic to alert him if anything fails with the application during the batch execution of the test. Needless to say the automated testing tool brought much convenience to this tester.

When built correctly automated test scripts that are data driven can cover more

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.