Does Test Automation Save Time and Money?


We've all heard the pitch: test automation saves testing time and resources. Test tools can execute tests faster than a person can, and in most cases they can do so in an unattended mode. So test automation should reduce test cycle time or the number of testers needed. Right? Not exactly. But Linda Hayes will tell you what it does save.

Many a test tool investment has been justified by a handy little spreadsheet that calculates the return on investment based on the reduction in the number of resources and amount of time being spent on testing. It's common to hear that test automation saves testing time and resources.

But is this true? Do companies really reduce their testing headcount and test cycle time as a result of automation? In my experience, the answer is no. Frankly, I have never seen a company reduce their staff or their schedule because they bought a test tool. Test tools do not save testing time or resources.

But does this mean that test automation has no value? Again, the answer is no. The key is to understand the true purpose—and value—of test automation. I submit that automation can save more time and money than are budgeted for the entire test organization and test cycle. How can that be, you ask?

Investment in Automation
Granted, automation is not cheap. Not only does it take time and effort to evaluate and select a test tool: it takes money to buy it and time to learn to use it; and more time to actually implement it; and the most time of all to maintain the test library as the application changes. In fact, experience shows that it takes between five and ten times as long to develop and debug an automated test as it does to execute it manually.

Okay. So you have to make the investment to get the return, and in most cases you will execute the same test at least ten times over the life of the application or even within the same release cycle. So, what's the problem?

The problem is that the goal of test automation should not be to reduce either test resources or cycle time. The goal should be to reduce the risk and cost of software failure by increasing test coverage.

Think about it. If your management's primary goal is to reduce the cost of testing, then I can propose a foolproof way of cutting 100 percent of your testing budget without even buying a tool. It's simple: stop testing! Just quit it. You can save all that time and cut all the staff. No problem, right?

Well of course this is a problem, you say. Then we would be shipping defective software, and that could lead to production downtime, customer crises, and all manner of development and support costs.

Precisely. It's really all about reducing the risk and cost of software failure.

User Comments

1 comment
George M's picture
George M

In the services sector also the clients are repeatedly asking for automation. In my practical experience I have been asked by a client a hypothetical question

"If you had used automation last year how much cost saving would you have made". The question lingers in my mind even after long and I have not found an answer that is technically and tactically correct.

People like us from the manual testing world find it difficult to do value additive work in this 'automated workshop'.

October 29, 2013 - 6:16am

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.