Five Ways to Make Test Automation Fail


Test automation promises much, but it can deliver disastrous results if implemented poorly. Laura Salazar takes a look at five common practices that can cause automation projects to fail.

Automation testing is undeniably one of the key strategies for any QA manager, and for good reason: Automation promises faster regressions, high productivity, good quality, and reduced costs. However, many QA managers fail to reach those results. They face late deliveries, acquire expensive tools, and deal with a lot of frustration. But what are the causes? In this article, I will review five of the most common practices that cause automation projects to fail.

top down1. We do not need an automation plan.
“Let’s start scripting ASAP,” says the QA manager.

Imagine this: You and your coworker just arrived in the US to perform a knowledge transfer. At the airport, you suggest looking for a road map, but your coworker says, “That’s not necessary. I’ve been here before and can find our way around.” You decide to put your concerns aside, get into the car, and start driving. After two hours on the road, you’re hungry, angry, and your gas tank is running on empty. You stop at a gas station to buy a map and ask a clerk for directions.

Sound familiar? I can assure you the same happens when you don’t have an automation plan—and you are not going to find one in a convenience store. An automation plan helps you attain a clear view of your scope, the resources that you need, the roadmap that you have to follow to achieve your objectives, and a way to measure how far you are from your goal, provide focus, and identify risks.

2. Let’s use the best, most powerful automation tool in the market.
“This guarantees our success,” says the QA manager.

Now, think of your dream car. For example, a Ferrari Testarossa—a high-quality piece of engineering with almost 300 horse power and luxurious interiors that costs half-a-million dollars. Now, picture that fine machine parked in front of an elementary school, 300 HP on hold. The mom in the driver’s seat is waiting for her children, trying to put on some make up, and singing to the baby in the back. The baby is spilling milk and passing his sticky fingers over the luxurious seats. The sweaty boys outside in their soccer gear are scratching the car and damaging the paint job.

I do not have anything against SUVs; I just think they are called “mom-mobiles” for a good reason. They are perfect for that kind of use. The same applies to automation tools. The most complete, powerful tool is not necessarily the best tool for your project.

User Comments

Jim Hazen's picture
Jim Hazen

Yep, it is all about misconceptions and false perceptions of what test automation and the tools that implement are about.

The fallacy of getting a tool and it solving all your problems is still an issue today as much as it was 20 years ago when the GUI based tools appeared.

My two favorite sayings apply here: "It's Automation, Not Automagic!" & "A fool with a tool is still a fool."

July 30, 2012 - 11:20am
Juergen Brueckler's picture
Juergen Brueckler

I see a software development project including test automation as a truck with trailer running down the road.

The truck is being steared by the project lead, software architects - whoever.

Inside the truck there's the development team piling up boxes.

Inside the trailer there's the test (automation) team also piling up boxes.

Very often the project lead makes the test automation team believe that they are driving down a highway without any curves. So the automation team does the best it can and builds a beautiful small but high pile of boxes.

Unfortunately the QA manager has not been invited to the meeting where the project roadmap has been discussed and he does not know that the road will make some 90 degree curves within the next few versions.

The developers will be informed and they start re-piling their boxes.

The test automation engineers (in the trailer) do not see what's going on in front of them and are happily continuing to pile up boxes.

Then comes the curve - the next version - and BANG all the piles tumble around the trailer. Test execution for this version: Only 20%, all other tests are broken. Maintainance effort to get it fixed again: Not planned in project.

I've been inside the trailer a few times before.

My lessons learned: If you feel that information flows poorly within the project, keep the pile low and wait for the first curves to come. They will come definitely!



July 31, 2012 - 8:28am

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.