Because of the problems with change, I've seen test automation efforts that try to minimize the changes in test strategy and staffing. Invariably, however, this requires architecting the automation inefficiently. Lots of extra scaffolding is constructed to make the automation run the tests just like the manual testers. And they suffer from the fact that automation just can't be as discriminating as a pair of human eyeballs. So things still get missed.
The approach I now take is to try and follow the existing test strategy to the degree that it does not burden the test automation, but then to insist on the strategy changes that are required to allow the test automation to be reliable and efficient. These test strategy changes may make some managers uncomfortable. They may require changing how some tests are executed or how test results are verified. And to avoid being overwhelmed by false alarms, fewer things are checked.
I can understand why this may be more than some managers are prepared for. Sometimes test automation is seen as just a faster and easier way to get what you are already getting with your manual testing. That is not how test automation works. I avoid projects where I can't change these expectations.
My previous column also discussed the importance of testers and developers working closely together. With a late start this also will be even more critical. I have yet to work on a software project that didn't see significant technical problems in getting necessary testability hooks into the software. With a mature product, these testability hooks may need to be inserted into code that developers would otherwise not want to touch. Will your developers and managers be willing to potentially destabilize your code in the interests of assisting the automation project? If not, forget it.
Thus late starts with test automation entail changes to established testing procedures and changes to tested code.
Early in a project, changes in code or strategy are less risky, because we have less confidence in what we are changing from. But with time, change becomes more difficult. I'd estimate that the difficulty increases by a factor of three or four. And our test automation project now requires additional skills. Now we need a change agent, someone who is effective at shepherding an organization through necessary changes. It's a lot to ask.
Three Keys to Test Automation, Bret Pettichord, StickyMinds.com, December 2000