Automated Software Testing

How I fought the Software Testing Automation War and have Determined I'd Rather be a Stone Mason
Member Submitted

example, the job description in Figure 1 below is from one of the St. Louis companies I mentioned at the beginning of this article. I was personally interviewed for this position and during that interview it came to light that there were actually two positions and that both were for test implementers not test designers. In fact, one position was from the beginning of November until the end of this year and the person who was to be hired must demonstrate substantial progress towards a set of regression tests by the end date. The only relevant parts of the description are the phrase “the creation of manual and automated test scripts” and knowledge of SQL and the DBMS as required skills. The rest of the verbiage refers to test planning and test design rather than test implementation. Why does a test scriptwriter need to have a working knowledge of the software development Capability Maturity Model (CMM)? There is no reason.

Needless to say I was not contracted for either of those openings. One reason was because I proceeded to explain to the interview committee that their automated testing effort was doomed to failure because they were ignoring most important aspect of automated test development, the design of intelligent tests. I explained why and offered my services in that area, but to no avail. They were determined to start writing test scripts with out doing the planning and design that leads to a successful automated testing implementation.

The point is that a suite of automated tests that shoot in the dark at the application to be tested are no better than random testing, or no testing at all. With respect to our geographic area and the companies that service it, this situation is QA and Testing management’s fault. When will these individuals take up the torch and do what it takes to implement software quality control procedures that are effective? I have seen two management types; those with no clue and those who pay lip service, but lack the follow through to have a successful automated testing function. As I said earlier, I have worked in a number of IT organizations and QA/Testing departs as a contactor and with one exception, I have yet to see test automation done properly.

With their tight/reduced budgets these same managers are, by not employing test designers to create smart tests, which attack known application weaknesses, not specifying what targets the test scripts must assault. Think about the war in Afghanistan. If the spotters were not on the ground to guide the high tech laser bombs, they would not hit the terrorist targets. All of the bombing would be ineffective unless they resorted to “carpet’ bombing such as was the case in World War II which required dropping tons of bombs at generalized targets in order to achieve results similar to those derived through the so called smart bombs. Carpet bombing requires tremendously more effort and resources.

The laser bombs are focused much as data driven tests are focused when they attack the application you are testing. Their accuracy is unprecedented and their target demolition rate is very high. The same can be said for the defect finding rate of focused tests such as those test data developed for data driven test scripts. An automated test suite that runs test that are not based on knowledge of who the application works are more akin to carpet bombing the application. A much larger number of tests is required and the defect finding rate of the tests is much lower, less than one third of the tests

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.