Automation Déjà Vu—Again!


Don't get me wrong; there's nothing wrong with asking these and other questions, particularly if you are new to IT, software testing, or even test automation. The problem, however, comes when these questions linger, seriously delaying the effective implementation of test automation and, even worse, leading many down the wrong path with regards to test automation. In addition, the over-preoccupation with these questions that have been asked over and over again for more than ten years—despite the fact that some relatively widely accepted answers are available—is one of the major symptoms of test automation's stunted growth. This stunted growth is also evident both in the fact that shelfware remains prevalent and in the missed opportunity to address more pressing concerns. Over the years, we have failed to come up with comprehensive solutions to several important automation issues, such as:

  • Detailed calculations for framework selection
  • Detailed Calculations for Automated Test Development and Maintenance Times
  • Making Risk-based (Quality-based) ROI Calculations More Acceptable
  • Moving to a Fourth Generation Automation Framework
  • Devising a Good Answer for an Acceptable Percentage of Tests that are Automated

Detailed Calculations for Framework Selection
Below is a formula that I often use to help define the level of complexity an automated test framework should have:

= AN + VN + BN + TN + CN
+ EN + Ti + P + R

  • AF = Automation Framework Definition
  • AN = Number of applications expected to be tested by your organization
  • VN = Number of versions/releases that each application is expected to have
  • BN = Number of builds/nature of application changes that each application build is expected to have
  • TN = Number of tests that you're expecting to automate for each application
  • CN = Number of configurations that an application may have to test
  • EN = Number of environments in which tests will be executed
  • Ti = Time period over which the tests will need to be maintained
  • P = Organizational process maturity
  • R = Resource technical level

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.