Open Source Test Automation

Member Submitted

In this article Babu Narayanan outlines the open source test automation (OSTA) procedures. He also describes the procedures for building and adopting open source scripting and how to customize it for your business needs. Babu also explains the benefits and limitations of OSTA to the software industry, open source frameworks, and ways of accessing and implementing open-source scripting.

The breadth, depth and iterative nature of testing in today's complex application environments does not allow comprehensive testing to be carried out unless the test group has an automation tool for testing. Despite its best intentions, the test group simply does not have the bandwidth to manually execute all the test cases developed.

Even if the test group invests in a commercial test tool, the steep cost of acquiring adequate number of licenses and tool inflexibility prohibits testing at a satisfactory level of efficiency predetermined by the test group. With these identified issues, the most convincing solution seems to lie in adopting open-source test tools and customizing them for in-house testing needs of the organization. Test tool functionality is extended to be time-tested and adaptable to changing technologies, thus assuring a longer shelf life, which extends beyond the lifespan of the development work of a few heroic individuals.

This paper introduces you to Open-Source Test Automation (OSTA) and outlines the procedure for building and/or adopting open-source scripting and customizing for the business needs. It also lists the benefits of OSTA to the industry, its benefits and limitations. In addition, the paper summarizes open source framework and ways of accessing and implementing open-source scripting.

Problem Statement
Despite of huge investment on commercial test tool for automation, their licensing defeats the very purpose of the usage of a test tool. The commercial licensing models keeps the test tools away from the people (like developers, manual testers) who could most benefit from them.

The need for customers to share their test tool licenses to offsite developers has also increased due to tremendous growth in off-shoring practices. This causes the tool user to pay additional overhead to their commercial vendors, based on agreed licensing models. In addition, the customer requires re-licensing of tool for each technology being tested, by which customer pays more than required.

There is a forced need to write test scripts exclusively in the tool's scripting language. This forces the user would need to learn the vendor script in order to use the tool. The tool user should write automation scripts even though tool recorders help generate code for making it run for more reasons than one. The test automation framework written for specific commercial test tool does not allow the user porting to another test tool as neither of them shares a common platform.

Most of the commercial tools are vendor locked with other test support tools like defect tracking, source controlling and test management tools, thus forcing the user to buy those tools for test management and integration. The user may end up in investing more than expected.

Moreover, the commercial test tools have many other limitations such as 'the test scripts are only foreground runners thus consume a machine completely for each test execution instance'. They have been criticized for bloated code, consuming larger memory and more processor cycles with successive tool version releases. There is commercial pressure on the buyers too to purchase the latest tool version due to increase in functionality and bug fixes.

Thereby, commercial test tool(s) causes greater concerns to test automation industry by unkind licensing models, closed source code, inflexible scripting languages and interoperability restrictions which impacts testing and business.

Proposed Solution
The basic idea behind open-source software is very simple: "When the programmer can read, redistribute, and modify the source code for a piece of software, it evolves. Others improve it, adapt it, customize it, and fix bugs. And this can happen at a speed that, if one is used to the slow pace of conventional software development, seems astonishing" .


About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.