Case Study in a Synthesis Compiler Test System

[article]
Uses of Test Automation Tools

default timeout, but some test drivers can overwrite defaults.

Test distribution between different machines can be supported, too. If you have a test lab with more than one machine, you can either distribute tests manually by dividing them into a few groups and running different test groups on different machines, or you can write scripts that distribute tests. In this test system, you have one test server and a few test clients. The main script is started on the server. A "client_ready" script is started on each client. The client_ready script does the original setup on the client computer and then takes "orders" from the server script. A script run on the server sends the test driver to the client. The client runs the test driver and passes the pass/fail (expected/unexpected) result to the server. The server script gets information from the client and reports the results. It also checks to see if the client machine is available to start a new test.

If the software you are testing is supported on multiple platforms, your test system should run on multiple platforms as well. Our synthesis compiler, for example, was working on the UNIX, Win95, Win98, WinNT, Win2000, and WinXP platforms.

Problems We Experienced Developing and Using Our Test System

  1. We didn't budget for the changes. We were just "sneaking the changes in." So when we hit release mode, the test system was not ready and was very unreliable. It began breaking in several places and it was a disaster. Management was very disappointed. It took us some additional time to clean things up.
  2. Only three people in our software group used the system. That was a huge mistake! More people could be familiar with the test system if we had better documentation sooner. And the system should have been available to developers so they could use it for their testing.

The system I described in the paper is very flexible. I have used this test system now for several years. It has demonstrated (after it became stable) significant advantages compared to other systems.

References
If you want to implement the system I described in the paper, learn about two tools: 

  1. Software Testing Automation Framework (STAF) at staf.sourceforge.net. STAF is an open source, multi-platform, multilanguage framework designed around the idea of reusable components.
  2. askMaster, developed by my former coworker Phil Tomson.

Acknowledgements to Phil Tomson for important contribution to this work.

Tags: 

About the author

Galina Golant's picture Galina Golant

Galina Golant has seven years of experience in the field of software testing. During this time, she has tested Web servers and browsers, voice recognition systems, Oracle database applications, synthesis compiler tools, and GUI applications. She has developed new tests and written test documentation. Her favorite task is test automation. For the last four and one-half years, she has worked for Cypress Semiconductor. You can contact her at golant@speakeasy.org.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

May 04
May 04
May 04
Jun 01