Software Testing Enters the Cloud

Opportunities and Challenges

What impact does cloud computing have on software testing? Matt Heusser takes a look at some of the opportunities, such as virtualization and distributed computing, and challenges associated with testing in the cloud.

Unless you've been under a rock, you've likely heard of cloud computing. But, you might not have a clear picture of how it impacts you as a tester.

Oh, sure, the impact is clear for programmers: The services they build will be different. Likewise, there is a clear change for system administrators and the folks in operations. In testing, we mostly get to hear slogans and feel anxiety.

But, cloud computing offers new forms of tools that can be valuable for testers. We should learn to use them.

What Is the Cloud?

Ten years ago, you could go to and get results. You didn't exactly know where the Google web server was. Google was running a number of server farms, and you didn't care which server provided your request. For that matter, you might search on something else five seconds later and hit a different machine.

In that sense, the cloud has been a reality for years. If we count Telnet and email, it's been a reality for decades.

Yet, in another sense, the cloud era is very new. Now, we get to build the cloud-based services for ourselves. Like Google spread its work over 10,000 computers, we testers may, in some ways, spread our work out.

This brings two very different ideas to the masses. The first is virtualization, which allows us to combine the services of many computers onto one physical machine. The second is distributed computing, harnessing a large number of computers as one.


In the bad old days, we used to have a thing called a test lab. It took up hundreds of square feet of space and contained between five and twenty computers. Each computer had different combinations of system software. We would have to install, test, and then possibly re-image the machine for the next test run.

Even if we only had to test different browsers, we'd likely need two different computers. Add different browsers and different versions of software—say, Microsoft Word to test cut and paste—and you'd be introducing a combinatorial explosion.

Today, you can use system software like VMWare Fusion (plus a big hard drive) to get all those systems running on one computer. Right now, for example, I have three versions of Internet Explorer and three versions of Firefox available on my MacBook, plus Safari running native on the Mac. Re-imaging a “clean” device takes a half-dozen clicks and a thirty-second pause.

Likewise, if you are testing server software, getting test equipment can be expensive. Even one server per tester can be prohibitive, but several servers in order to check critical fixes? Forget about it.

Once again, virtualization comes to our rescue, with the ability to run several instances of the server software on one physical device. OpenVZ, for example, is a Linux virtualization tool with a scriptable interface. Combining the scripts with your install could make creating a test environment a one-line command. (We used OpenVZ at my last position at Socialtext, where building an environment was literally as easy as running “vest -v boxname -r buildname”).

We can get even more benefits by adding distributed computing into the mix.

Consider, for example, the typical test-GUI-driving test-automation strategy. Over time, the test programs tend to take longer and longer. Eventually, something happens—a test run stops in the middle for no apparent reason, developers check in fixes during a test run, or the lead manager doesn't want to wait forty-eight hours for the two-year-old test suite to finish. Suddenly, we have a problem.

Now, it's time for distributed computing to come to our rescue. Imagine splitting the test


User Comments

Michael Czeiszperger's picture
Michael Czeiszperger

That's a great overview of cloud testing. We're using the cloud two ways-- to generate lots of load, and to host the servers under test.

The angle we haven't tackled yet is how to test infinitely scalable cloud-based servers. The best I can come up with so far is to use one-cloud based service to test another, one cloud hosting the servers, and the other generating the load.

June 1, 2011 - 10:34am
Mark Cole's picture

What is the easiest and cheapest way to bring up a Windows Server 2008 R2 Standard VM instance on the Cloud?

June 3, 2011 - 10:56am
ruchi nayyar's picture
ruchi nayyar

This is a good overview on Cloud computing.

I'll be interested in knowing how many companies have already started testing services on cloud computing.

Also, whats the view from financial firms on adoption/migration to cloud computing??

June 7, 2011 - 12:29am

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.