Software Testing Enters the Cloud

Unless you've been under a rock, you've likely heard of cloud computing. But, you might not have a clear picture of how it impacts you as a tester.

Oh, sure, the impact is clear for programmers: The services they build will be different. Likewise, there is a clear change for system administrators and the folks in operations. In testing, we mostly get to hear slogans and feel anxiety.

But, cloud computing offers new forms of tools that can be valuable for testers. We should learn to use them.

What Is the Cloud?

Ten years ago, you could go to www.google.com and get results. You didn't exactly know where the Google web server was. Google was running a number of server farms, and you didn't care which server provided your request. For that matter, you might search on something else five seconds later and hit a different machine.

In that sense, the cloud has been a reality for years. If we count Telnet and email, it's been a reality for decades.

Yet, in another sense, the cloud era is very new. Now, we get to build the cloud-based services for ourselves. Like Google spread its work over 10,000 computers, we testers may, in some ways, spread our work out.

This brings two very different ideas to the masses. The first is virtualization, which allows us to combine the services of many computers onto one physical machine. The second is distributed computing, harnessing a large number of computers as one.

Opportunities

In the bad old days, we used to have a thing called a test lab. It took up hundreds of square feet of space and contained between five and twenty computers. Each computer had different combinations of system software. We would have to install, test, and then possibly re-image the machine for the next test run.

Even if we only had to test different browsers, we'd likely need two different computers. Add different browsers and different versions of software—say, Microsoft Word to test cut and paste—and you'd be introducing a combinatorial explosion.

Today, you can use system software like VMWare Fusion (plus a big hard drive) to get all those systems running on one computer. Right now, for example, I have three versions of Internet Explorer and three versions of Firefox available on my MacBook, plus Safari running native on the Mac. Re-imaging a “clean” device takes a half-dozen clicks and a thirty-second pause.

Likewise, if you are testing server software, getting test equipment can be expensive. Even one server per tester can be prohibitive, but several servers in order to check critical fixes? Forget about it.

Once again, virtualization comes to our rescue, with the ability to run several instances of the server software on one physical device. OpenVZ, for example, is a Linux virtualization tool with a scriptable interface. Combining the scripts with your install could make creating a test environment a one-line command. (We used OpenVZ at my last position at Socialtext, where building an environment was literally as easy as running “vest -v boxname -r buildname”).

We can get even more benefits by adding distributed computing into the mix.

Consider, for example, the typical test-GUI-driving test-automation strategy. Over time, the test programs tend to take longer and longer. Eventually, something happens—a test run stops in the middle for no apparent reason, developers check in fixes during a test run, or the lead manager doesn't want to wait forty-eight hours for the two-year-old test suite to finish. Suddenly, we have a problem.

Now, it's time for distributed computing to come to our rescue. Imagine splitting the test