Software Testing Enters the Cloud

Opportunities and Challenges

suite into a dozen tests (or a hundred) and spooling up fifteen computers, each to take a test, report back with results, and then take another test. If you max out a virtualization tool, you can rent space from a cloud-hosting provider for pennies per processor-hour. A test run only takes as long as your slowest test.

Or, consider model-driven test automation , which defines input and valid paths and then takes random walks through the software, looking for inconsistencies. Imagine using cloud computing to do a model-driven test run on a new build over lunch—not overnight.

Is it just me, or is the future looking pretty bright? But wait, there's more!

Sure, you can rent space and use your own tool, but there is also a host of companies offering tools to do it for you with cloud computing. These range from load testing to functional testing to automatic backup. If you've ever wanted to press a single button and have dozens (Hundreds! Thousands! Tens of thousands!) of computers from all over the world attack your website, you can hire someone to do that. Just like tapping into your electric grid, if your socket is big enough, you can have a lot of power—you've just got to pay for it.

The cloud also enables a new sort of software: zero-install applications delivered through your browser. While these might not be considered classic “cloud computing” platforms, they still mean more and more organizational data is being shared on a bunch of computers "somewhere over the Internet." This means we'll be able to access more and more tools without an installer and share those tools and their data when we are at home, on the road, or even on our smart phones.

Limits and Challenges

Cloud computing has also made some huge promises on the development side. One of those promises is that as usage ramps up, your company will automatically be able to spool up new servers and redirect requests to the new servers. That means you'll need some sort of monitoring of performance and system resources, as well as some code—or to be able to rent it from a cloud-provisioning vendor.

Likewise, as a tester, you'll want to ask, "What happens if the cloud fails?" This is no idle question; a recent Elastic Compute Cloud outage automatically brought down a half-dozen name-brand services, including Reddit and FourSquare. The bad press appeared on mass-media outlets including CNN and The Economist magazine. Many Web 2.0 companies have decided to live with apparently random downtime for apparently random intervals; will your company be one of them?

You'll also want to worry about backups. What happens if the cloud service fails and restores from a backup that is an hour old? A day? A week? What if some of the data is just “lost”? Who owns it? How much risk exposure will your company take on, and how can you mitigate that risk? This problem has happened to me and may happen to you, and these are the open-ended, investigative questions that testers can help answer.

In addition to reliability, cloud computing suffers from another challenge in security. True "cloud computing" generally involves renting servers from some outsourced provider. That means migrating your data to the provider, which can involve questions of trust, security, and, in some cases, federal regulation. Some of these are compliance issues, some legal, but a good tester who can analyze and communicate risks could be invaluable to this process.

Finally, make no mistake: A cloud conversion will cost time and money. Building test tools will take time


User Comments

Michael Czeiszperger's picture
Michael Czeiszperger

That's a great overview of cloud testing. We're using the cloud two ways-- to generate lots of load, and to host the servers under test.

The angle we haven't tackled yet is how to test infinitely scalable cloud-based servers. The best I can come up with so far is to use one-cloud based service to test another, one cloud hosting the servers, and the other generating the load.

June 1, 2011 - 10:34am
Mark Cole's picture

What is the easiest and cheapest way to bring up a Windows Server 2008 R2 Standard VM instance on the Cloud?

June 3, 2011 - 10:56am
ruchi nayyar's picture
ruchi nayyar

This is a good overview on Cloud computing.

I'll be interested in knowing how many companies have already started testing services on cloud computing.

Also, whats the view from financial firms on adoption/migration to cloud computing??

June 7, 2011 - 12:29am

About the author

Matt Heusser's picture Matt Heusser

The Managing Consultant at Excelon Development, Matt Heusser is probably best known for his writing. In addition to currently serving as managing editor of, Matt was the lead editor for "How To Reduce The Cost Of Software Testing" (Taylor and Francis, 2011). He has served both as a board member for the Association for Software Testing and as a part-time instructor in Information Systems for Calvin College.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!