large-scale Web applications that are reliable, scalable, and in keeping with a company's critical business objectives. The testing and test planning can be done remotely without the need to travel to a physical site. This can keep costs down considerably, even those with very large-scale Web applications.
KeyLabs, for example, evaluate testing methods before actually running the test-monitoring how a site performs in combination with the test solution, the number of machines participating in the test, and the geographic locations where the load generators are located.
To minimize the risk of major structural issues being discovered late in the lifecycle, consultants should advise clients to perform scalability testing well before the bulk of the program's data coding has been completed and design decisions have been finalized. This makes it possible to flesh out all the scalability issues from the top to the bottom-in each application tier, subsystem, or component-well before final integration takes place.
To provide real-world tests, I advocate taking a structured approach-doing multiple tests, testing on different days, and conducting multiple test rounds. This gives site and application developers time to analyze performance statistics they are getting, optimize the back-end processes they are running, and fix bugs. It often takes two or three rounds of testing for companies to reach their targets. To thoroughly troubleshoot before going live, I recommend not just one test, but a true testing process that builds solid Web sites.
Industry analyst firm Hurwitz Group supports the integration of testing saying, "Functional testing is the traditional turf of QA. No matter how well the QA team performs its functional testing tasks; however, the ever-broadening scope of e-Business puts new demands on the entire development team…Quality processes must adapt and evolve to extend functionality and quality into this new environment…Technologies and procedures that were once limited to QA require distribution across the lifecycle."
In planning for the unknown factor of the World Wide Web audience, consultant professionals should also account for the international factor. Because Europe and the Asia Pacific regions have rushed to embrace the Web almost as quickly as the US, it is all the more important for consultants to factor global testing into their development cycle. Global access to their customers' Web applications means different amounts of latency, meaning the time between initiating a request for data and the beginning of the actual data transfer, which cause the connections between the browsers and the server to take different lengths of time. A realistic test environment that factors in these latencies and time lag is critical for real-world testing.
Scalability: How Large Is Large?
In mapping out a test plan, a consultant's first step is often accounting for scalability. The scale of the testing process varies, depending on the complexity of the Web site. Most new large-scale companies are initially shooting to test between 1,000 to 5,000 concurrent users; however, this is a test that typically many organizations cannot conduct themselves. It would cost them more to test accurately in-house than to build out the back-end production infrastructure.
In determining that the tests will be effective for the site in question, consultants should keep in mind one of the biggest factors-the number of users that can run on a specific load generator or specific piece of hardware. Every Web site is different, and the actions that the user takes on the Web site are different. Load generators emulate the multiplicity of users and their various Web browsers and connection speeds. A seasoned testing consultant or testing organization should plan ahead to create test scripts that are able to test the Web site as