for the testing phase to make sure it was linked with early-stage development and met business objectives.
- Develop business metrics. This is where the performance criteria for the site are defined. Good performance criteria describe an end user's experience in relation to a certain load. For example: "While under the load of 5,000 concurrent users each page on our site will load within eight seconds of being requested."
- Establish requirements and expectations. Clearly define the test objectives. Specify what will be reported.
- Document performance methodology. Describe exactly how the test defined below should be run.
- Get management buy-in. This part is up to you, but the earlier steps provide a ogical basis for a solid presentation to management.
- Choose whether to outsource or bring dedicated resources, such as hardware, software, and personnel, in-house; either way, one must
- make sure testing activities are automated, that is, able to run without manual influence
- establish dedicated test data
- document results
But what should you actually test Web applications for?
- Functionality testing ensures that all aspects of the site function properly: that objects such as pictures, text, and forms appear correctly, links work, form submissions succeed, etc.
- Compatibility testing ensures functionality with different browsers and operating systems.
- Usability testing measures the ease with which a user can accomplish predefined tasks.
- Stress testing determines the system's breaking point based on predefined failure criteria.
- Load testing generates user traffic on the Web site to determine if the site is capable of handling a predetermined peak load.
Essentially, testing should exercise and evaluate site performance from a user's perspective. Some companies know the value in testing peaks in traffic after a promotion or a Super Bowl advertisement, for example, but what about the unexpected catastrophe, or a run on the stock price? Reliability and load testing can be important supporting factors to a company's crisis-communication and investor-relations plan. It is also important to remember that functionality and scalability testing are not separate testing activities, and need to be performed concurrently for a realistic emulation of the live Internet.
Testing performance under a virtual load of Internet traffic prior to being posted on the public Internet is vital to gain a clear understanding of how a Web application will perform when it does go live. For example, another large, well-known financial services organization detected certain problems only when they tested with 500 concurrent users, problems that would have gone unnoticed if they had merely performed scalability testing. They found that when a certain Web application supported a single user to 499, users were able to receive correct financial information online. However, once the concurrent users reached 500 and above, financial records and confidential information were given to the wrong users. The large financial services company became aware of these issues only when testing functionality under heavy virtual loads. * In turn, they were able to correct the problem prior to the Web application going live and public, and before it would become a significant cost to fix.
Industry analyst firm Newport Group believes that the structure and management of those testing practices is in need of improvement. For example, 51 percent of the e-commerce companies surveyed reported that no test standards were followed within their organizations. And 83 percent of that survey population segment indicated that there were no plans to implement or adhere to testing standards within the next twelve months.
Ultimately, testing Web applications-large or small-early and often-requires a level of expertise that few companies are able to develop in-house. But QA professionals, testing consultants, and testing labs are available to help companies develop