attributes of an e-commerce site that will not be easy (or even possible, in some cases) to validate in this way. Some form of final testing that can address issues such as performance and security needs to be included as a final confirmation that the site will perform well with typical user interactions. Where RAD is not used, the scope of the provider’s internal testing coverage and user acceptance testing coverage should be defined early in the project development lifecycle (in the Test Plan) and revisited as the project nears completion, to assure continued alignment of goals and responsibilities. UAT, however, should not be seen as a beta-testing activity, delegated to users in the field before formal release. E-commerce users are becoming increasingly intolerant of poor sites, and technical issues related to functionality, performance or reliability have been cited as primary reasons why customers have abandoned sites. Early exposure of users to sites with problems increases the probability that they will find the site unacceptable, even if developers continue to improve the site during beta testing.
Principle 7. Regression testing. Applications that change need regression testing to confirm that changes did not have unintended effects, so this must be a major feature of any e-commerce testing strategy. Web-based applications that reference external links need regular regression testing, even if their functionality does not change, because the environment is changing continuously. Wherever possible, regression testing should be automated, in order to minimise the impact on the test schedule.
Principle 8. Automate as much as possible. This is a risky principle because test automation is fraught with difficulties. It has been said that a fool with a tool is still a fool, and that the outcome of automating an unstable process is faster chaos, and both of these are true. Nevertheless, the chances of getting adequate testing done in the tight time scales for an e-commerce project and without automation are extremely slim. The key is to take testing processes sufficiently seriously that you document them and control them so that automation becomes a feasible option—then you select, purchase and install the tools. It will not be quick or cheap—but it might just avoid a very expensive failure.
Principle 9. Capture test incidents and use them to manage risk at release time. A test incident is any discrepancy between the expected and actual results of a test. Only some test incidents will relate to actual faults; some will be caused by incorrect test scripts, misunderstandings or deliberate changes to system functionality. All incidents found must be recorded via an incident management system (IMS), which can then be used to ascertain what faults are outstanding in the system and what the risks of release might be. Outstanding incidents can be one of the completion criteria that we apply, so the ability to track and evaluate the importance of incidents is crucial to the management of testing.
Principle 10. Manage change properly to avoid undoing all the testing effort. Things change quickly and often in an e-commerce development and management of change can be a bottleneck, but there is little point in testing one version of a software application and then shipping a different version; not only is the testing effort wasted, but the risk is not reduced either. Configuration Management tools, such as PVCS and ClearCase, can help to minimise the overheads of change management, but the discipline is the most important thing.
E-commerce is both familiar and novel. Some of the technology is relatively novel, and the application of that technology to a complete business is certainly novel, but the problems