possible. Scripting is effectively programming and means the tester spends most of their time writing code. Test scripts can contain defects of course, so the scripts themselves need to be tested, corrected and retested before the performance test can be executed.
By choosing a tool that requires as little scripting as possible – with some tools requiring zero scripting - the complexity and effort associated with setting up a performance test are significantly reduced .
Easy to learn. Removing the need to program test scripts greatly simplifies the task of establishing a performance test; however, some of the new tools do this better than others. These tools also try to remove the need for formal training. By choosing a tool that is easy to learn, more of your Agile project team can be involved in performance testing and expert resource scheduling challenges are no longer an issue.
Quick to use. A requirement for testing within an Agile project iteration is that the time to update the test assets (from those used in the previous iteration), repeat the test and analyze the results needs to be as short as possible. The new breed of tools often allow this to be achieved in a matter of hours whereas traditional tools would require many days or weeks to do the same – and that’s presuming a tool expert was available!
Encourage correct testing. Why do systems that have apparently been properly performance tested still fail when user load peaks? The answer is simple. The performance tests executed did not simulate the way the application would be used by the real users. It is therefore imperative that any tool chosen allows the tester to correctly and fully simulate everything the real world user will do. Only by correctly testing an application can the test results be trusted and the decisions to release the software made with confidence.
Easy maintenance. In the example project used above, a performance test would be run at iteration end, every 3 weeks. Agile processes actively encourage major changes in requirements and therefore any selected tool should allow test assets to be updated easily and quickly.
Immediate diagnosis of problems. When testing identifies poor performance or scalability bottlenecks, it is imperative that you can quickly identify the real underlying causes (as opposed to the symptoms) of the problem. Therefore, a selected tool must also monitor ‘deeply’ into your system and provide a mechanism for quickly correlating performance problems and the monitor data to identify the true underlying cause(s).
Performance testing within the Agile process
So, how do you now implement performance testing within the Agile process with a performance testing tool that can correctly simulate how real users will interact with the software, is easy enough for many Agile team members to use, and allows rapid configuration, execution and analysis of performance tests?
If we again consider the example above – a 10 iteration Agile project with each iteration lasting 3 weeks - the goal is to treat performance testing with the same importance as unit and acceptance testing and therefore to execute a correct performance test at the end of every iteration.
The initial test would be at the end of the first iteration when only a small amount of software will have been developed; of course this may be the implementation of the applications architecture which is very important software as it will affect every business transaction subsequently developed. It doesn’t actually matter how much code has been developed – it should be performance tested.
Using your new breed of performance testing tool, within approximately a day, the