test environment (see below) can be configured, the tool installed, the initial ‘test scripts’ developed (of course you want to be scripting as little as possible) and a performance test executed.
Then, as part of every subsequent iteration, it will be literally a matter of hours to update the test assets and repeat the tests.
But what about the test environment?
It is often stated that performance tests must be executed on environments as similar to the production system as possible. This is correct for system performance testing (where the system includes the hardware and layered software as one item), but is not the case for application performance testing.
To quickly jump into some performance testing theory, there are four main reasons a system will fail to perform under peak load: the hardware is insufficient, the infrastructure software (database management systems, web servers, etc.) is not up to the job, the infrastructure software is incorrectly configured, or performance defects exist within the application code.
These causes are placed in the order they are least likely to occur i.e. very rarely nowadays are performance problems due to insufficient hardware resources and, conversely, the most common cause for poor performance is application code. In addition, in our experience of testing literally thousands of application releases for hundreds of organizations, the vast majority of application performance defects are identified with a simulated load of between 50 and 100 concurrent users.
Using this knowledge, it is obvious you don’t need hardware as powerful as your production servers. It does however makes sense to have the same infrastructure software in your test environment as will be installed in production.
Therefore, the test environment during an Agile project can consist simply of allocating two developer machines for the period of the test – one will host the performance testing tool and the other the latest build of the application. Alternatively, a couple of average specification desktop machines can be dedicated for this purpose for the duration of the project. The key point is that you do not need anywhere near the specification of the machines in your production environment.
Agile processes promote regular testing and the goal of every iteration is to produce working software. Up until recently it has been believed that it was impossible to adopt performance testing within the project’s iterations, and hence when it was stated that software was ‘working’, a ‘subject to performance testing’ caveat had to also be used.
The potentially high cost drawback of this approach was that performance testing was deferred until near the end of the project, and any problems detected during testing at this point often caused significant delays to the software being released and required extra effort and cost to fix. Of course, any benefits the business would receive from the software being released on time would be delayed as well.
Today, however, there is a new breed of performance testing tools available; the tools are lower cost, easy to learn and significantly quicker to use. These benefits mean the tools can be used within Agile iterations without delaying the projects, there is no need to hire or ‘borrow’ tool experts, and the danger of performance defects causing significant problems later in the project is eradicated.
Case Study: StressTester™ in action
One of the new breed of performance testing tools available is StressTester™ from Reflective Solutions. The tool was designed from first principles to be easy and quick to use but still allow organizations to correctly test their applications and understand the causes for any problems identified. Independent performance testing consultancies and adopting