One of the key purposes and benefits of Agile development processes (“Agile”) is that they emphasize working software as the primary measure of progress. To ensure that the current state of the software is working, every iteration includes unit and acceptance testing.
Many organizations report that, following the adoption of Agile, the quality of their released systems has been improved.
When talking about ‘quality’ in this respect, the focus is solely on functionality; non-functional quality, and particularly the ability of the application to perform and scale at times of peak usage, is excluded from testing that occurs within iterations.
The main reason for excluding this type of testing is that performance tests have traditionally required weeks to set-up and hence the popular belief is that there is no time for performance testing within an Agile project’s iterations.
Why is this a problem?
Let us presume that an Agile project’s iterations last 3 weeks and needs 10 iterations before enough software has been developed and signed off to form a release to the business/client.
Following the commonly believed approach that the only time to performance test applications delivered by Agile projects is just before the software is released, that would mean that the performance testing would occur 30 weeks after the first line of code was written. In addition, using traditional performance test tools, the performance test project itself could easily take 4 weeks to deliver.
So, 34 weeks after the first line of code was written, and therefore potentially the first ‘performance defect’ implemented into the code base, the problems caused by the defect would be detected. Of course it is highly unlikely that the first day of coding would implement problems, but it is definitely feasible that defects could be implemented in the first 3 or 4 iterations; many weeks, and potentially months, before the first round of performance testing delivers the ‘bad news'.
The obvious risk with the above, and an all too common approach, is that performance defects discovered towards the end of the project, long after they were implemented, can significantly delay the actual project delivery. Identifying and fixing the problems will often require significant amounts of unscheduled effort.
If the software’s purpose was to reduce the business’ costs or drive more revenue, problems discovered at such a late stage will delay the business receiving the benefits the software was meant to deliver.
Some of you may be thinking that it’s not your fault, as performance testing is not possible during the main part of an Agile project. This statement is correct when referring to traditional performance testing tools that require extensive tool experience, and a large amount of time and effort to write and maintain test scripts.
So, the widely accepted conclusion is that Agile projects have to live with the fact that they run the risk of finding major performance or scalability bottlenecks late in the project and, if performance issues are identified, project delivery timescales and budgets will slip, possibly significantly.
A solution exists
Within the last year or so a new breed of performance testing tool has appeared and has quickly gained significant numbers of adopters. The new tools are innovative, easy to use and reduce performance testing timescales from weeks to days.
Adoption of one of these tools into the Agile development process allows performance testing to be treated with the same importance as unit and acceptance testing, and integrated into every iteration of the project. By doing this, projects ensure that at the end of the development, any performance problems detected will be due to infrastructure software configuration or hardware limitations as they will know that their application software contains no performance defects. Therefore, any detected problems will be much quicker and more straightforward to correct and any knock-on delay to the software’s release will be minor.
The new tools all share similar attributes and it is important that any chosen tool provides the following features:
As little scripting as