Lessons Learned in Performance, Stress, and Volume Testing

[article]
Member Submitted

affecting multiple users or bringing down the LAN/WAN connections down completely. Before starting a stress/volume test find out how many KBs of data are currently being transmitted for a typical workday for all applications and if possible try to isolate how many KBs of data the application under test sends on a given day for an end-user. Also the test engineer should work closely with the infrastructure team to obtain a network topology diagram to assess which network segments and routers will be affected as a result of the stress/volume test. The network topology diagram and the KBs of data sent per day should help the test engineer plan and mitigate risks associated with the stress/volume test.

No forum for reporting and interpreting of testing results
In many projects after a stress test is completed and problems are encountered there is much finger pointing as to what group is responsible for the problem. Some groups will say the database is not tuned properly, or the LAN needs to be upgraded or the hardware needs to be upgraded etc. To avoid finger pointing the test engineer should schedule a meeting after the conclusion of the stress test where all monitoring support personnel bring their graphs and reports that they gathered from the stress test and all the teams interpret their results. Any graphs with abnormal spikes or deviations should be thoroughly discussed to help pinpoint potential bottlenecks or degradation points. The supporting personnel should also discuss and examine that their graphs and charts behave as expected and report this information to the other monitoring groups if such is the case to help eliminate the potential causes of problems.

No knowledge of Hardware limitations
Some projects that I have worked with spent hundred of thousands of dollars in automated testing tools to conduct performance, stress, and volumes tests only to find out 2 weeks before the execution of the test that they did not have enough hardware to emulate more than a few hundred end users when in fact their production environment would have over a thousand end users. Before a stress/performance test begins it is unnecessary to understand that each emulated end users requires a few MB of RAM and that only so many end users can be emulated on a desktop. Investigate as soon as possible how many end users will need to be emulated with the automated testing tools and how much hardware equipment is available in the project before spending untold thousands on licenses to emulate thousands of end users when in fact this will not be possible due to existing hardware limitations. If the project has sufficient budget to procure new desktops and servers to support a stress test then these pieces of hardware should be procured as soon as possible to prevent any delays to the execution of the stress test.

Duplication of testing efforts
It’s possible for a project to be so large that testing efforts are duplicated. I saw first hand in a large ERP project where supply chain and distribution processes were thoroughly stress tested with an automated testing tool. Yet one of the process team lead did not have first hand knowledge of the work that the QA team had performed to test the ERP systems response times and this process team lead had his entire team re-test the response times of shipping orders and thus duplicating many aspects of the previously executed automated stress test. This effort was unnecessary and wasted a lot of man-hours.

No access to SMEs (Subject Matter Experts)
The test engineer may have extensive knowledge of the

About the author

Jose Fajardo's picture Jose Fajardo

Jose Fajardo (PMP, M.S., and SAP certified) has worked as a test manager for various companies utilizing automated testing tools. He has written and published numerous articles on testing SAP and authored the book titled Testing SAP R/3: A Manager's Step by Step Guide. Throughout his career Jose has helped to create testing standards and test plans, mentor junior programmers, audit testing results, implement automated testing strategies, and managed test teams. Jose can be contacted at josefajardo@hotmail.com.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!