Understanding Software Performance Testing, Part 4

[article]

to Excel to generate graphs. Be wary of reporting early results: If the results from early testing appear to be too good to be true, investigate before reporting to ensure they are accurate.

All the relevant data sources need to be brought together for reporting:

  • Data from the load tool
  • Data from all relevant monitors
  • Data from any probing tool
    • This is why clocks must by synchronized

Ideally, run the final set of test loads for each test type several times before reporting "final" results. If the test was set up correctly, there will be some variance in each execution. This is exactly what we would expect if the load generator is properly executing the operational profile we defined.

No two runs of a test type should ever be identical. One of the primary goals of a performance test is to model or mimic the real world. The real world is "chaos"-every day, every hour, every session is different each time it occurs. Our load test needs to create this same type of chaos. Therefore, we run the final tests three to five times and average the results. This helps ensures consistency and accuracy in the test results.

I have found that the first time you attempt to build a software performance test it can take anywhere from six to sixteen weeks. This may seem like a long time, but we are dealing with calendar time, not 24/7, full-time involvement. There are natural delays between activities that make this happen. For example, when creating an operational profile it may take several days for the groups providing the data for analysis to get the data together. These delays occur at several points in the process and are normal.

There are many details and issues this series of articles did not address. The goal was to provide an understanding of the basic problems and complexities involved in building a successful software performance test.

Read "Understanding Software Performance Testing" Part 1 of 4 .
Read "Understanding Software Performance Testing" Part 2 of 4 .
Read "Understanding Software Performance Testing" Part 3 of 4 .
Read "Understanding Software Performance Testing" Part 4 of 4 .

About the author

Dale Perry's picture Dale Perry

With more than thirty years of experience in information technology, Dale Perry has been a programmer/analyst, database administrator, project manager, development manager, tester, and test manager. Dale's project experience includes large-systems development and conversions, distributed systems, and online applications, both client/server and Web based. He has been a professional instructor for more than fifteen years and has presented at numerous industry conferences on development and testing. With Software Quality Engineering for eleven years, Dale has specialized in training and consulting on testing, inspections and reviews, and other testing and quality-related topics.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Sep 22
Sep 24
Oct 12
Nov 09