Understanding Software Performance Testing, Part 4


the security mechanism and possibly the service connection.

To apply the load outside the network requires access to proper network resources to generate sufficient load for all types of test types, but especially for stressing the connection and the security apparatus. Some tools use methods such as IP spoofing to make the load more realistic. This can be a problem with some security elements (firewalls) as they are designed to prevent spoofed IPs from accessing the system. Careful coordination with the security personnel is essential.

Load tools can also use agents placed around a network to better simulate the distribution of activity over the network. Agents need to communicate with the main tool control host (console). This communication consumes some network resources so care must be taken in the application of the load. With a segmented LAN/WAN ask yourself:

  • Do you test on only one segment or location?
    • To test multiple points you may need agents for the load generator, will these affect the test (they use network resources)?

Figure 2

Figure 2 shows the application of load to a networked environment.

Executing Tests and Reporting Results
The last part of performance testing is execution and reporting. Too often we over focus on the tools and the data they provide. Execution and reporting are the "last" issues to address, not the first.

If the planning, analysis and design of the performance testing went well, execution becomes a straightforward process. We should now have a good idea of what we are trying to accomplish and where we think potential problems exist. Prior to executing the performance test, there are a few things we need to check.

  • Is the test lab equipment set up and ready?
    • This includes having the technical staff up to speed on any new equipment,
  • Are the tools installed and calibrated?
  • Have the scripts been created and the data sources set-up and synchronized?
  • Are all the clocks on the various components synchronized, especially if multiple time zones are involved?
  • All monitors must be active and running prior to the execution of the load test and probing clients.
  • Notify people of the test start and end.
    • During the test, the system cannot be used by unknown sources.

Test execution is not a single event. There are three stages that test execution typically goes through. The first stage is for correcting errors in the test scripts, environment, etc. Several sequences of tests may be required just to get the bugs out of the scripts. Script debugging is critical to avoid reporting false information. The initial set of tests, once debugged, can also be used for a smoke test. The second sequence of tests is used to tune and fix the system, application, network, etc. And the final stage is where you run the test with the intent of reporting the end results.

Resource monitoring occurs at every stage of the tests so that the information can be used to correct any problems and also to document progress. Depending on many factors-size of application, complexity of functions, stability of the system, availability and usability of tools, etc.-the number of runs at each stage will vary.

Performance test execution typically goes through three stages, as can be seen in Figure 3.

  • In the first series of tests, the scripts are debugged and the characteristics of the tools are assessed.
  • The second series of tests are the ones that assess the system and serve as the basis for tuning and debugging the system.
  • The final series of tests are the ones we use to report results. The final tests should be run

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.