the load or stress tests.
Typical Randomization Factors
6. Structure and Format the Automated Script
Depending on the mix of transactions that you want to include in the automated script, you can create different types of virtual users by defining user types within the automated script. Once you have designed the user types and the actions that each user type should perform, you are pretty much ready to run the script. This enables you to incorporate these user types while running different workload models, giving you the capability to add different flavors to your tests. Keeping all functions and methods together in a separate file makes the automated script shorter and easy to read and maintain.
7. Baselining and Benchmarking the Tests
A benchmark test is basically conducted to profile performance for different software releases. Benchmarking helps you to run tests under similar conditions in order to compare results for different software, to verify that new features and functions do not impact response times of the Web application under test, and to find system bottlenecks. Defining what each virtual user does and measuring response times for a single virtual user performing a certain transaction is what baselining is all about. Once you have data for a single user, you can then ramp up the number of users linearly to "n" users and measure the response times to look for deviations.
8. Determining the Capacity of the System
In our case, once we had the baseline test in place, we increased the number of virtual users to find the system limit. We encountered a good number of errors when we ramped up to about ten users. Typical errors were socket errors, access denied, socket connection refused, etc., depending on the configuration of the Web server. This may happen if a simulated version of software is used or any of the dependencies for the application are not available at the time of performing the test. This may pose a serious problem, because the response times would not be an exact replication of the production environment. Under these circumstances we should know when to stop. Once we know the threshold number of virtual users the Web server can handle, we can specify our workload scenarios for our load and stress tests.
9. Designing Test Variables
Test variables can be a good way of altering the workload by controlling these variables. These variables will depend on the maturity of the software that's being tested. The variables are named Vx (e.g. V1, V2, etc).
10. What to Measure?
Most performance tools measure round-trip response times. "Round trip" can be defined as the time taken between the request being sent from the client and the response received from the server. Round-trip time includes the network time that is not a constant depending on the environment in which the test is conducted, the external dependencies on databases, and network latency. Hence, round-trip response times are not an accurate indicator of the performance of the application. You should measure the server-side response times from the server logs and compare them with the round-trip times to chart how the response times vary depending on the network traffic during different times of the day. It might also be a good idea to run an all-day workload and get measurement of response times at sample intervals of time to get an average number. In our case, we also measured parameters like average number of transactions/sec, hits/sec, throughput, etc., as shown in Graphs A, B, and C.
Graph A
Graph B
Graph C
11. Reporting Performance Numbers
Performance numbers are typically reported
About the author
The opinions and positions expressed within these guest posts are those of the author alone and do not represent those of the TechWell Community Sites. Guest authors represent that they have the right to distribute this content and that such content is not violating the legal rights of others. If you would like to contribute content to a TechWell Community Site, email editors@techwell.com.
