Typical Randomization Factors
Structure and Format the Automated Script
Depending on the mix of transactions that you want to include in the automated script, you can create different types of virtual users by defining user types within the automated script. Once you have designed the user types and the actions that each user type should perform, you are pretty much ready to run the script. This enables you to incorporate these user types while running different workload models, giving you the capability to add different flavors to your tests. Keeping all functions and methods together in a separate file makes the automated script shorter and easy to read and maintain.
Baselining and Benchmarking the Tests
A benchmark test is basically conducted to profile performance for different software releases. Benchmarking helps you to run tests under similar conditions in order to compare results for different software, to verify that new features and functions do not impact response times of the Web application under test, and to find system bottlenecks. Defining what each virtual user does and measuring response times for a single virtual user performing a certain transaction is what baselining is all about. Once you have data for a single user, you can then ramp up the number of users linearly to "n" users and measure the response times to look for deviations.
Determining the Capacity of the System
In our case, once we had the baseline test in place, we increased the number of virtual users to find the system limit. We encountered a good number of errors when we ramped up to about ten users. Typical errors were socket errors, access denied, socket connection refused, etc., depending on the configuration of the Web server. This may happen if a simulated version of software is used or any of the dependencies for the application are not available at the time of performing the test. This may pose a serious problem, because the response times would not be an exact replication of the production environment. Under these circumstances we should know when to stop. Once we know the threshold number of virtual users the Web server can handle, we can specify our workload scenarios for our load and stress tests.