Agile Performance Testing

2) Web forms are used to enter and save data. For example, one form can be used to submit all income-related data for a department for a month. Such a form is a web page with two drop-down lists (one for departments and one for months) on the top and a table to enter data underneath them. You choose a department and a month on the top of the form and then enter data for the specified department and month. If you leave the department and month in the script hard coded as recorded, the script would be formally correct, but the test won't make sense at all—each virtual user will try to overwrite exactly the same data for the same department and the same month. To make it meaningful, the script should be parameterized to save data in different data intersections. For example, different departments may be used by each user. To parameterize the script, we need not only department names but also department ids (which are internal representation not visible to users and should be extracted from the metadata repository). Below is a sample of correct LoadRunner pseudo-code (where values between { and } are parameters that may be generated from a file).

web_submit_data("WebFormGenerated.asp",

"Action=hfmtest.us.schp.com/HFM/data/WebFormGenerated.asp?FormName=Tax+QFPamp;caller=GlobalNavamp;iscontained=Yes",

ITEMDATA,

"Name=SubmitType", "Value=1", ENDITEM,

"Name=FormPOV", "Value=TaxQFP", ENDITEM,

"Name=FormPOV", "Value=2007", ENDITEM,

"Name=FormPOV", "Value=[Year]", ENDITEM,

"Name=FormPOV", "Value=Periodic", ENDITEM,

"Name=FormPOV", "Value={department_name}", ENDITEM,

"Name=FormPOV", "Value=lt;Entity Currencygt;", ENDITEM,

"Name=FormPOV", "Value=NET_INCOME_LEGAL", ENDITEM,

"Name=FormPOV", "Value=[ICP Top]", ENDITEM,

"Name=MODVAL_19.2007.50331648.1.{department_id}.14.407.2130706432.4.1.90.0.345", "Value=lt;1.7e+3gt;;;", ENDITEM,

"Name=MODVAL_19.2007.50331648.1.{department_id}.14.409.2130706432.4.1.90.0.345", "Value=lt;1.7e+2gt;;;", ENDITEM,

LAST);

If the department name is parameterized but department id isn't, the script won't work properly. You won't get any error, but the information won't be saved. This is an example of a situation that never can happen in real-life usage—users working through GUI choose department name from a drop-down box (so it always will be correct) and matching id would be gotten automatically. Incorrect parameterization leads to sending impossible combinations of data to the server with unpredictable results. As far as we actually save information here, the validation would be to check what is saved after the test—if you see your data there, the script is working.

Performance Tuning and Troubleshooting
Usually, when people are talking about performance testing, they do not separate it from tuning, diagnostics, or capacity planning. “Pure” performance testing is possible only in rare cases when the system and all optimal settings are well known. Some tuning activities are usually necessary at the beginning of the testing to be sure that the system is properly tuned and the results are meaningful. In most cases, if a performance problem is found, it should be diagnosed further up to the point when it is clear how to handle it. Generally speaking, “performance testing,” “tuning,” “diagnostics,” and “capacity planning” are quite different processes and excluding any of them from the test plan (if they are assumed) will make it unrealistic from the beginning.

Both performance tuning and troubleshooting are iterative processes where you make the change, run the test, analyze the results, and repeat the process based on the findings. The advantage of performance testing is that you apply the same synthetic load, so you can accurately quantify the impact of the change that was made. That makes it much simpler to find problems during performance testing rather than waiting until they happen in production where workload is changing all the time. Still, even in the test environment, tuning and performance troubleshooting are quite sophisticated diagnostic processes usually requiring close collaboration between a performance engineer running tests and developers or system administrators making changes. In most cases, it is impossible to predict how many test iterations would be necessary. Sometimes it makes sense to create a shorter and simpler test still exposing the problem under investigation. Running a complex, real-life test on each tuning or troubleshooting iteration can make the whole process very long as well as make the problem less evident due to different effects the problem may have on different workloads.