Logical Capture/Replay

Capture/replay is a much maligned but potentially useful means of accelerating the development of automated functional tests. In capture/replay, the test developer interacts with an application under test, typically through the graphical user interface (GUI), while some capture tool simultaneously generates an automated test script that is configured to play back an equivalent sequence of interactions. This is supposed to save time that would have otherwise been spent hand-coding test scripts. The problem is that GUI-based tests tend to be fragile, and the recorded interactions tend to contain a lot of content, but little meaning about the intent of the interactions. GUI capture/replay tools know only how to interact with gesture widgets. They have no idea what sort of application they are operating on or what domain terminology the application under test uses.

Instead, wouldn't it be great if test scripts were written in terms of what they were trying to accomplish? When you make an airline reservation, you don’t tell the booking agent which buttons to click and which fields to enter text into. You just tell him where you'd like to go and when. Shouldn't test scripts have the same clarity? It would certainly make them a lot less fragile, and much easier to understand.

Perhaps a better place to capture and replay interactions is within the application itself, at the business and control logic level. This frees test developers from the constraints that GUI-centric automation places on them and opens up the possibility of creating scripts that are couched in terms of the application's business terminology.

GUI Interactions
Consider the “logon” shown in Figure 1.

What if we want to enter an invalid password and verify that the application responds with the text of "Invalid name or password"? A typical GUI capture/replay tool might generate a script that looks something like this:

set_focus_to_window("Log on");
enter_text("field1", "Mike");
enter_text("field2", "invalidpassword");
click_button("button1");
set_focus_to_window("Error!");
verify(get_field("field1"), "Invalid name or password");
click_button("button1");

There are (at least) four problems with this:

1. XIf the structure of the view were to change substantially, the script would break.
2. It is difficult to determine the intent of the generated script just by reading it, partly because the script ends up performing four discrete steps in order to initiate the single operation of logging in a user, and three steps to verify and dismiss an error dialog.
3. The control names (such as "field1" and "button1") are not descriptive, so the script is hard to read. Many GUI construction tools automatically name controls.
4. The fact that "field1" and "button1" appear in both the "Log on" window and the error dialog further obfuscates the intent of the script. Even if this were not the case, we still have references to controls in two different windows to sift through, which can be confusing to someone trying to make sense of the script. This is especially true for more complex windows.

Tests that interact at the GUI level possess a high degree of fragility and complexity. In addition, each line of a script contributes only a small part to the whole. Combined, these factors make GUI-centric scripts difficult to comprehend and increase the likelihood that they will degrade over time, either through changes in the views or through errors caused by changes to the scripts themselves.

Logic-Level Interactions
So, how do we fix this? The procedure for implementing capture/replay within the application itself, bypassing the GUI entirely, is relatively simple. The core idea is to capture interactions in a form that is concise and meaningful, at locations within the application where that meaning is known. To do this, the application developer(s) will build