Timeboxed ROI

What I'm about to write may initially seem insane, but here it goes: Although we constantly rail against it, low test-automation return on investment (ROI) is not always a bad thing.

I know what you're thinking. "This guy has lost his mind! He's drunk with power!" Just hear me out, before you make up your mind.

To explain my crazy talk, I will borrow from the concept known as timeboxing. A timebox is a set window of time given to accomplish a set of tasks. In agile development, teams are tasked with producing new software releases, timeboxed to a specific number of weeks. The concepts in this article are not specific to agile, however, so some liberties will be taken with the concept of a timebox.

Given that most projects have periods of time that are deemed more consequential, more important, or at least faster-paced than others, for the purposes of this article I will define a timebox in terms of key snapshots in time, which may or may not coincide with a project's official timeboxes.

Mature organizations with well defined processes generally focus on how they operate over the life of a project, while less mature organizations give significantly more attention to these timeboxed periods of a project. Working as an automator in a mature organization is the ideal situation.

I once did a long-term contract for a relatively mature organization, and it was bliss. I functioned as the lead automation architect in charge of a team that designed and implemented a highly structured keyword framework, and we successfully implemented that framework for years with high returns on investment. In the time since I left the contract, it has continued to grow and thrive, spreading to other divisions within the organization and being ported to new tools and technologies. It was definitely one of the highlights and greatest success stories of my career. Not every automation experience follows this story line, however, and too often we have a problem adapting to the reality with which we are presented. We attempt to force a mature test automation framework down the throat of a relatively immature, timebox-focused organization and then wonder why we are met with fierce resistance.

To understand, why test automators often have this issue, let's turn to the basic return on investment (ROI) formula:

The investment is an expenditure of money, time, or effort with the expectation of future benefits, while the gain is the result of that investment. The lower the investment and/or higher the gain, the better the ROI will be. ROI is tricky, however, because its outcome largely depends on the time period in which you calculate the investment and gains. Skilled automators are innately programmed to design and implement automated test frameworks that will produce a positive ROI over an extended period of time (EP ROI). This means they start tabulating costs and benefits from an early point in the automation implementation and continue tabulating to the present, and even some date in the near future. Conversely, test automation ROI can be viewed with a timeboxed mentality that only looks at investments and gains within a specific timebox (TB ROI). This approach may produce some quick returns , but it will more than likely greatly reduce EP ROI due to excessive maintenance during slow periods and lack of reliability during subsequent timeboxed periods, specifically as the functionality increases in size, complexity, and modifications. To an automator, it seems fairly cut-and-dry; make the early investment in your automation framework for an ultimately high EP ROI. Why would anyone not understand this? Well, depending on