Training Versus Automating
Even though every new system rollout is, hopefully, accompanied by rigorous training and thorough documentation, it is obsolete by the next release or new hire. Most business-critical software (as opposed to desktop productivity tools like word processors and spreadsheets) exists in a constant state of change as the business adapts its technology to fluid competitive and customer demands.
Unfortunately, training classes usually can't be justified for only one or two new features or new hires at a time, and pressure on delivery schedules doesn't always allow for updating documentation and training materials. So, training becomes organic: Carla trains Darla, and by the time you get to Marla variances have crept in. It's like the telephone game. The result, of course, is the very unpredictability that ends up distorting the test process.
But what if, instead of documenting the processes and training the users, we automated the processes that the users should follow? In other words, what if we trained the software, not the person?
The technology to do exactly that exists already. Many training systems and most test-automation tools are capable of walking a user through a task and controlling her interaction within certain parameters. All you have to do is shift these from training or testing environments to the production platform.
Think about it: User predictability and efficiency would improve, errors would be reduced, the learning curve for new employees would be shorter, and—best of all—we would know exactly what to test and how to test it. In fact, the automated processes simply could be pointed at the test environment as a comprehensive acceptance test.
Sound radical? It might be, but it's not as crazy as trying to test for every possible outcome of any particular individual user's behavior in a given situation.
It's not impossible. I know of companies that have done this successfully. The trick is not implementing it; it is to get management buy-in to define and adopt the processes and require the users to follow them. But maybe by educating the organization about the interplay between user behavior and testing requirements, we can do our part to effect a new level of automation.