Repeating the Unrepeatable Bug

When I was a new recuit at a software company, I stumbled across an application crash. I tried to repeat it by replicating the steps I had taken to get there, but I couldn't. I had saved a screen shot and the error log, so I asked some of the other testers about the bug. They told me it was an "unrepeatable bug," and that each one of them had found it at some point. While I spent a couple of hours that afternoon working on it, in the end I told the testing team that I couldn't repeat it either. The team repeated the mantra that we often use in these situations: "If we don't have a repeatable case, we can't fix it." Without a repeatable case, I was just a voice in the dark.

When it became clear that this sporadic, unrepeatable bug was negatively impacting customers, the team decided that we should spend time looking into it. The next time I found it, I paired with a programmer and watched how he attacked the problem. The first thing I noticed was that he wasn't interested in the details of what I had been doing when the bug appeared. Instead, he looked at the big picture, envisioning the entire system in his mind, along with the intricate interactions within the system. I asked him to explain what he was thinking, to show me the clues he saw in stack traces, and what he had done in the program code to help trap the error the next time it occurred.

Eventually, we were able to repeat the bug regularly at a layer behind the GUI. However, because we couldn't always repeat it at the GUI layer, it took some time to convince the team that we had tracked down the cause. The developer and I spent time building our case. We followed hunches and tried out different theories. When we felt our case was solid, we presented our findings. We got a green light to fix the bug and, later, were satisfied to learn that we had eradicated a sporadic bug, which our customers would no longer have to deal with.

So-called "unrepeatable bugs" have always bothered me, especially sporadic, high-impact bugs, such as application crashes, data corruption, and memory leaks. Often, what seems like a sporadic problem in the test lab may be a constant problem for a customer.

When I was starting out as a tester, I worked on unrepeatable bugs during lulls between release cycles. Through a combination of luck and stubbornness, I learned that I could often find repeatable cases for these bugs. Since then, I have honed my testing skills by finding repeatable cases for high-impact unrepeatable bugs. Here's what I've learned along the way.

Getting a Repeatable Case
There isn’t a set formula to repeating an unrepeatable case. It takes a combination of activities to narrow down the cause. When you find a high-impact unrepeatable bug, continue to gather evidence as you perform other testing tasks.

Whenever you see the bug, save all the information you can. Create a special folder and save stack traces, screen shots, notes, and anything else that seems relevant to the case. Revisit the folder and review the data, so that when you work with developers on these kinds of bugs, you have information to help them troubleshoot. They often will recognize patterns or clues that you missed. In your spare time, check the bug database for similar bugs. I have noticed that several seemingly unrelated unrepeatable bugs have identical stack traces in the bug reports. Investigation revealed