Why Not Exploratory Testing?

Member Submitted

to every possible input, fine. Give us a list of your test cases. We can write a program that will pass all your tests but still fail spectacularly on an input you missed. If we can do this deliberately, our contention is that we or other programmers can do it accidentally"–Cem Kaner

A well documented Test Scripts can also miss bugs. Unplanned Adhoc Testing can also miss bugs. For the former, Time + Money is invested for finding the bugs. As every process is having its own entropy associated with that, this is not an odd process. This have its own entropy, leaving some bugs, covered. The same applies for the latter too. In an organization with no Software Configuration Management, experience proves that the number of bugs covered by well planned Test Scripts is less than the Unplanned Adhoc Testing. But when tried to put a Cost Benefit analysis, the Test Manager is pushed to follow an Unplanned Adhoc Testing approach. Instead of going to an Unplanned Adhoc Testing, why not Exploratory Testing?

"Myers described a much simpler program in 1979. It was just a loop and a few IF statements. In most languages, you could write it in 20 lines of code. This program has 100 trillion paths; a fast tester could test them all in a billion years."–Cem Kaner

Testing cannot be claimed that it is completed. Testers cannot claim that the program is 100% error free. 20 lines of code is having 100 trillion paths. We normally deal with Klocs, that gives a very big figure, when extrapolated. If we wanna uncover most of the bugs, we have to select Test Cases from the Test Cases for trillion paths, which is trillion*n in number. Planning these type of tests requires more time, which we cannot think if we are in Web Time. If no time is available, then the tests become Uplanned and Adhoc. Started with a good plan, morphed to Unplanned and Adhoc at the end. Instead of welcoming this, Why not Exploratory Testing?

"If you want and expect a program to work, you will be more likely to see a working program – you will miss failures. If you expect it to fail, you'll more likely to see the problems. If you are punished for reporting failures, you will miss failures. You won't only fail to report them–you will not notice them."–Cem Kaner

Testing relies on the mindset of the testers. It is an art. Most of them says that tests have to be well planned and executed. Let us take a case that we are well planning 100 tests for a program which is going to leave, say 20 bugs covered. Testing is a creative work . It the testers have to go by these well planned 100 tests, they will not be exploratory while executing them. Instead they are required to be more exploratory, while they are executing the tests. They have to be exploratory to find the remaining 20 bugs. If that 20 bugs are going to be very major ones and these 100 tests are going to give only very little number of minor bugs, then what is the advantage in investing a huge amount of money in designing those 100 tests. Instead of this strategy, why not exploratory testing?

Testing is creative work . Agreed, the exploratory nature of the tester is needed very much while designing the tests. At the same time, it is needed very much more than while designing the tests. We cannot deny the fact that exploration is not needed while executing the tests and

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.