is a blend of intelligence about what an application does (the model) and how it does it (the automation). Good models can make your automation smart enough to respond to errors and cover code that is out of reach of dumb automation. Modeling is an exercise that at the very least will make you more prepared to test, even if you don't bother to automate it.
6. Thou Shalt Hold Thy Developers Sins Against Them
Development work is hard, very hard. Developers over the past few decades have had to solve the same problems over and over again and in doing so, often make the same mistakes over and over again. We testers must remember those mistakes and design tests that ensure that lessons are being learned.
If one developer makes a mistake coding some module, then we should assume that other developers might make the same mistake on similar modules. If a particular developer is prone to coding infinite loops, then we need to make sure we test for such errors in every module that the developer writes. This is "learning from experience" and we are here to make sure that is what our developers do: understand their patterns of mistakes so those mistakes can be eradicated.
Let me know what you think of these commandments. Next month part 3 will explain numbers 7-9. I hope you enjoy reading and thinking about them as I have over the years.
- Random Testing and 'App Compat': The Ten Commandments of Software Testing, Part 1
- Test-Physicians, Source Code, and BSODs: The Ten Commandments of Software Testing, Part 3
- Massive scale random testing is a sub discipline of model-based testing. Learn more about both at www.model-based-testing.org.
- Test Oracles are elusive, but there is help out there. My very favorite article on the subject is from long time testing guru Elaine Weyuker. The paper is called "On Testing Non-testable Programs," The Computer Journal, Vol. 25, No. 4 (1982), pp. 465- 470. Sure it's an old paper, but I consider it a classic.