Oracles, Failures, Models, and Sins

The Ten Commandments of Software Testing (make that nine), Part 2

model is a blend of intelligence about what an application does (the model) and how it does it (the automation). Good models can make your automation smart enough to respond to errors and cover code that is out of reach of dumb automation. Modeling is an exercise that at the very least will make you more prepared to test, even if you don't bother to automate it.

6. Thou Shalt Hold Thy Developers Sins Against Them
Development work is hard, very hard. Developers over the past few decades have had to solve the same problems over and over again and in doing so, often make the same mistakes over and over again. We testers must remember those mistakes and design tests that ensure that lessons are being learned.

If one developer makes a mistake coding some module, then we should assume that other developers might make the same mistake on similar modules. If a particular developer is prone to coding infinite loops, then we need to make sure we test for such errors in every module that the developer writes. This is "learning from experience" and we are here to make sure that is what our developers do: understand their patterns of mistakes so those mistakes can be eradicated.

Let me know what you think of these commandments. Next month part 3 will explain numbers 7-9. I hope you enjoy reading and thinking about them as I have over the years.


About the author

James Whittaker's picture James Whittaker

James A. Whittaker is is a technology executive with a career that spans academia, start-ups, and industry. He was an early thought leader in model-based testing where his Ph.D. dissertation became a standard reference on the subject. While a professor at the Florida Institute of Technology, James founded the world's largest academic software testing research center and helped make testing a degree track for undergraduates. He wrote How to Break Software, How to Break Software Security (with Hugh Thompson), and How to Break Web Software (with Mike Andrews). While at Microsoft, James transformed many of his testing ideas into tools and techniques for developers and testers, and wrote the book Exploratory Software Testing. For the past three years he worked for Google as an engineering director where he co-wrote How Google Tests Software (with Jason Arbon and Jeff Carollo). He's currently a development manager at Microsoft where he is busy re-inventing the web.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!