just started MyRand. If you left MyRand running, you had an even chance of getting any given number. However, if you'd just started MyRand, 9 was once again much more likely to come up. Another definitive "fail." Back to the drawing board.
Tips for Experimenting
The MyRand experiments demonstrate the power of asking, "What if...?" Adopting an experimenter's mindset adds a new dimension to testing. When I'm experimenting, I like to:
- Gather lots of data. Notice that my sample for the MyRand tests was rather large: 10,000 numbers.
Perform the same tests under different conditions. By restarting MyRand before each "Randomize!" I learned that a small change had a huge, unexpected impact on the test results.
- Perform a series of slightly different tests and analyze the patterns of results. When I began using the mal-formed SQL query I mentioned earlier, inconsistencies in the data alerted me to a problem. What patterns or inconsistencies might alert you to problems in your system?
- Finally, question the tests. If the system were producing incorrect results, would the tests make the failure obvious?
Scientists formulate hypotheses, design experiments, and compare predictions to observations. As testers, we also follow the scientific method. We hypothesize that we'll find a bug if we exercise the software in a particular way. By taking our method a step further, using test results to suggest new tests, we increase our chances of exposing camouflaged bugs and gaining a deeper understanding of the complex systems we test.