organizations have gone wrong in their test automation efforts over the years has been in ignoring this whole middle layer of service testing. Although automated unit testing is wonderful, it can cover only so much of an application’s testing needs. Without service-level testing to fill the gap between unit and user interface testing, all other testing ends up being performed through the user interface, resulting in tests that are expensive to run, expensive to write, and brittle.
The Role of Manual Testing
It is impossible to fully automate all tests for all environments. Further, some tests are prohibitively expensive to automate. Many tests that we cannot or choose not to automate involve hardware or integration to external systems. A photocopier company I consulted to had a number of tests that needed human intervention before they ran. For example, making sure there were exactly five pieces of paper in the paper tray was easier to do manually than to automate. In general, manual testing should be viewed primarily as a way of doing exploratory testing. This type of testing involves a rapid cycle through the steps of test planning, test design, and test execution. Exploratory testing should feature short, feedback-generating cycles through these steps in a manner analogous to test-driven development’s short cycle of test-code-refactor. Beyond finding bugs, exploratory testing can also identify missing test cases. These can then be added at the appropriate level of the test automation pyramid. Further, exploratory testing can uncover ideas that are missing from the user story as initially understood. It can also help a team discover things that seemed like a good idea at the time but seem like bad ideas now that the feature has been developed. These situations usually result in new items being added to the product backlog.
What Building In Quality Looks Like
A team that has integrated testing into its day-to-day work will look and behave very differently from a team that attempts to test quality at the end. Some of the observable traits of a team that builds quality in include the following:
The use of good engineering practices. A team focused on building in quality will do whatever it can to write the highest quality code possible. This will include pair programming or thorough code inspections for at least the most complex parts of the system. There will be a strong focus on automated unit testing, if not test-driven development. Refactoring will happen continuously and as needed rather than in large, noticeable spurts. Code will be continuously integrated, and failures in the build will be treated with almost the same urgency as a customer-reported critical bug. You’ll also notice that code will be owned collectively by the team rather than by individuals so that anyone noticing an opportunity to improve quality can take it.
The hand-offs between programmers and testers (if they exist at all) will be so small as not to be noticeable. Doing a little of everything (designing, coding, testing, and so on) all the time helps teams work together. When working that way, a programmer and tester talk about which capability (or partial capability) will be added to the product next. Then the tester creates automated tests and the programmer programs. When both are done the results are integrated. Although it may be correct to still think of there being hand-offs between the programmer and tester, in this case, the cycle should be so short that the hand-offs are of insignificant size.
There should be as much test activity on the first day of a sprint as on the last day. A