Extreme Testing

Very rapid development of high-quality applications software is quickly becoming an expected norm. How can you and your team ride that wave and deliver the goods? Extreme Programming, the rapid application development practice formulated by Kent Beck, may be the answer. This humanistic discipline combines simplicity, communication, feedback, and aggressiveness to produce high-quality software very quickly.

To develop software rapidly, you start simply, and evolve quickly from release to release. This is the standard "incremental development" approach recommended by many. But to go fast and win, you have to live through change after change after change. Feedback is vital, and the most basic and critical feedback is that of Extreme Testing-the best way to survive extreme change.

Testing isn't just for testers anymore. With Extreme Testing, you can develop software more quickly, with more confidence, and with higher quality.

We'll get to the details of Extreme Testing shortly, but first let me tell you a little story of how it works.

More than two-thirds of the way through the development of a famous Extreme payroll project, the customers asked for a very tricky change. We had been storing and displaying entitlements-the money you get-as positive, and deductions as negative. It made sense to us.

But the payroll people wanted both entitlements and deductions handled as positive-a lease car deduction of $110, for example, not ($110). This was awful because the system saves hundreds of internal values and displays them on demand. Their request affected everything.

Part of the change wasn't very challenging. There were "bins" of money, and the system already knew which ones were entitlements and which were deductions. We just changed things so that deduction bins were always subtracted from entitlements when they were combined.

The devil, as always, was in the details. There were lots of places where entitlements and deductions were added, using procedural code rather than bins. Each of these additions had to be changed to include some subtraction. Similarly, there were tests for negative that had to be changed to positive, and so on. After a couple of days of getting the base support in place, a team of two of us went ahead and started changing things.

Over two-thirds through the project, would you want to change this critical assumption? With Extreme Testing, it was almost easy!

We had, at that time, around 2,000 Extreme Tests. We had Unit Tests testing the behavior of each class, and we had Functional Tests computing hundreds of paychecks and validating thousands of values. We just started running all of the tests.

Most of them worked, but hundreds did not. Each of these failures was due to a comparison or arithmetic operation somewhere, and of course, each such operation was affecting a lot of tests. The two of us worked through the tests one at a time, while the rest of the team pretended to ignore the tests being wrong. In one week, we had all the tests back where they had been, and had stumbled across and fixed a number of unrelated problems as well.

That was when we knew that Extreme Testing was magic: we had fearlessly changed an essential assumption of the system, well into development, and in a week, we were certain that it had worked! Over the life of the system, we have found few defects, if any, that were due to this extensive change.

Extreme Testing is all about confidence. Unit Tests let developers evolve the system rapidly and with confidence, and Functional Tests give customers and developers confidence that the whole product is progressing in the right direction.