Victims of Our Own Success
This method worked too well. The testers started splitting up their test suites and were creating new build jobs left and right, mainly because it was so simple. They didn’t have to deal with hostname issues and whatnot. They simply had to create the new job from an existing Hudson job and change whatever they needed in order to run a different set of tests. When the job ran, it did not care what else was running because it was running in its own separate environment.
The build environment consisted of the Dell PE 2850 as the Hudson Master and Build 1, Build 2, and my old Linux workstation as build slaves. That workstation was working so hard that its big fans would turn up to full speed and make a ruckus by my desk. I eventually moved it to the server room and got a new Linux workstation. The problem was that there weren’t enough build slaves to handle the load and builds were getting queued up. When I sent a screen shot of Hudson’s home page to my boss, four builds were running and seven jobs were queued for a place to run. Our feedback loop was long.
If you checked in some code to fix a WebTest test before lunch, you would be lucky to find out whether it had fixed the problem by the time you headed home. If you checked in after lunch, you’d wait until the next day for results. Something had to happen.
In the second part of this article, Virtual Hudson Build System: The Rest of The Story, I will discuss the solution we implemented to solve our long feedback loop problem.