Migration to Hudson
One thing I could never get to work in CruiseControl was the “distributed” build. This seemed to be critical if we wanted to get the build time down. When our build time got out of hand again, I started looking at other build systems, including Hudson. I had played with Hudson before and liked it, but I hadn’t had the time to convert our build to it.
Every six months or so, we get what we call a “refactoring” or engineering sprint, where we upgrade tools and libraries and refactor really bad code. Leading up to this sprint, I installed Hudson on my Linux workstation and started to play with it. We also had a new coding project with its own source tree to deploy that needed its own build but had no home. I sent the team some eye candy screenshots of the new project in Hudson and it didn’t take long before they were sold. This would be an official project for the engineering sprint.
During the engineering sprint, we successfully converted Build 1 and Build 2 into Hudson build slaves and left my Linux box as the Hudson master until we purchased a Dell PE 2850—the first real server in our build farm.
Hudson Is Too Good
Now that we had Hudson and everyone liked it, requests started coming in. First, Lisa wanted the Fitnesse tests fully integrated. We had a somewhat cheesy way of kicking off the Fitnesse tests, but not a good way to get the results. She polled her community and found a plug-in that someone wrote to present Fitnesse tests into Hudson. At the same time, we upgraded Fitnesse, which changed the format of the test results. We used someone else’s XSL style sheet to convert the results back to the previous format. But, now that Fitnesse tests could be integrated into the build, the system started to get overloaded yet again.
The Obvious Way to Configure a Hudson Build System
Our tests did what you might expect and had to use external servers outside Hudson’s control. For example, the Canoo WebTest tests had to run against a web application server with the latest code deployed. Though we used a single web application server and a single suite of WebTest scripts, this would take hours to run. The Fitnesse test was a similar process, with a single external Fitnesse server outside of Hudson’s control. Parallelizing the test became necessary in order to keep the test-results feedback loop short. However, this presented a problem. The tests were never meant to run in parallel. Refactoring the tests to work in this fashion didn’t seem like a good idea. They worked as they were, and rewriting would invalidate them—not to mention that it would simply be a lot of work. Another problem was that we would need more backend external servers to run the web application and to run Fitnesse.
The solution that worked best—I like to call it the “Obvious Way to Configure a Hudson Build System, but We Don’t Do It That Way for Some Strange Reason”—was to create a generic slave that had all of the software needed to run any test without hitting an external server. This meant that instead of the build slave running a WebTest test against an external server, it would start up its own web application server and run the tests against localhost. This worked well. Creating a new build slave was simple—the key was to have all the tools installed the same way, with the same usernames and passwords.