build tool you pick, make sure your build infrastructure also runs your unit tests. The results of running the unit tests will provide a wealth of quality information at the code level.
Once you can make a build in one step, then you can move onto continuous integration. According to Martin Fowler, continuous integration [ http://www.martinfowler.com] is the practice of frequent integrations of new work, where each integration is automatically verified with a build that runs unit tests. There are many continuous integration tools available.
The most popular open source tool specifically built for continuous integration is CruiseControl [ http://cruisecontrol.sourceforge.net/]. But continuous integration isn't all you will want to accomplish, so you may want to widen your tool search as needed. Typically you will want to create a build for each integration, but you will eventually want to create a nightly, deployable build. As you introduce functional automation testing, you may have several variants of builds that execute different levels of automation. The open source version of CruiseControl wasn't designed for this.
I prefer Hudson [ http://hudson-ci.org/] for multi-project build orchestration. With Hudson, you will want to configure a new job and schedule it to look for changes to your source code repository and build for each atomic check-in. Then create another job and schedule it to run nightly to produce your nightly build. Running a build for each atomic check-in is important because it allows the team to understand the specific check-in that negatively impacted the build or degraded quality. This is the most effective feedback loop you can create. When you can tie quality metrics with each check-in, you will have constant feedback about the state of your code. This has the side effect that when developers go to the source control system to check-out the latest code, they will have explicit knowledge of the build and test quality of the code at each revision point; no more getting code and spending minutes or hours finding out if it will build.
Once you have regular builds that are emitting quality information, you can use tools like Sonar [ http://sonar.codehaus.org/] to graphically understand how your code-level quality is trending. Sonar can display information like the number of passing unit tests for each build, or build time, or percent of passing continuous integration builds. There are other tools like static analyzers and lint tools that will generate objective data about code-level quality. Some good Java analyzers are Findbugs [ http://findbugs.sourceforge.net/], and PMD [ http://pmd.sourceforge.net/]
Automated Functional Tests
Once you have code level automation, you can move on to automated functional testing. There are several levels of infrastructure you will need to build before you focus on the actual automation tests. First, you will need a dedicated machine for testing that will receive the automated deployments or installs. Next, you will need an automated way to take the build artifacts from your scripted build environment and package them up for deployment or installation. This process tends to be very specific to how you deploy or install your code. Like with the build process, the goal is to have one thing to execute that will deploy or install your build artifacts onto a separate testing machine. Tools like Cargo [ http://cargo.codehaus.org/], and Capistrano [ http://capistranorb.com] are useful for automating the package and deploy process.
A critical part of packaging is having an architecture where environment-specific configuration details aren't part of the build artifacts. This is a critical control point for quality; you want the same object code that was thoroughly tested in a test