Fast, Automatic Builds: the Agile Heartbeat

[article]
Summary:

I think that the person most affected by the introduction of agile or extreme programming techniques is not the software or quality assurance engineer, but the build manager. Agile techniques are a throwback to the age when developers were able to work on small projects in small teams. Each developer once again concentrates on small building blocks of code and integrates regularly with other developers to ensure that the overall software project is progressing. For developers, agile techniques are a natural fit because they reflect how developers like to work best: on small, manageable pieces of code with regular feedback. However, even though developers are working on small sections of code, their overall projects are now large and continually growing. And it's the large body of code that the build manager is expected to work with, not the manageable chunks. While daunting, this precipitous increase in builds can be managed by carefully implementing continuous integration and making fast, automatic builds the quot;heartbeatquot; of your agile development.

So there's no going back to small builds on a single machine for build managers. In fact, the build manager is expected to cope with ever larger software, on ever more platforms, while at the same time dealing with developers' requests for fast integration builds. Integration builds are invaluable because they enable agile development at the software engineer's desktop; however, they force build managers to produce one or two orders of magnitude more builds per day.

Continuous Integration Redefines Builds
The seminal paper on Continuous Integration (and an excellent, and readable, introduction to the topic) is Martin Fowler's article entitled, simply, Continuous Integration. [i]. It's abstract states:
quot;Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.quot;
To the build manager, the {sidebar id=1} words quot;multiple integrations per dayquot; and quot;each integration is verified by an automated buildquot; are a worrying and radical change from once-nightly builds!
Fowler goes on to list a number of quot;Practices of Continuous Integration.quot;
The first of Fowler's build-related best practices is quot; Automate the Build .quot; He argues that the set of tasks required to go from source code to the final shipping binary is so complex that it should be automated and repeatable. If the build is not automated the build process will be error prone. He also points out that the build includes much more than just compiling the software and building a binary. And he sets the goal of quot;anyone should be able to bring a virgin machine, check the sources out of the repository, issue a single command, and have a running system on their machine.quot;
It's worth stopping and thinking about the implications of that statement for your build organization. It's very likely that getting to such an automated stage seems impossible, when starting from what is often a creaky build system held together by a collection of Makefiles, Perl scripts and other programs whose exact function or operation is often unclear. Clearly, the goal of going from a virgin machine to fully running code is a stretch, but I think it's a good overall goal. Once you reach that stage, you will have put together a build system that is very reliable, and should, if written and documented well, be easy to modify and adapt as software changes.
Such a fully-automated build has advantages outside the direct realm of agile development. How many times has your build team been asked to rebuild an old version of your software and been unable to do so? Important customers can sometimes demand that old versions of code be patched or updated, or a security problem can dictate that all versions of a company's currently supported code need to be fixed and released.
In general, only very well-prepared teams are capable of building an arbitrary version of their code. Even if a good archive of the sources was made at the time of release, other components of software are likely to have decomposed: you may not have the right build scripts, or the right compiler, or the right version of some third-party component any longer. Fowler's goal, which I refer to as a quot;pickled buildquot; (the entire build is pickled in a jar ready for use whenever demanded), means that old versions of software can be rebuilt at will, which helps support developers in their move to agile development.
Fowler's second best practice

Tags: 

About the author

John Graham-Cumming's picture John Graham-Cumming

John Graham-Cumming is Co-Founder at Electric Cloud, Inc . Prior to joining Electric Cloud, John was a Venture Consultant with Accel Partners, VP of Internet Technology at Interwoven, Inc. (IWOV), VP of Engineering at Scriptics Corporation (acquired by Interwoven), and Chief Architect at Optimal Networks, Inc. John holds BA and MA degrees in Mathematics and Computation and a Doctorate in Computer Security from Oxford University. John is the creator of the highly acclaimed open source POPFile project. He also holds two patents in network analysis and has others pending.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Nov 09
Nov 09
Apr 13
May 03