Continuous Staging: Scaling Continuous Integration to Multiple Component Teams

Continuous Integration for Component-Based Development

In small team development, the practice of Continuous Integration [2] is an effective technique for keeping every one on the team coordinated with the latest results of everyone else's changes. Practicing Continuous Update[3] and Private Build[4] in one's Private Workspace[5] , as part of a Two-Phased Codeline Commit strategy helps ensure workspaces and work tasks stay "in sync" while Integration Builds ensure the team's codeline remains stable, consistent, coherent, and correct.

Any built version of a component that needs to be accessed by internal stakeholders (such as a QA/V&V group) needs to be identified (e.g., using a label/tag). This ensures that anyone who needs to look at it (even after it is no longer "latest and greatest") can easily do so, and know which version of the component and its corresponding source code they are looking.

In an ideal world, we could build the entire system directly from the sources in a one-step process, for everyone working on any component. And (again) ideally - we have the storage capacity, and network bandwidth, and processing power, and load distribution, to build the whole system (or at least incrementally build the whole system) every time before a developer commits their changes to the codeline.

Sometimes, for reasons of build-cycle-time, or network resource load, or schedule coordination (e.g., multiple time zones, or interdependent delivery schedules of components), or other reasons, this is not always feasible. And what happens when I have multiple teams and multiple components, each with their own integration schedule or "rhythm" and needing to coordinate with a larger-grained system integration strategy? There are many dependent factors to consider, including:

• Relationships between subteams, and their respective integration "rhythms",

• Build-time dependencies between components (libraries, APIs, etc.)

• Geographically dispersed teams and team-members (and differences in time zones)

• Repository size and performance

• System-build performance and available network resources, etc.

System Integration for Multiple Component Teams

Many large projects and systems require multiple teams of people to work together. In component-based development and elsewhere, it is common practice to see a system partitioned into multiple subsystems and/or components, with a team allocated to each component of the larger system (a component team [1] ):

• Each component-team develops a separately buildable part of the overall system, and typically develops and modifies only the source for their component (the component team takes ownership of the component's source code)

• Each component may be used or reused in one or more products of the overall delivered system (or within a product-family).

• Some components may require delivery and distribution of their source code in order to build other components; while other components may require delivery and distribution of only binaries, or binaries and interface definitions (e.g., header files in C/C++).

• The resulting delivered component versions can then be assembled together into the final system (subject to system and integration testing of course).

Each team should be responsible for ensuring it delivers working code and executables to other internal stakeholders. When a Two-Phased Codeline Commit protocol is used, each Task-Level Commit to the codeline is essentially signing and sealing that you have successfully compiled and tested the entire component with your changes incorporated.

If the team is large or dispersed enough to actually warrant "subteams", it may become necessary for each component-team to deliver "tested, closed, sealed and signed, versioned binary libraries" to the rest of the component teams. If the repository contains several components, and people build only their components to test their changes (rather than the entire system), then they are ensuring they have "tested, closed, sealed