SCM Design Patterns: Version Control & Multiple States

[article]
Part 2
Summary:
The Version Control design pattern is seen when you only want to capture iterations (versions) of static items such as documents, simple applications, and project baselines. Other features of simple version control are that you want to capture and control items while guarding against the loss of any item, maintain an audit trail of each item, and allow for roll back of items to an earlier version. The next design pattern is Multiple States, where you need to track unique collection of items that represent the application at a certain point in the lifecycle of the development process

The Version Control design pattern is seen when you only want to capture iterations (version control) of static items such as documents, simple applications, and project baselines. Other features of simple version control are that you want to capture and control items while guarding against the loss of any item, maintain an audit trail of each item, and allow for roll back of items to an earlier version.

jjdesignpatterns2-1

The roles I described in Design Pattern 1 are the same in this Design Pattern, except now they execute their functions much more efficiently with the implementation of an automated SCM system. The illustration shown in Figure 6 shows this topology, very similar to that shown in Figure 1.

Most of the processes defined in Design Pattern 1 can be used as a base for Version Control. Sometimes this Design Pattern is called a Single State Lifecycle since there is no promotion (more on this terminology in Design Pattern 3). Since most of the processes necessary for this Design Pattern have been defined using manual methods
described in Design Pattern 1, all that has to be done is to automate them with
a SCM System (Figure 7).
jjdesignpatterns2-2

Here are some of the processes that could be automated:

    • Check-out
    • Check-in
    • Generation of on-line forms for managers to enter change requests into the system
    • Secure access to CIs
    • Generation of CI version numbers
    • Email notification to approvers that indicate their need to sign-off on CIs
    • On-line approval of Changes
    • Report generation for auditing and compliance purposes

Depending on the SCM tool and your implementation of it, there may be fewer or more functions that you may want to automate based on internal and external development and support requirements. If you went through a "dry run" with a paper based Design
Pattern, you now have requirements to evaluate a SCM tool with.

This design pattern is fine as long as you have static CIs that do not require any "compartmentalization". By that I mean you do not need to have multiple configurations or versions of your CIs at any one point in time. The requirement for multiple views of your
CIs is essential when you have a more complex Application that needs various groups to access the CIs that form an application at a single point in time. This is what Design Pattern 3: Multiple States will address. 

Utilizing the first two patterns that we went through, we can now progress to the following more complex Design Patterns.

Design Pattern 3: Multiple States
Now that an automated/computerized SCM system is in place, the topology for this Design Pattern and Design Patterns 4 & 5 looks something like Figure 8. 
jjdesignpatterns2-3

Figure 8 shows the SCM system controlling access to all the CIs that make up an Application. It also shows access to the CIs by the previously defined Developers, Technical Team Manager, and Development Project Manager. But now we have two new roles: Testers and Customers.

Testers*: An independent group that specializes in executing testing. Activities they perform include:

    • Generation of test cases
    • Validation of test cases to user requirements
    • Stress testing

Customers*: Existing users of the Application in the field or internal users that test the Application for "usability" from a customers' perspective.

*Note: The roles I have created for this Design Pattern and those that follow are for illustrative purposes only. Your development environment and SCM system/tool may call for other/different roles.

Once you start down the road of more sophisticated software development and SCM, you are going to want to segregate the development and SCM activities into secure areas which support the lifecycle of your Application.

An important concept of this Design Pattern is that of a "State". A State is a unique collection of CIs that represent the Application at a certain point in the lifecycle of the development process. Every State is separated within a SCM tool from all other States so that different collections of CIs can be configured to present different views of an Application without impacting the configuration of CIs in
other States. Figure 9 shows a simplified flow chart of how this "Multiple State" design pattern looks. 

jjdesignpatterns2-4


All CIs (code, make files, graphics, etc..) that are needed to build an Application must enter the SCM system through a Check-in to the Development State. Any CIs that need to be altered must also leave the SCM system from the Development State through a Check-out. This may seem obvious but it is key to controlling the integrity of the States that follow. The States are named according to what function they serve in the development lifecycle. CIs are moved between adjoining States through the concept of Promotion (moving from a lower State to a higher State) or Demotion (moving from a higher State to a lower State). The motion enabler for these movements are primarily centered around stakeholders'
testing activities and CCBs that will be highlighted in Figure 10. All States beyond the Development State are "View Only". This means that no changes can be made to the CIs directly, they can only be moved around from State to State.

Now that the basic concepts are in place we'll move to a more complete perspective of the Multiple State Design Pattern shown in Figure 10.

FIGURE 10

jjdesignpatterns2-5


In Figure 10, starting in the upper left hand corner, you see the same sorts of inputs entering the Change Control step that you saw in the Version Control Design Pattern. Change Control functions the same as it did in Design Patterns 1 and 2, but from this point on CIs will be handled differently. As we discussed before, we want to insulate the States from one another by the needs of the stakeholders' activities, primarily testing and CCBs.

An important concept are the rules for moving CIs from State to State. 

  • To promote CIs from one State to another there must be a CCB approval (except between the Development and Integration States)
     
  • Approval of CIs from a CCB does not necessarily mean they are automatically promoted. A Technical Team Manager/Development Project Manager controls the promotion of CIs from State to State
     
  • A demotion from one State to another (also called a rollback) should only
    occur when an error is detected by the stakeholder of the State after one
    or more CIs have been promoted to that State. The Technical Team
    Manager/Development Project Manager controls the demotion from State to State

The source code and other CIs (indicated by the grey and green boxes) and the run-time CIs (indicated by the yellow triangles for incremental builds and yellow octagons for complete builds) indicate a relationship between these 2 components. The run-time Application in each State is unique to the CIs that are used to compile the Application in that State. The run-time Application is most often the target of testing and CCBs. But the
source code/files as well as other non-compiled CIs are the primary components
manipulated by the SCM system. 

Our first State is the Development State.

The Development State is used by Developers on a team to Check-out and Check-in CIs as changes impact the need for new or modified CIs. This might mean that you are checking-in some CIs for a new feature in your Application, you need to check-out a CI to fix a bug, or you are Checking-in a large amount of new CIs into the SCM system for a new
release of your Application. Developers probably do their work on the CIs within an Integrated Development Environment (IDE) on their local PCs as is indicated in Figure 10. The work done on the developers' PCs could be considered as a type of State but is usually not. Compiles are executed as needed in the IDE. Some SCM tools actually interact with the file systems on the developers' PC so that CIs remain in synch with the Development State. Developers quite often use an IDE to create a virtual development environment to produce and unit test their code. Many SCM tools today also have integration features with popular IDEs that enable easy Check-in and Check-out of CIs from and to the SCM tool. Once the completed changes or new CIs are officially
Checked-in to the Development State, they need to be compiled and unit tested with other CIs from other developers' changes. The yellow triangles represent this incremental compiled code. The compiles at this level are usually done on an ad-hoc basis driven by the needs of the development team.    

The Integration State is similar to the Development State with two important differences:

  1. It provides developers access to a larger set of system resources. This could include Web servers, Application servers, Databases, and a wider pool of developers' code that have been promoted to the Integration State.
     
  2. It does not allow Check-out or Check-in. This is because we do not want any new CIs or CI changes to creep into the SCM system without going through a Change Control first. This concept is key to the integrity and stability of the following States and the CIs stored within them. 

The Integration State facilitates the next stage of testing, Integration testing. This is a view-only State where Developers, Managers, and CCB members can only see the source CIs and the complete compiled Application (indicated by the yellow octagon), but not alter
its composition or interact with the source code. The need for developers to move source code and other CIs freely between the Development State and the Integration State
is indicated by the lack of a CCB and approval step. CCB approval cycles prove to be extremely cumbersome in today's iterative, rapid development styles. To move CIs out of the Integration State to the Quality Assurance (QA) State, CCB members must review both the changed/new source code and other related CIs as well as the compiled versions of the changes that impact the Application at this point in the lifecycle. This is a unique
perspective as the CCB members are looking at both these CI types (source code,
compiled code, and any other CIs relevant to the change in the Application). After this review, these CIs are formally approved or rejected by the CCB and if approved, promoted to the QA State. If problems are found, the CIs are demoted to the Development  State
where a Manager evaluates the error(s) and decides what corrective actions to take. This could include demoting the CI to the Development State to perform a checkout to modify the code(s) causing the error and then promoting it back to the Integration State, creating new versions of the CIs, promoting and replacing the CIs that were causing the problem, or possibly other CIs need to be changed that interact with the problem CIs.

The Quality Assurance (QA) State is where the QA group is going to be involved with a much different set of activities. This is also a "view-only State" where the QA group is focused on the compiled Application (indicated by the yellow octagon). The source code is present only to build the run-time Application. The QA group is a independent team that takes a fresh look at specific areas where changes have been made, as well as interactions with the complete Application. The QA team is usually responsible for creating test cases that will validate the quality of the Application. They may also
perform requirements validation. This is the process of making sure that all requirements
defined for the Application are tested. Finally, a CCB is performed to officially approve the changes. As with the Integration State if problems are found, the CIs are demoted to the Integration State where a Manager evaluates the error(s) and decides what corrective actions to take. This could include demoting the CIs back to the Development State for checkout to modify the code(s) causing the error and then promoting it back to the QA State, creating new versions of the CIs, promoting  and replacing the CIs that were causing the problem, or possibly other CIs need to be changed that interact with the problem CIs.

The User Acceptance Test (UAT) State involves "Customers" validating the run-time Application as a whole and ensuring that requirements from a user perspective are met.

Depending on your definition of Customers, this group might be external Beta Testers that are select Customers of the current release of your Application. Customers might also be internal personnel that represent the views of Customers. UAT is the State in which all the compiled versions of all the CIs that will make up the release of an Application are presented to this select group of Customers as well as the other members of the CCB to review and sign off on. All CIs in this State must be approved before the Application is
promoted to its final State, Production. As with the QA and Integration States if problems are found in any of these activities, the CIs are demoted to the QA State where a Manager evaluates the error(s) and decides what corrective actions to take. This could include demoting the CIs back to the Development State for checkout to modify the code(s) causing the error and promoting it back to the UAT State, creating new versions of the CIs, promoting and replacing the CIs that were causing the problem, or possibly other CIs need to be changed that interact with the problem CIs.

The Production State is the final State where the production Application resides. All CIs that make up the Application are promoted to this State after the Customer CCB approves the Application, or changes therein, and all the CIs that comprise the Application are promoted to the Production State where the complete Application is compiled (indicated by the yellow octagon) and ready for use. If problems are found in any of these activities, the CIs are demoted to the UAT State where a Manager evaluates the error(s) and decides what corrective actions to take. This could include demoting the CIs back to the Development State for checkout to modify the code(s) causing the error and promoting it back to the Production State, creating new versions of the CIs, promoting and replacing the CIs that were causing the problem, or possibly other CIs need to be changed that
interact with the problem CIs. 

If you look closely at Figure 10, on the left side of the States, you will notice a Managers circle with lines pointing to the Integration through Production States. This indicates an important feature of this Design Pattern. The States not only function as areas for testing and approval activities, but they control the makeup of the Application at that point in the lifecycle. Managers are not only involved with the approval and rejection of CIs but they also control the makeup of the States by controlling the CIs that are in them. Several of today's SCM tools employ a concept called "Packages"  that control the grouping of CIs as they move through the States. This concept makes the job of the Managers who control the makeup of the States much easier by knowing what the CIs are for (bug fix, new feature, enhancement, etc.). Just because a group of CIs are approved/rejected does not necessarily mean that they are automatically promoted/demoted. 

Quite often there will be different levels of CCBs. There may be more informal CCBs in the lower States and  more structured CCBs when CIs get close to Production. As stated before, CCB approval must occur prior to promotion. Promotion must also be a secure process only accessible by Managers or other designated personnel. 

During the process of moving CIs through the States you want to be sure that the security of those CIs increases as you move towards the Production State (see Figure 11). 

FIGURE 11
jjdesignpatterns2-6

Security is a resultant of risk. Whenever you make changes to an Application you are inherently introducing a certain amount of risk. Risk that changes will break your stable Production State as well as create problems in other States. In the earlier States you need developers who have access to the SCM system to be able to move CIs freely between Development and Integration. Risk here is low, but still present. After you leave the development area you need to have escalating levels of Security. This could take the form of having more Senior levels of management be part of your CCBs to approve promotion/demotion of CIs and implementation of standard Builds. However you want to manage it, Security is an important factor that you must address.

This Design Pattern is very good at managing complex applications that reside on the same computer as the SCM system through a multiple State concept. The notion of controlling lifecycle activities (promotion, demotion, testing, incremental/complete Builds) is supported by this Design Pattern very well. The ability to generate reports on all these
activities is also important. SCM tools need to support these functions with a well designed user interface and a database that can track all the lifecycle activities. Close interaction between the SCM tool file system and the file systems of the developers is also key. However one critical function is lacking....Deployment. This is what our next Design Pattern will address.

Read Part 1 at SCM Design Patterns: Paper Forms

Read Part 3 at SCM Design Patterns: Build and Deployment

Read Part 4 at SCM Design Patterns: Parallel Development and Content Management

About the Author

Jim Johnston has worked in the IT industry for over 20 years. He has held multiple positions including CAD/CAM programming, applications integration coordinator, manager of a application launch center for testing and integration verification of PC applications, and testing architect for major web systems. Jim has been involved with SCM for over 6 years primarily in the areas of enterprise web applications, retail web sites, SCM process development and J2EE build and deployment methodologies.

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.