Improving Project Agility

Member Submitted

PMI processes are frequently applied to software development projects. Given that information technology has its own set of software engineering processes (SEPs) and that the underlying technologies continues to change at a rapid pace, the application of the Project Management Body of Knowledge (PMBOK) to software development can produce a wide and unexpected range of outcomes. In order to improve process coherency and drive down risk, an IT project manager should be aware of the software "process stack," or the alignment of PMBOK to a SEP.

One popular SEP is the IBM Rational Unified Process (RUP). RUP is a product and SEP that provides a disciplined approach to assigning tasks and responsibilities within a development organization (RUP). Its goal is to ensure the production of high-quality software that meets the needs of its end users within a predictable schedule and budget (RUP). Both RUP and the PMBOK are structured, include standardized roles and documentation, and stress analysis before execution. When provided with sufficient resources, a project manager can produce a positive outcome with relatively low risk in a large and structured software development environment. The PMBOK tends to work well with robust SEPs.

However, as large and small projects have their belts tightened in the face of leaner budgets and increased oversight, organizations are choosing smaller and more nimble SEPs like Agile or Extreme Programming (XP) (See Figure 1). The XP community has identified many best practices that are especially applicable to small, co-located project teams. These practices require less training, stress small releases, extensive testing, refactoring, pair programming, collective ownership, continuous integration and on-site interaction (see Figure 2). It is more difficult to successfully apply the PMBOK to an XP project. Where there is process, XP relies on informal and verbal communication, and tends to delay decisions until they must be made. This is clearly a different approach than that which is recommended by the PMBOK. Rigorous work breakdown structures and Gantt charts will not be adopted easily by an XP development team. Introducing a network diagram may be a tactical error–an XP and highly technically competent audience will expect a plot with pictures of servers and systems deployment information. Any individual who uses a “network diagram” for project planning is likely to be quite confused. In fact, XP has a completely different set of terminology. Essentially, scope is called a "product backlog," stakeholders “personas,” stakeholder requirements "stores and retrospectives," person-days "story points," and a logically related sequence of development activities an "iteration."

Process differences are also cultural. XP focuses on the journey and not the destination (Hussman). Rather than plan ahead and assign clear roles and responsibilities prior to project execution, XP advocates collective ownership. Team members may be empowered to make development changes in the absence of a project plan. This approach may work with a small team for a limited duration, such as for a prototype, but in a large environment and over time, this practice can be detrimental. It is not practical to have collective ownership across a large enterprise organization. If developers believed themselves to be owners without proper oversight, they would likely modify and ultimately accidentally break key infrastructure with the potential for a disastrous impact.

Moreover, XP attempts a useful compromise between no process and too much process, providing just enough process to gain a reasonable payoff (Fowler). The differences in this approach are clearly visible with software refactoring. In a project guided by the PMBOK, a need for software refactoring might need to be formally communicated, assessed, and if approved by a change control board, carefully applied to a new baseline. Instead, XP views refactoring as an ongoing and healthy aspect of software development. XP takes a practical approach to software refactoring, expressing a view of “"when the code smells, fix it." This may mean identifying and repairing a software defect. Or it may mean that 20% into a task a developer has realized that a high level architecture or design can be built or "realized" by two different implementations. By re-doing the 20%, the task will be completed faster and with an overall increase in quality of the final product. XP’s design strategy resembles a hill-climbing algorithm (Beck). You get a simple design, then you make it a little more complex, then a little simpler, then a little more complex. The problem with the hill-climbing algorithm is reaching local optima, where no small change can improve the situation, but a large change could (RUP).

XP's focus on small or short-duration releases makes it more difficult to manage large and complex projects. The rate at which a customer can accept and deploy new releases will depend on many factors, typically including the size of the system, which is usually correlated with business impact (RUP). A two-month cycle may be far too short for some types of system; the logistics of deployment may prohibit it (RUP). The PMBOK and RUP are often at odds with XP’s feverish pace.

There is clearly a degree of conflict and redundancy between XP and the PMBOK. This is not a straight forward process stack. There are a number of approaches to make the combination successful. One approach is to vary the stack over the duration of the project. In other words, some parts of the project might be governed more heavily by the PMBOK and other parts by XP. In terms of over-all involvement, the project manager may be heavily involved in project initiation activities as one would expect under the PMBOK. In development iterations or “sprints” the process may shift towards XP, and the project manager may check with the development team periodically, perhaps every 2 to 4 weeks.

A project manager can also leverage XP’s note card based process. The use of note cards is a core XP practice and is in part intended to create an "aha moment.” A deck of note cards is created, and each card lists a specific task and its estimated duration. The note cards are then presented to the stakeholders like a deck of cards. If there is agreement that each card needs to be completed, but the sum of the estimated durations exceed capacity, then there is clearly a problem. If time and budget are constrained and the parties decide to negotiate scope, it is easy to add and remove cards, and re-arrange the sequence to produce the desired outcome. Once this exercise is complete, it is then a relatively simple task to convert the note cards into a network diagram, identify dates and the critical path, and manage the execution of the project. This approach should be encouraged. Note cards are tangible and may be more palatable to both developers and stakeholders. It is also easier for team members to participate; shuffling around cards is easier than formally reviewing and amending a network diagram.

Another XP best practice is to normalize estimates vs. actuals after a sequence of development activities. Even if the estimates were not produced by a formal process such as PERT and the numbers are inaccurate, normalizing the estimates over the course of a few iterations will produce reasonably accurate numbers. It is trial by error so it may be slow, but by identifying and factoring in variances, the estimates will converge with the actuals.

Collective ownership can also be managed and inserted in targeted places within a program. An infrastructure team may feel empowered over infrastructure development and functional development teams over business verticals. This approach may be conducive to organizations migrating to and soliciting more buy-in for a services oriented architecture (SOA).

A hybrid approach can even address refactoring. From a technical perspective, refactoring is essential practice and must be observed on any project of significant size. Project managers must be sensitive to developers’ requests to perform it, and the consequences of omitting it. Discussion around refactoring should occur openly and without retribution from managers or architects. In cases where refactoring reduces duration there will be a positive variance and so long as the refactoring is well understood, there will be little or decreased risk to the estimate to completion. In cases where refactoring might create a short-term negative variance and / or introduce new dependencies, a project manager might decide to push refactoring towards the end of the project and rebaseline the schedule, or leave it in the hands of a development team's "sprint."

Moving on to deployment, the stated conflict between the PMBOK and XP implicitly assumes that there is no need for change where there are deployment processes and teams that are capable of delivering periodic or “big bang” deployments to schedule. Realistically, these efforts are often manual, brittle, non-repeatable and insufficient. Nearly any application, even secure and redundant ones with users in multiple locations, can be deployed automatically. Most of the necessary tools and technologies have been around for years, if not decades. If a software deployment process and team is sufficiently improved, it will enable the use of many SEPS including RUP and agile ones such as XP.

By embracing XP in an organization that has decided to become more "agile," a project manager can increase a team's buy-in and commitment, and even shift the burden of project planning to the architects and developers. In order to accomplish this, a project manager should further embrace personal interaction, learn a new vocabulary, be willing to apply PMBOK documentation standards alongside and behind the scenes of XP practices, and not be afraid to on-board a Process Engineer and engage the development team to help bridge the gap.

Beck, Kent. Extreme Programming Explained. Addison-Wesley Professional; 1st edition. October 5, 1999.

Fowler, Martin. The New Methodology. Apr 2003.

A Guide to the Project Management Body of Knowledge. The Project Management Institute. 2000.

Hussman, David. Overview of Agile Estimating & Planning. Northern Virginia Software Symposium. October 30, 2005.

Principles Behind the Agile Manifesto. 2001.

Wells, Don. What is Extreme Programming? 1999.

About the author

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.