David Thach and Rick Rene share what they have learned are the most effective and readily adoptable agile processes, as well as a few techniques to integrate hybrid waterfall approaches. Companies adopt an agile software development framework to become more effective and more efficient, not to become a model of purist agile utopia—which, if attempted, ironically can be immensely costly and detrimental to progress, if not disastrous.
Through our combined forty years of experience consulting and running IT organizations in Fortune 1000 companies, we outline in this white paper what we have learned are the most effective and readily adoptable agile processes, as well as a few techniques to integrate hybrid waterfall approaches.
Companies adopt an agile software development framework to become more effective and more efficient, not to become a model of purist agile utopia—which, if attempted, ironically can be immensely costly and detrimental to progress, if not disastrous. A purist agile approach is not realistic for highly complicated Fortune 1000 companies that inherit unideal agile conditions. Teams with technical specialization, complicated reporting structures and governance committees, long-term roadmap planning and budgeting, conglomerate and subsidiaries in geographically dispersed locations, multiple shared product owners, mature legacy third-party installed applications, inherited roles absent in purist agile teams, a variety of IT vendors and partners with near-shore and off-shore blended workforces, and entrenched waterfall frameworks throughout hundreds of integrated applications are just a few Fortune 1000 company conditions that make it difficult to adopt agile.
As an example, one of our clients attempted to reconcile the Fortune 1000 conditions mentioned above with a full-on purist agile approach. This resulted in a fifty-six-page slide deck of prescribed instructions on implementing agile in their inherited Fortune 1000 environment that would also pass muster with all other stakeholders and their ancillary Fortune 1000 processes. It was complete with eight process flows, half-a-dozen swim lanes, and dozens of decision trees. They created this agile document after a year-and-a-half was spent gathering requirements in a waterfall-based approach. Their management, frustrated with the lack of progress, wanted to adopt what they heard was a new framework to push work forward more rapidly. However, the team ended up spending even more unproductive time creating the agile slide deck through countless discussions and approval rounds. And this was all before iteration 0 had even started.
In the end, they abandoned the fifty-six-page agile implementation slide deck, disbanded their attempts to socialize it, and instead called on us. In twelve weeks we not only implemented our simplified framework version of agile—what we are calling the “least common denominators” of agile—but we also developed the foundational components of their next-generation operational software.
As you can infer, there are aspects of agile principles that are more easily implemented in large, complex IT organizations that generate almost immediate benefits with very little relative disruption. These “least common denominators” of agile offer the biggest bang for the buck while being unintrusive enough to phase in at Fortune 1000 corporations. There are also hybrid waterfall and agile techniques that can be combined in order to gain the benefits of agile while both maintaining the benefits of the waterfall approach, which has been entrenched in the majority of Fortune 1000 companies, and enabling the framework to be more readily adoptable within these highly complex IT organizations.
Least Common Denominators of Agile
Agile is meant to be simple, but like all processes that start simply, it can quickly spiral out of control as layers are added and canonized in an attempt to standardize behavior. As a result, there have been tens of thousands of pages written about agile. But in this article, we are going to try to do the exact opposite and distill agile to its least common denominators. Agile can be broken down into three main process themes: artifacts, ceremonies, and engineering techniques.
The three key artifacts of agile are the product backlog, the iteration backlog, and the burn-down chart. Any individual team within a complex Fortune 1000 IT organization can adopt these artifacts by simply making a list of software requirements and to-dos (product backlog), breaking them into logical groups of work with estimates and assignments (iteration backlog), and tracking the work remaining (burn-down chart).
The four key agile ceremonies are the release planning session, the iteration planning session, the demo, and the retrospective. Any individual team can adopt these ceremonies, even within a complex, nonagile, waterfall-heavy Fortune 1000 IT environment. Scheduling a meeting to discuss comprehensive software needs and high-level estimates? That’s a release planning session. Scheduling a meeting to break requirements into tasks, make estimations, and give assignments over a two-week period? That’s iteration planning. Scheduling a meeting to showcase your work over the iteration to your stakeholders? That’s a demo. Scheduling a post-mortem meeting to discuss what went well and what went wrong? That’s a retrospective. By holding regular demos and retrospectives with stakeholders and product owners every two to three weeks, the agile and iterative feedback will ensure the project stays on track.
Engineering techniques associated with agile, including test-driven development, continuous build integration, automated regression testing, and scripted deployment, are more difficult to achieve but still fundamental. Start with a pilot team and ask developers to create test harnesses that fail until the functional code they write passes the unit tests. That’s test-driven development. Ask all developers to run a build of the code along with the test harnesses every time they check in files. That’s continuous build. Adopt a quality assurance tool that can automate behavior testing. You’ve got automated regression testing. Spend an iteration having developers work with the infrastructure team to create a deployment script. Scripted deployment complete.
The reason the least common denominators of agile are so easy to adopt is that it can be done in small pilot teams where it makes sense, without large organizational behavior change, major schedule impacts, new documentation requirements, additional resources, software adoption, costs, or substantial approval requirements. But by implementing these basic common denominators of agile, the benefits gained include an increased rate of software success, decreased software development costs, and more alignment with stakeholder vision due to the framework’s inherent flexibility and iterative process.
Hybrid Agile Techniques
At Daugherty, we have lead some of our clients to engage in thorough requirements elicitation prior to the start of software construction, very much like the start of a waterfall process. However, afterward we iteratively construct software using the least common denominators of agile described above. What makes this method ideal for Fortune 1000 companies that must engage in detailed planning for steering committees is that it provides the benefit of being able to have clear requirements up front, as well as associated benefits such as being able to accurately estimate a budget and resources and establish a high-fidelity timeline. All the benefits of the waterfall methodology are preserved, but you also get the benefits of agile.
At Daugherty we have honed the requirements process into what we call the RPM, or the rapid process map. Largely developed by one of our principal consultants, Phil Delanty, internally we also call it the 4-4-2. Those numbers represent the prescribed method of spending four hours eliciting requirements, four hours reconciling the requirements, and two hours reviewing them, then iteratively repeating the process until the scope of requirements is documented. As part of the four-hour requirements capture and four-hour requirements reconciliation, technical architects, user interface designers, database administrators, and other contributors capture and document use cases and wire frames and create data models for the two-hour review. It is rapid and it is aggressive. It requires talented individuals who can think and work on their toes. Once captured, the information is documented in a proprietary orthogonal view that allows for accurate work estimation and timeline planning to carry out in either a waterfall or an agile fashion.
The Old with the New
When the Agile Manifesto was published in 2001, it ushered in a time period of rapid agile software methodology adoption by IT organizations. Research began to show organizations that adopted agile practices created software with higher quality, better return on investment, and greater stakeholder satisfaction while delivering solutions faster and less expensively than traditional waterfall approaches. However, contrary to those statistics, on large enterprises within Fortune 1000 organizations the authors started to see high-profile project failures associated with agile strategies. Without a more disciplined approach to agile at the enterprise level, the agile movement will risk losing the hard-earned momentum agile pioneers have worked to generate. Sometimes it is better to adopt the least common denominators of a process and create a hybrid of new and old than to throw away the old and adopt a purist framework.