The Decline and Fall of Agile SCM—and the Rise of Lean SCM

[article]
Summary:
There are definite signs of Agile methods (with a capital A) may be losing mind share or, at the very least, suffering from a certain amount of perhaps over-hype or over familiarity. Or maybe it is just the rise in cases of AFS (Agile Fatigue Syndrome). For some, agile is giving away to the rise of lean software development. Where will it end?

Conceived in 2001 by members of the Agile Manifesto, Agile had a tempestuous upbringing. She became well known development circles as the new bright young thing. Admirers were always at her feet. She was cited as the inspiration for major successes and also for bringing some fun back in to development. Gradually the apparent novelty factor began to wear off, though, and became colored by a certain corporate greyness, and began to fade into the background.

Alistair Cockburn presented a similar theme in his presentation at the Agile 2009 conference, “I Come to Bury Agile, Not to Praise It”:  (http://www.infoq.com/presentations/cockburn-bury-not-praise-agile):

Agile has spread to large, globally distributed commercial projects and affected the IEEE, the PMI, the SEI and the Department of Defense.  In last year’s review column, we touched on the Agile Backlash.

There are many signs that Agile has “crossed the chasm” and become mainstream. Many organizations, and perhaps a majority, now claim they are “doing Agile” in some shape or form. How successfully it’s being done will, of course, depend on many factors. There remain, perhaps, misunderstandings of the discipline required for successful Agile development.

There is a relatively common sequence of steps for the adoption of new ideas:  First comes, “That’ll never work!”  As some examples of success come in, this changes to, “It’ll never work here!”  This implies the belief that each set of challenges differ too much for a set way of doing things to.  Finally, “We’ve always done it that way!”

Regarding Agile SCM, we feel that the same sequence has been followed with practices such as iterative development, continuous integration (and all that means for the improvement and repeatability of the build process), test-driven development, and patterns of working.

Some CM people objected to the term Agile SCM, thinking of it is as simply “SCM for Agile development teams” and that calling it "Agile CM" suggests it is somehow different from CM or a "separate branch" of CM. Mario's Moreira's blog-entry The Chicken (CM) or the Egg (Agile)is a clear example of this. It treats the meaning of "Agile CM" as that of "CM for Agile projects", and maintains the status quo of CM thinking.  This suggests that one really doesn't have to do or think about CM in any genuinely different way nor in any legitimately different style in order to accommodate agile projects. From this perspective, it's another buzzword development method and we only need to make some tweaks and tunings here and there. And it's still the same CM as before.

We have always maintained that Agile CM is CM implemented "Agile style" in the agile mind-set, according to the agile values and principles. Our contention was that the phrase “Agile SCM” did not imply throwing away sound CM principles, but rather focusing on satisfying those principles and requirements in a slightly different way by emphasizing collaboration, flow, and working results.   There has been raging disagreement and very little acceptance of this in the SCM community. With the aforementioned "agile backlash" and every ALM vendor and product marketer touting "Agile wares", everything is now "agile".  The term has been so commoditized and watered down that it has lost much of its original meaning and impact.

There has, however, been agreement that SCM has needed to respond to Agile methods and principles, by using Kanban or continuous improvement principles to reduce cycle times, smooth friction in the development process, ensure appropriate automation while still being able to satisfy traceability requirements, etc. We feel that there is a different reason to, perhaps, let the term Agile SCM just be subsumed into SCM, so we will assume that all SCM is agile.

The Rise of Lean

An issue for us is that Agile development has its scope specifically targeted on software development.  Extending it beyond that to other disciplines, such as systems development and whole enterprises, is less obvious. It may also be less applicable. In contrast, Lean applies to the whole enterprise/organization and its attitude toward management, including the perceived attitude toward CM, is different. Brad discussed this in his blog entry Agile Self-organization versus Lean Leadership.

The key difference between business-agility and software-agility is the extra emphasis of the latter on "the people factor" and on the notion of dynamic self-organization of knowledge-workers as empowered, self-organizing teams. This difference between agility at the business-level versus software-level is also the key difference between Lean-vs-Agile.

That is not to say that Lean (or business-agility) doesn't emphasize trusting and empowering workers and teams, they do. They don't have the underlying inherent attitude, however, of just trust us and stay out of the way. The "trust" often doesn't seem to extend as far in the other direction with Agile development (e.g., not trusting management direction to the same extent that management is asked to trust the team).

◦       Lean focuses more on flow whereas Agility focuses more on adapting to change (this is true of both software agility and business-agility). As it turns out, focusing on flow requires being able to adapt to change.  Focusing on adaptiveness and responsiveness requires a focus on flow. Here, the end-results may be quite similar.

◦       Agile software development emphasizes a very hands-off management style of not only trusting and empowering the team(s), but often to the extreme of basically saying to management, "Leave us alone and don't interfere.”  In return, extreme transparency and frequent tangible end-results are guaranteed.

◦       Much of this is due to the fact that the scope of Agile development is limited to software projects and teams, whereas Lean is targeted at enterprise-wide scale across the whole value-stream.

This seems to stem from the fact that software agility started more at the grass-roots level, with developers and teams struggling to create good quality software in the face of what was all too often a fundamentally broken management framework for the governance, visibility, and lifecycle of software development projects. Because they were working and trying to get support from the bottom-up, they needed to be shielded from, and unshackled by, all the dilbert-esque managers and their big, bad governance framework with its "evil" waterfall lifecycle.
In that context, the advice of "just get out of the way and trust us and we promise to deliver working software every 2-4 weeks" is actually a very effective strategy to employ. When management doesn't understand, their attempts to steer, direct and intervene are often the equivalent of friendly fire.  In spite of good intentions, this still yields calamitous results.
Lean comes from a history of a much more enterprise-wide scope, often using a top-down deployment strategy. When management is asked to trust and empower workers and teams, it expects the corresponding change in its leaders and their leadership-style.  They also expect to still play a strong, active, and participative role with their projects and teams.

Agile teams often get a bad rap for having this isolationist attitude toward management.  Sometimes that reputation is warranted.  When leadership understands and values the new paradigm of why and how agile works, along with the why and how the old way didn't, it is then time for leadership to play a different, more active role.  These changes need to take place while still trusting and empowering teams and also enabling self-organization. This is the view that Lean takes toward management and is a big part of why it is better received by management and more broadly applicable for scaling agility up and out in larger enterprises.

Upon understanding this, we can then realize why, perhaps, we need to pass the "Agile" baton on to Lean and the likes of the Lean SSC (www.leanssc.org), rather than the agile alliance.

Furthermore, perhaps the notion of "Agile SCM" (and even "Lean SCM") is inherently flawed, as it limits the focus to software CM, rather than to the whole system that creates and delivers software products.  Focusing it on SCM, without the larger context, might likely result in sub-optimizing by optimizing SCM at the expense of another aspect of the business/enterprise.

That leaves us with "Leaning CM for Agility," which doesn’t invent a new brand name, but rather simply describes the act of applying Lean to CM, presumably as part of applying Lean overall, not just CM.  This is done with the purpose of achieving the attributes of agility: adaptive, goal/end-result driven, iterative, lean, and emergent. Where the results of agility are the ability to quickly and easily sense and make-sense-of, respond-to, and create change through a balance of adaptable people/planning and simple yet flexible structures.

While we haven’t given up on the original purpose of agile, we do recognize the course that the ship has sailed and look more to Lean for additional direction and focus. By virtue of Lean, this supports the notion of making something, in this case CM, more "lean", or "leaning it out" by focusing on value in the value-stream and balancing the load to achieve fast, flexible flow of valued changes. Often we can do this with a relentless focus on simplicity. Granted, many misconstrue the meaning of simple here.  See Simple Ain't Easy: Myths and Misunderstandings about Simplicity for what we mean.  This blog-entry was the original and more in-depth version of Brad's Nov'07 article in Better Software magazine, as well as through simple generativ rules, as in the Dee Hock quote, from Birth of the Chaordic Age: "Simple, clear purpose and principles give rise to complex, intelligent behavior.
Complex rules and regulations give rise to simple, stupid behavior."
   

This also gives rise to the notion of identifying waste in our current CM implementation and eliminating (or preventing) CM "debt". Regarding "CM debt", we need to define what it is, how to recognize it, and thecommon forms of it in CM, such as integration-debt (delayed or big-bang integration), branching anti-patterns, and other SCM anti-patterns. We hope to do this in future articles. For now, you can read Technical Debt - Definition & Resources and see what Chris Sterling has to say about CM debt in Architecture in an Agile Organization.

So perhaps it is more along the lines of “Agile SCM is dead, long live lean and agile for CM”?






J

About the author

About the author

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.