Agility Throughout the LifeCycle: The Rise of DevOps

[article]
Summary:
DevOps is steadily gaining traction and currency, particularly in the world of web apps. Below you will find an introduction and some pointers to resources and further reading. Many DevOps principles have been around for a long time. This is similar to agile methods and, in some ways, a repackaging of existing principles.

Until it is released into the hands of end-users, new developments are, to reference the lean software term, inventory.  A great deal of time, effort and resources have been invested in making changes, but no value is realized for the organization until the development is actually released and used by the end-users.

What is DevOps?


The term DevOps suggests the combining of development and operations together and then applying some of those skills of development to the issues of deployment. It is also about trying to break down many of the barriers that have grown between these two parts of many organizations.

Is It New?


As mentioned at the beginning, we believe that the principles have been around in various forms for many years.

For those of us who do internal software tool development as part of their job (either from scratch or to customize and configure), DevOps has always been a part of the reality that we have lived with day in and day out for at least a decade or two. Just as agile development calls for generalizing specialists across functions, we have had to do the same across the enterprise:

  • We have had to be experts not merely (nor even primarily) with tool administration/usage, but also with the organizations business processes. Most, if not all, of the enterprise application architecture.
  • We have had to work across not only the software development lifecycle of these tools, but also the deployment lifecycle for installing, upgrading and supporting the tools.
  • We have had to do far more than mere administrative support, including new development for any existing internally developed customizations and their configurations.
  • We have had to be process experts, application development experts, application deployment exports, tool support and administration experts, etc., for most all the business-activities and their processes surrounding those tools.

Is it Elitist?


One of the dangers is that DevOps is perceived as elitist and as relevant to only a few organizations. See Stephen Nelson-Smith’s article which addresses some of these concerns. Does it mean that all sysadmins must be able to program (or at least script actions)? To some people the answer is yes.  A new breed of sysadmin coders will sweep all before them. I believe that is not strictly necessary, though it can be an advantage. The key principle for me is that the operations team needs to consider how to improve the automation of their tasks, and if they can't automate it easily, to seek help in achieving this automation and the business should allocate necessary resources to doing this.

Is It Just a Fad?

Another danger is that of being perceived as a trendy fad and, thus, something that can safely be ignored by the vast majority!

The underlying principles being espoused under the term DevOps have been around for a long time and successful organizations have been implementing them well for some time.

Let’s look at some of the issues and principles involved.

Global Versus Local Optimizations


Too many companies still have local optimizations rather than optimizations across the complete lifecycle.  Their development may utilize agile methods such as TDD, and seem dramatically improved from previous efforts, and yet releases happen infrequently and are not always successful. It doesn’t matter how good your development is if there is a large stumbling block later in the life cycle.

Improving Communication


There is often a lack of communication between development and operations. Each department is often incentivized very differently.  Development gets rewarded for making changes, operations get rewarded for lack of outages, which often means resisting change. It is important to break down the “Great Walls of Ire” as Brad so memorably put it!

This means improving global communication and, thus, co-ordination. Make sure that new releases are not a surprise to operations.

What Are the Constraints For Dealing With Releases


Even if the software team doing the development and support of the tool(s) has the ability to release & deliver frequently, the organization (enterprise) may not have the capability to assimilate release as frequently

  • Some deployments/upgrades may require downtime of business-critical systems or unavailability of business critical repositories and databases, and the business simply can’t tolerate that kind of disruption as frequently or during crunch-times of any particular business-product that uses the tools
  • Some software changes go hand-in-hand with some corresponding process-change and/or organizational change to the business, and the customer organizations cant begin utilizing and realizing the value of your latest software/tool changes until they have made the necessary organization and process changes (which means many of us also have to become organizational change agent/experts in addition to process and tool experts)
  • Any one of many other existing business-critical systems or IT assets which are needed for the deployment of your tool (or its users) may have critical dependencies or freeze periods making it difficult to deploy new releases of your own tools as frequently as desired.

By improving your communications you become aware of and deal with these issues.

Increasing Automation


This means automation throughout the lifecycle. Agile practices such as continuous integration are now commonplace, but continuous deployment is much less so.

However, some companies are achieving multiple successful deployments of new functionality per day, e.g., John Allspaw’s presentation, "10+ Deploys Per Day: Dev and Ops Cooperation at Flickr".

The key here is having the right mind set to seek out opportunities for automation and then to actually implement them – whether through acquiring tools, or by developing scripts or local tools.

What About ITIL?


Many organizations have reaped large benefits from implementing ITIL practices within their service and support teams. However, the development side of the organization has often not been included in such initiatives. Thus there is a still often a “wall” over which new releases are thrown. Once things are in production, the operations team may very effective at supporting it – but there is still that hurdle to get over.

There is nothing in ITIL that says this is the way it must be. Indeed, ITIL V3 takes a more holistic view of the whole lifecycle starting with service design. The concept of an application service then allows organizations to wrap up application development as part of the overall process.  Adding some useful checks and balances, while also improving communications and co-ordination. The service transition book covers the transition into production.   While looking at how to control it does not mean that it has to be slow or a burden, just well managed

By taking a DevOps mindset, we are not asking operations to give up control – we are working with them to improve the overall process, and typically achieve smoother and faster releases with much less friction. The underlying principles are still the same.

About the author

Brad Appleton's picture Brad Appleton

Brad Appleton is a software CM/ALM solution architect and lean/agile development champion at a large telecommunications company. Currently he helps projects and teams adopt and apply lean/agile development and CM/ALM practices and tools. He is coauthor of the book Software Configuration Management Patterns, a columnist for the CMCrossroads and AgileConnection communities at Techwell.com,  and a former section editor for The C++ Report. You can read Brad's blog at blog.bradapp.net.

About the author

Steve Berczuk's picture Steve Berczuk

Steve Berczuk is a Principal Engineer and Scrum Master at Fitbit. The author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, he is a recognized expert in software configuration management and agile software development. Steve is passionate about helping teams work effectively to produce quality software. He has an M.S. in operations research from Stanford University and an S.B. in Electrical Engineering from MIT, and is a certified, practicing ScrumMaster. Contact Steve at steve@berczuk.com or visit berczuk.com and follow his blog at blog.berczuk.com.

About the author

Robert Cowham's picture Robert Cowham

Robert Cowham has long been interested in software configuration management while retaining the attitude of a generalist with experience and skills in many aspects of software development. A regular presenter at conferences, he authored the Agile SCM column within the CM Journal together with Brad Appleton and Steve Berczuk. His day job is as Services Director for Square Mile Systems whose main focus is on skills and techniques for infrastructure configuration management and DCIM (Data Center Infrastructure Management) - applying configuration management principles to hardware documentation and implementation as well as mapping ITIL services to the underlying layers.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Sep 22
Sep 24
Oct 12
Nov 09