In his Behaviorally Speaking series, Bob Aiello discusses hands-on software configuration management best practices within the context of organizational and group behavior.
The new year starts out with the perfect storm for DevOps and highlights the need for robust IT operations. In the coming year, many companies will be focused on stabilizing IT infrastructure and addressing many of the challenges that have so dramatically impacted businesses in recent memory. I predict that DevOps will mature in 2013 and evolve into a widely-respected industry best practice—and this change couldn't come soon enough. IT controls will also take center stage as the White Knight comes riding in to save the day.
These trends are picking up speed with the backdrop of a year behind us that saw an unprecedented number of serious IT incidents and outages. Many companies must raise the bar for how they handle automated application build, package, and deployment or suffer the consequences, as we certainly observed in 2012.
The most dramatic incident in 2012 was the collapse of Knight Capital Group due to the wrong version of networking software that impacted the upgrade of New York Stock Exchange software in July and August. This incident resulted in Knight Capital unintentionally purchasing over seven billion dollars of stock and ultimately resulting in a four hundred and forty million dollar loss. That loss deepened as other customers left the firm; Knight Capital Group was eventually acquired by Getco Holdings Company. The wrong version of a piece of software ultimately resulted in Knight Capital ceasing to exist as an independent corporate entity. I do have good news, though, for professionals whose organizations plan upgrades: it doesn't cost $440 million dollars to implement DevOps!
Understanding the Need For DevOps
I am often asked to review technical books. Recently, I found myself in possession of an awesome new novel titled The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win by Gene Kim, by Kevin Behr and George Spafford. This book makes me wonder if Gene Kim and his coauthors have been lurking somewhere in the room when I have been having my own meetings.
The Phoenix Project does a great job of describing the complex interplay between IT operations, development, as well as QA and data security. This book is a must read for IT professionals who want to understand the need for DevOps. DevOps is itself a must for any organization that wants to effectively implement complex computer systems and maintain a high level of service and support, ultimately enabling the business to excel in a very competitive marketplace.
Death of DevOps
A few years ago, I created a stir within the agile community by predicting the death of agile as we know it unless agile matured into a more robust process model. At the time, I was widely criticized, although a few of my colleagues did understand my point. Industry best practices need to mature to a point where they are well understood and support repeatable scalable processes. IT service management (ITSM) is today well served by the comprehensive ITIL v3 framework and a fairly mature training and certification culture. Without such support, few people would really understand what ITSM is really all about and, more importantly, how to implement it.
Focus on Knowledge Sharing
DevOps is essential because it puts the right focus on the struggle to gain IT systems knowledge that often results in mistakes and major systems outages. There are some psychological issues that impact the individuals, groups and the organizational culture as a whole that Leslie Sachs will be explaining in her upcoming article on CM Crossroads, but I want to focus on the fact that development often gets the most time and resources to get up to speed and build their expertise. Then the systems get dropped into operations (with us, release managers playing a key role on the way).
DevOps puts the proper focus on improved communication and knowledge sharing that can help address the lag in productive interactions. The trick is to push the automation of application build, package, and deployment upstream so that you start by automating the work early in the application lifecycle. This gives IT operations the time to build their tools to support the release and also build the much-needed knowledge base and practical experience. Release engineering is a key player in this effort by automating the entire deployment framework. There are other functions that need to be automated as well, including event and incident management.
Focus on IT Process Automation
IT service management needs to have a strong knowledge base and automated IT processes in order to maintain acceptable levels of service that will keep the business running. This job is getting increasingly complex and we are seeing that the few available subject matter experts (SMEs) need to share their experience and help the entire team build automated processes to support event and incident management. In one recent incident, a high speed trading system was malfunctioning for over forty minutes before someone was able to shut it off. This was incredible and resulted in a steep loss for the firm that should not have happened once the problem was recognized by the operations support team.