Pini Reznik explains how containers can help shorten the software development feedback loop by drastically reducing the overhead involved in deploying new software environments. This leads to faster build and test executions and simplifies the standardization of the development and production environments, allowing for an easier transition to continuous deployment.
In his seminal article “Continuous Integration,” Martin Fowler defines a set of practices to improve the quality and increase the speed of the software development process. These practices include having a fast and fully automated build all the way from development to production and conformity between testing and production environments.
Since Fowler’s article was published, continuous integration has become one of the key practices of modern agile development, and many of us are in a constant battle to speed up the build process and test automation stages. The growing complexity of software and our aspiration to deliver it to the customer in a matter of days or even hours doesn’t make this battle any easier.
The recent rise of containers as a tool to ease the journey from development to production may help us address these challenges.
Containers (OS Level Virtualization)
Containers allow us to create multiple isolated and secure environments within a single instance of an operating system. As opposed to virtual machines (VMs), containers do not launch a separate OS but instead share the host kernel while maintaining the isolation of resources and processes where required.
This architectural difference leads to the drastic reduction in the overheads related to starting and running instances. As a result, the startup time can commonly be reduced from thirty plus seconds to 0.1 seconds. The number of containers running on a typical server can easily reach dozens or even hundreds while the same server would struggle to support ten to fifteen VMs.
The following article written by David Strauss provides an excellent explanation about containers: “Containers—Not Virtual Machines—Are the Future Cloud.”
I started building deployment pipelines about a decade ago when I moved from a software development role into configuration management.
At my first job I reduced the build time from a week to around two to three hours, including the creation of a VM image and the deployment of a full system on a hypervisor. Over 50 percent of the build time was wasted on the creation and deployment of the VM while the build and the automated system testing required less than an hour.
Throughout the years virtualization technology took a few leaps forward and now we can deploy a fully functional multi VM system on a private or a public cloud in just few minutes. Although this represents a huge amount of progress, it still creates enormous difficulties when trying to create a real continuous deployment pipeline. Having a fast build means keeping the continuous integration build under ten minutes as suggested by Martin Fowler. Achieving this speed normally means confining the testing to running a suite of unit tests; there just isn't the time to build a full image, copy the image over the network, deploy the VMs and run a set of system tests.
But what if we could build a new image in just a few seconds, copy to the cloud only the changed pieces of the software and boot up a fully functional system in under one second?
As unimaginable as it sounds, it is already possible using containers. In addition to being able to deploy and test a full system at the continuous integration stage, containers can also help to reduce the complexity related to supporting variations in operating systems. The same container image created during a normal build can be reused in production environments to remove any potential differences between the operating systems on the developer’s laptops and those on the production servers.
The same functionality is theoretically possible using standard VMs, but in practice it never really works. The overhead of maintaining huge VM images, distributing them and re-deploying them multiple times per day has prevented this way of working from becoming popular.