A Productivity Comparison of Kanban and Scrum

how we were able to support our conclusion.

For comparison purposes, we used story points per staff week—something that should be used with the same care as lines of code—meaningful at the project or macro level though not at the individual or micro level. We found that the metrics supported our feeling that kanban is more productive than Scrum  given the right circumstance .  I would expect that a reader would agree that our sheer size in productivity gain indicates that kanban merits investigation.

We had one Scrum team working for a total of twenty-six sprints (fifty-two weeks).  After fifteen sprints, we added a second sprint team for eleven sprints.  Both teams pulled from the same backlog, which made coordination less difficult than if the two teams depended on each other. The Scrum team completed 655 story points in 518 staff weeks.  That’s 1.26 story points per staff week.

After twenty-six sprints, the first sprint team moved to kanban and absorbed one person from the second Scrum team.  The remainder of the second team was disbanded.  Our kanban team consisted of eight people working for ten weeks for a total of eighty staff weeks.  During this time, we completed 340 story points. That works out to 4.26 story points per person week (340/80). 

These metrics are summarized in the table below:

A Discussion of the Metrics
At first blush, the difference seems like a tremendous increase, too good to be true. Certainly, the process alone could not account for all of this increase.  A number of factors contributed to the productivity gains.

The Scrum and kanban teams were more or less the same people, meaning that variability inherent with changing team members wasn’t pronounced, but was something of note as the kanban team consisted of eight people rather than seven.  This didn’t seem to be much of a factor since the “new” person was at one point a coworker with the original team, had worked on the product for some time, and was familiar with the agile techniques. 

There was a small impact on the process learning curve.  When starting with Scrum, the development team took about eight sprints before velocity stabilized. We were learning the process and our estimation skills took time to mature.  While this may seem to skew the Scrum metrics, even ignoring the first eight sprints had only a marginal effect on the comparison. Since the teams already collaborated well, understood many of the mechanics of agile development, and had experience estimating, the velocity’s stabilization wasn’t pronounced. Kanban was introduced as an evolutionary change from Scrum and many of the concepts carried over.

In both cases, story point estimations were based on a list of reference stories using estimation by analogy.  When a new story was estimated, it was compared to a reference list of completed, well-understood stories with already established story points.  The new story was then assigned the same number of story points as the analogous reference story.  While there was an inevitable degree of variability when estimating story points, having the reference stories on hand helped keep variability small.  In Scrum, the reference stories had to be grown.  By the time we began using kanban, the reference list was well established.

Perhaps the quality and size of stories was the biggest difference between kanban and