A Post-dot-com Story
Between 2002 and 2006, I lived in New York City and worked for an Internet advertising startup called WhenU, which made a software product and distributed it to a very large number of users. I was one of the few early-stage software engineers at the company. The product displayed relevant ads and search results based on users’ browsing patterns.

The Internet advertising field attracted many players that came to be known as “spyware.” Unlike them, our company didn’t send user data to our servers. Instead, we ran all logic on the users’ computers using a frequently updated, impersonal set of heuristics. We were the good guys!

WhenU was a precursor of lean startups. Of course, we cannot say it was a lean startup, because the term did not exist yet. The company represented a resolute departure from the dot-com ways of the 1990s. It actually made a profit and wasn’t eager for an IPO. More importantly from today’s perspective, we built, measured, and learned.

The company challenged its software engineers to assume much broader roles. For example, we were always actively involved in deployments. After delivering the software to the users, we also measured how it performed in the field. The company quantified user feedback, tested hypotheses, and processed large amounts of (impersonal) statistical information. The business learned from the statistically validated results, for example, that its new algorithm increased the click-through rate in the travel category by 50 percent.

I was trained, experienced, and believed I understood the software development process or whatever that meant at the turn of the millennium. Then, I landed in what today we might call DevOps, and I had to dust off my statistics textbook, too.

Reflecting on what went on in our New York office with the benefit of today’s knowledge and hindsight makes it very clear how the modern kanban and lean startup concepts can be combined. But first, we have to revisit Eric Ries’ lean startup principles and apply some lean-kanban thinking to WhenU’s operations.

Interpreting Eric Ries’ Lean Startup Principles
When I read Eric Ries’ The Lean Startup [1] years later, some things began to look familiar. Before I offer my interpretation of his five lean startup principles, let me restate them:

1. Entrepreneurs are everywhere
2. Entrepreneurship is management
3. Validated learning
4. Innovation accounting
5. Build-measure-learn

The first two principles define what lean startup is about, while the last three give guidance on what you can do with lean startup and how. The third principle, validated learning, defines what is being produced, and the fifth principle, build-measure-learn is an outline of its value stream. Identifying the value that is being produced and mapping its value stream are among the first steps of applying the kanban method. The fourth principle, innovation accounting, seems to be independent from kanban.

Figure 1: Interpreting the lean startup principles

We can now apply the kanban method [2] to the production of validated learning in a lean startup environment.

Specifying Value, Identifying the Value Stream
The most important point in a value stream is its downstream end, where something valuable is delivered to the customer. Start mapping the build-measure-learn process from that end by specifying value. James Womack and Daniel Jones describe this very clear approach in their book, Lean Thinking [3]: “Value can only be defined by the ultimate customer.” Therefore, in order to apply lean startup‘s third principle, we first need to know who the customers of validated learning are.

The customers of validated learning are not the same as the customers or users of a company’s product. They are the founders who consume pieces of validated learning and decide what they need next as they guide their company. Getting the next piece of validated learning done requires building a product, delivering it, getting facts about what customers and users do with it, and learning from those facts.

After specifying value, we can move to the next step advised by Womack and Jones, identifying the value stream. The fifth principle of lean startup, build-measure-learn, postulates what the value stream should be and broadly outlines it: Build the product, measure how it’s used after the delivery, and learn from the data.

Having identified the value stream, I‘ll now discard it in favor of a concept that is more modern and also much more suitable to knowledge work: the knowledge discovery process described by David Anderson in a blog post in late 2011. [4] The knowledge discovery process focuses on dominant activities rather than work centers and handoffs of work items between them. The measure and learn phases we experienced at WhenU can best be described through this process.

Back to Our Startup
At the starting point of WhenU’s measure-learn process, a new software product version was already built, tested, and ready for release. One activity dominated this process early on: ensuring that our VP of advertising operations didn’t have to yell the next morning, “Dude, where’s my revenue?”

This potential exclamation can be rephrased as a statistical hypothesis and tested on a small percentage of users. We aimed to prove that our latest features didn’t do any harm and all pre-existing functionality worked as before and produced the same revenue. As the results came in and our confidence grew, the activity began to produce diminishing returns and a new activity started to dominate.

Our next activity was to measure any improvement we could attribute to the new features, such as increased revenue per user. We usually released the product to larger percentages of our users and ran different statistical tests. This activity also eventually faded away as we gained confidence for the full user-base rollout. We were then faced with the last test: Could we use the improved product and the statistical proof of improvement to attract more paying customers?

Figure 2: Example of a measure-learn process

Just as David Anderson describes in his knowledge discovery process blog post, the key points in our process were not the handoffs between individuals, teams, or departments, but rather the changes in the dominant activity. For example, our initial experiments typically involved a core product engineer (usually me), the vice president of advertising operations, a senior advertising campaign manager, and an operations guy. When we moved on to quantifying improvement, we needed to create new ads that weren’t even possible with the old system. Therefore, we brought in additional collaborators, including  another campaign manager, a UI programmer, a graphic designer, and the creative director. In the final stages, the core engineer’s role diminished greatly, but the same campaign managers and the creative staff worked more closely with sales—collaboration and no handoffs!

Build-Measure-Learn as a Kanban System
The above measure-learn process can be mapped onto a simple kanban board. The boundaries of this Kanban are “ready for release” (upstream) and “validated” (downstream). There are policies for pulling work from one state to the next. For example, in order to proceed with improvement tests, you had to finish the controlled experiments and have statistical proof in hand that everything was OK up to that point. There are, obviously, tight work-in–progress (WIP) limits, as we were constrained on how many experiments we could run at the same time.

To be sure, there was no kanban method back then. David Anderson’s kanban book was still years away from being published. I am merely using a modern apparatus to think about a past situation. If we knew back then what we know now—for example, visualize—our measure-learn kanban board might have looked like figure 3 at some point. (I dropped the WIP limits from the board, because they would evolve anyway and the precise limits are not the point here.)

Figure 3: The measure-learn kanban board

“Ready for release” was both the upstream point of the measure-learn kanban and the downstream state of our build kanban. The exact software development process used to build a software feature is not important here, but we can use the modern kanban method to model it. The board visualizing it could look like figure 4 at one point.

Figure 4: A build (software delivery process) kanban board example

By joining the two kanbans together, we get the build-measure-learn kanban.
 

Figure 5: The combined build-measure-learn kanban board

The Thousand-foot Views
I had the image of this long kanban board in my head while reading Eric Ries’ book, and when I reached pages 138-139, I found a kanban diagram right there, with four columns: backlog, in process, built, and validated. There were WIP limits on “in process” and “built,” and the author not only explained how pull and WIP limits work but also showed how to use them to create tension leading to a continuous-improvement culture or kaizen. This knowledge was not available to us in 2005, and our startup never took those steps.

The build-measure-learn loop is a 40,000-foot view. It is a simple diagram that may be sufficient to inform a non-technical company founder. Everybody else at the company needs not only to understand Eric Ries’ kanban diagram but also to get more detailed views from 30,000 feet and down. The long build-measure-learn kanban diagram from the previous section shows an example from one particular company of what a 30,000-foot view may look like. Build, measure, and learn should not be viewed as atomic steps, because a variety of activities informing the lean startup practitioner may be discovered behind those labels.

Optimizing for Response Time
Having the new build-measure-learn kanban board in front of us, we must ask ourselves: What is our optimization goal? For the lean startup, the optimization goal is definitely the response time. Why? Remember, the “product” here is not the product itself, but the validated learning (principle 3). The consumers of validated learning are the founders or venture capitalists. They want to discover their next pivot point as soon as possible. They don’t care as much about throughput (the rate of accumulation of validated learning), as they do for the key bits of validated learning that lead them to the next pivot point. Therefore, the response time is king.

Innovation Accounting Is Independent from Kanban
The third and fifth lean startup principles are about execution and are based on lean thinking and the kanban method. The fourth principle, innovation accounting, is about guidance and is independent from kanban. Innovation accounting can tell the founders when to pivot and when to persevere. It can inform them that something is a vanity metric or point them to an actionable metric. The fourth principle is about how entrepreneurs can use validated learning to guide their companies to new, successful business models. You cannot expect to get all this from kanban.

Kanban Can Help Lean Startup, and Lean Startup Can Help Kanban, Too!
Although we have so far considered ways the kanban method can help lean startup, we need to consider an alternative pattern: Lean startup can help kanban, too! I strongly encourage you to check out the work of Canadian kanban consultant Jeff Anderson [5]. He lays out a process of improving an existing software delivery process—a gradual enterprise change enabled by kanban. Anderson is the “entrepreneur,” the improvement process is his “startup,” and how the organization responds to change is the validated learning. He applies innovation accounting at each step of his improvement initiative. As Anderson’s ability to guide his improvement initiative to its next pivot point and onwards produces validated knowledge of how the process change is working.

Another good example of this pattern can be found in the work of Israeli kanban coach Yuval Yeret ([6]). “Start with a minimally viable change based on the core principles of the kanban method,” he writes to describe his approach to coaching organizations in kanban, which parallels the work of many Internet entrepreneurs who release minimally viable products to their users in order to obtain validated learning.

Observations
Optimizing build-measure-learn kanban systems is difficult, as many of today’s agile and lean implementations seem concerned only with the software delivery process or the “build” part—and even that is proving difficult. The build-measure-learn system is more complex and gives a preview of what the future holds for today’s agile practitioners.

If the software delivery process is optimized for either response time or throughput, it can lead to local optimization in the bigger build-measure-learn system, making the whole system sub-optimal. A company may need to rethink its practices when making a transition from its agile build process to build-measure-learn.

Optimizing for response time may at some point conflict with optimizing for throughput. Startup founders want optimization for response time, but not every human activity has the same optimization goal. If you need to optimize for something else, you may be outside the boundary where you should apply the build-measure-learn principle.

Optimizing for response time is likely to involve a lot of slack in the system. Already, startup programmers are rapidly learning the newest technologies and using their 20 percent time to create many open source tools that make other companies leaner.

Conclusions
Kanban is an important process-improvement tool for technology organizations. Lean startup is a new approach to discovering new, innovative ways to do business. To get the most from both, it is important to understand how they relate to each other.

Lean startup is called lean because it has kanban at its core. The build-measure-learn loop is a kanban system optimized for response time. It is therefore important for those trying to apply the lean startup principles also to understand the kanban method.

While kanban is necessary, it is not sufficient. Other elements of lean startup, such as innovation accounting, are independent of kanban. And while kanban can help lean startup, lean startup also can help kanban. In this pattern, the lean startup concepts apply to the evolutionary change of a technology organization enabled by the kanban method.

References

1. Ries, Eric. The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. New York: Crown Business, 2011.
2. Anderson, David J. Kanban: Successful Evolutionary Change for Your Technology Business. Sequim: Blue Hole Press, 2010.
3. Womack, James and Daniel Jones. Lean Thinking: Banish Waste and Create Wealth in Your Corporation, Revised and Updated. New York: Free Press, 2003.
4. Anderson, David J. “Understanding the Process of Knowledge Discovery. Accessed in Oct. 2011.
5. Anderson, Jeff. “Applying Lean Startup techniques to Kanban (or any other) Organizational Transformations.” http://agileconsulting.blogspot.com/2011/11/eric-ries-definition-of-startup-is-as.html Accessed in Nov. 2011.
6. Yeret, Yuval. Kanban Method Change Machine – As a Story Map. http://www.slideshare.net/yyeret/kanban-method-change-machine-as-a-story-map Accessed in Mar. 2012