Estimation Roller Coaster

Estimating is hard. How do we do it right? Why do we have to do it, anyway? New agile teams can turn to excellent books, such as Mike Cohn’s Agile Estimating and Planning, to learn how to size user stories and estimate development and testing tasks. Estimates are often abused and treated as hard facts. Newbie teams underestimate, overcommit, and never achieve a sustainable pace. Do we even need estimates? Many teams do well with a lean or kanban approach, where work in process is limited and story estimates are unnecessary. 

My current team has ridden the estimating roller coaster for the past seven years. At times, we’ve become complacent, rolling along at a steady pace. Then suddenly, there’s an unexpected plunge into the unknown. I’d like to share our story of how we’ve adapted our sizing and estimating over time so that our business managers can track and plan and we can deliver value in a timely manner at a sustainable pace. 

Early Learning and Mistakes
Like many teams, we started with basic sizing and estimating practices. It seemed like we spent hours and hours in estimating meetings. We used Planning Poker, a consensus-based estimating technique, to size user stories, learning to compare user stories and assign relative story point estimates. Each person has a set of Planning Poker cards with these point values: 0, ½, 1, 3, 5, 8, 13, 20, 40, and 100. Our average two-point story takes two or three days to complete, including all testing activities. 

Here’s how we use our poker cards to size stories: After the product owner explains the story, we spend a few minutes discussing it and asking questions. Then, we count to three and each team member holds up a card. If we all hold up the same value, we’re done. If not, the people with outlying values explain why they chose them, and we estimate again until we agree on the size. We usually finish about fourteen story points in a sprint. Sometimes, we estimate a “theme,” which is a group of related user stories that provide a new feature. A theme estimate might be something like “20 to 40 points.” 

Sizing stories is tough early on, when your team doesn’t have much history to guide it. A story that we gave three points during our first year but that actually became a forty-point theme remains legendary (and cautionary!). Too often, we sized stories at a point value that was most of a sprint, learning the hard way to slice these into smaller increments. Still, our estimating process was good enough. When business development needed a new, complex feature in order to sign up a new partner, we could deliver it in a predictable timeframe. 

Complacency and Nasty Shocks
After a few years, we had boxes full of estimated story cards that were clearly never going to be prioritized. We studied the work of lean development experts, such as Mary and Tom Poppendieck, and started estimating only stories that would be done in the next two or three iterations. Through experimenting, we found we could best tackle complex new projects by starting with a few one-hour brainstorming and design meetings. We often map the theme on a whiteboard, showing workflow and interaction with the database and other parts of the system. We identify core system behavior and “nice to haves.” Then, we break the big theme into smaller stories and estimate them. If a story seems too large to be completed in two or three days, we break it up into smaller stories or “threads,” usually starting with the end-to-end happy path. We prioritized stories based on risk, dependencies, and importance. Fewer stories “blow up” and take way more time than we anticipated, so we’re able to maintain a steady pace. 

But software development is never certain. It seems that just as we are lulled into a sense of complacency, thinking we rock at estimating, our roller coaster track drops out from under us. Last summer, a five-point story that we thought we could finish in a week stretched into three two-week iterations. We hadn’t anticipated that its requirements necessitated a change in a basic model of our application, requiring not only code changes in other areas but also major database changes. The five-point size should have been a warning; when we get over three points on the Fibonacci-number scale, we enter risky territory. Overestimating? 
At another point in the past year, we were surprised in a completely different way. Our product owner (PO) noted that, for several sprints, our velocity had been much higher than normal. The PO wanted to know if the business could count on velocity remaining at that super-high level or if something else was at play. Had we achieved some kind of bizarre breakthrough after our years of agile development? Were we purposely overestimating stories to look good? Or, was it just a series of weird coincidences?

We decided it was a weird series of events. Production support requests were at an all-time low, so more time could be devoted to new development. Some stories had simply turned out to be much easier than anticipated. We resolved to go back to basics, using a small,  “known” story as a basis for relative sizing of others and breaking stories that would take longer than a few days to finish into smaller stories. If a story turned out much larger or much smaller than anticipated, we had a quick retrospective about what happened. 

A More Agile Approach
Not long after this period of high velocity, our PO presented us with a highly difficult new theme that would eventually lead to rewriting a core part of our application. The existing code for this functionality was very old, poorly understood, and supported by very few tests. In fact, we had never successfully run the whole process in our test environment. We couldn’t even write the stories, much less estimate them. 

We decided to take an extreme agile approach. There wasn’t a hard deadline, as there is with many of our themes, but it was desirable to finish by the end of the year. Our business experts trusted us to figure out the right design and take the time to implement it in a sustainable and maintainable way, while accommodating business priorities. After several sessions where we talked through the desired system behavior and business problems to solve with the PO and other stakeholders, we did a spike to try out a proposed design. During the spike, which took two iterations to finish, the team did performance testing to make sure the design scaled. We felt good about the approach, but there were still a lot of unknowns. 

At this point, we worked with the PO to write and size user stories, and we started them. We did extensive exploratory testing on the code as it was written, learning more about special cases and issues with the technical implementation, requiring more changes to the code and to the database design. We found performance issues in the process and in reports, which required more tuning in the code and database. Complex new test fixtures were needed in order to automate adequate regression tests. Our original story estimates were way too low, and it took three iterations to complete the stories. We decided that next time we have a theme of this complexity, with so many unknowns, we will revisit the stories more often and keep breaking them into smaller increments as we learn more, rather than carrying the same story cards from iteration to iteration. 

Smoothing the Ride
Our business managers want story and theme (or epic) estimates to help them plan just enough ahead. Though we no longer spend time estimating stories that aren’t planned for the immediate future, I don’t see us getting away from estimating entirely. We continually get better at sizing user stories, but they will always just be estimates! 

We’ve been successful by applying agile values and principles. If a project has a hard deadline, we know we have to start as early as possible and work on the riskiest stories first. We accept that unexpected stuff will happen. We learn from it and keep improving. We spend time to manage our technical debt so it won’t drag down our ability to deliver in a timely manner. We’re careful to avoid overcommitting, so that we can work at a sustainable pace and continue to meet business expectations. 

We don’t ever get bored in supporting our business with valuable software, but we don’t need any adrenaline rushes either. The estimation roller coaster can be scary, but, over time, we can apply incremental and iterative agile development to smooth out the bumps and curves.