Software Project Estimation

Findings
Management Buy-in
Some interesting outcomes occurred once this was presented to the team. The technical team was worried that management would reject the estimate, but management was much more amenable than they thought. Intuitively, the process and the graph made sense to them. The graph with a date range of possibilities softened the blow of an estimate that was longer than what management had in mind. In fact, management approved almost immediately with little complaint. They were much more comfortable dealing with a probability and a target range of dates than one date with little behind how that date was determined.

Gaming Behavior Needed to be Addressed
Once we started the project, something strange occurred. While the team was pleased that they finally had enough time to complete a project, we caught some of them rushing through tasks to try to complete them more quickly. A bit of investigation revealed that there was far less management pressure on schedules than we had been led to believe. Team members were putting more pressure on themselves, thinking that management always wanted an earlier release. In fact, management was more interested in a quality release. Team members were gaming the system in a way contrary to what management was interested in.

High Degree of Accuracy
At project end, we found that our estimation was very accurate. We did require some fine-tuning, as some of our estimates were wrong. For example, a high-risk estimate may have been scheduled for much more time than our actual effort, but another task that was underestimated filled up that time. We looked at areas of poor estimation and worked on ways to improve them in the future. We also tweaked the values of our uncertainty factors to try to get an even higher degree of accuracy.

Improved Collaboration and Trust
One remark I heard a lot was that estimation finally had some sort of rigor or scientific process to it. Rather than pulling numbers and dates out of some unknown process, we could use a tool to help answer project estimation questions. For example, if a manager asked if we could add more people or cut scope from the project to reach an earlier date, we could easily adjust our inputs and run a new simulation. Sometimes it would help to some extent, and in other cases, we found adding two people and cutting three scope items made a negligible difference on the date range. However, we worked on these efforts together, and we could point to the output of the tool rather than focusing on individuals-"You're pressuring me!" vs. "You're not giving me the right estimate!" Using a tool and process became a shared resource that was less threatening and less stressful than the prior process.

Estimates went from inaccurate, fear-inciting tasks to a process the team looked forward to. All tasks were accounted for and managed in the process, and the team felt they finally had enough time to do what was required. Management felt that they had more visibility on what actually needed to be completed and a more accurate system for providing them with release targets. As time went on, trust was developed between various stakeholders, and estimates became increasingly accurate.

If you are struggling with estimates and haven't thought of using uncertainty factors, I'd encourage you to learn more about this process.

References
[1] McConnell, Steve. Software Estimation: Demystifying the Black Art, Microsoft Press, 2006.
[2] Spolsky, Joel. Beat the Odds, Better Software magazine, March 2007.
[3] Computational Science Education Project Introduction to Monte Carlo Methods.
[4] Galton Estimation Tool (alpha version), Available at: http://estimate.bz
Also see: Wolfram Mathworld Monte Carlo Method.