Modeling Uncertainty


Some may think that predicting a project's future is impossible and a waste of time. Payson Hall did, but when he was forced to think otherwise, he learned there are ways to manage uncertainty. While these methods may not generate perfect predictions, they help conquer the unknown. In this week's column, Payson explains a process for gaining insights into a project's future and getting buy-in on your predictions.

My adventure with uncertainty started with the question "How much hardware will be needed to support the new application once it is in production?"

It was 1988. I was employed by a large systems integration firm, and my job was to design an advanced image processing system to scan, compress, move, decompress, OCR, index, and store forty million document images. As the lead architect I was supposed to be the expert, but this was bleeding-edge stuff. We were boldly going where no one had gone before.

When the technical review manager asked me to predict the final hardware configuration early in the design phase, I thought it was a stupid question that was keeping me from more important work. Until we could build the system, we wouldn't have enough information to answer such a question. I tried to explain this as diplomatically as I could, concluding with what I thought was a helpful suggestion: "Let me build it, and then I'll tell you."

The technical review manager, flown in from company headquarters to make sure the project was likely to be both feasible and profitable, listened patiently to my response and explained, "We need an educated guess about the configuration now so we can make some projections, and you are the best person to make that guess."

I was flattered, impatient, and unimpressed. "I can't know until we have built the system--then we will add hardware until it performs well enough for production," I said.

He went on to explain that our company needed assurance that the system would run on some reasonable configuration before it invested much more in the project.

I still didn't get it. "We will need as much hardware as we need," I said, trying not to sound belligerent.

"Think of it this way," he countered, becoming frustrated, "If you need $20 million worth of hardware, we can cancel your project now. We must develop a crude model of system behavior and assumptions about its use today, based on what we know now and on what our experience tells us are reasonable assumptions." He explained that if the initial model of the system suggested it would run on a reasonable hardware configuration, the project would continue and we could refine the model as more information became available.

Over the next few days, he helped me build a spreadsheet model of the system's components based on assumptions about the memory and processing requirements for each part. We documented assumptions about the response times needed for various tasks and established a "budget" for processor cycles, memory, and network bandwidth for different activities. We identified the tasks that required real-time processing and those that could be done asynchronously off hours. We imagined together what the application usage patterns might be and documented assumptions about the transaction mix and throughput requirements. When we finished, we had a guess about the necessary configuration.

We gave headquarters the results of our preliminary analysis to provide crude assurance the system would run on a reasonable configuration. We agreed to revisit the configuration prediction after a few months when we would have real performance and resource consumption information about some of the components that would allow us to validate and refine our assumptions and the model.

Those first estimates were wrong, but not ridiculously off. In the end, we had not allocated enough processor by a factor of four and had configured too much memory by a factor of two. That ballpark estimate allowed our management to make an informed decision to continue the project. Our work on the model also gave us

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.