It's late at night after a long, hard day at a customer's site. The normal installation panic has been worked through, the list has been punched, and the customer is happy... at least until the next crisis. It's "Miller Time" and the beers and stories are going around the table. John is quietly sipping his brew, not saying much. Finally, he puts his beer down, looks around the table, and says, "I've got a story that tops all of these."

Once upon a time, many years ago, I worked for a company that manufactured and installed Automatic Guided Vehicle (AGV) systems in factories. This company has long since gone out of business, but at the time it was on the bleeding edge of technology. AGVs are electric vehicles that follow wires embedded in the floor, automatically trundling material. They are extremely quiet, so to prevent people from absent-mindedly stepping in front of them and getting run down, they make a beeping noise as they move.

One time the company was installing a system for a large airplane manufacturer. This system was to transport the tooling for the tail section of a jet airplane. The tail sections are made entirely out of carbon-fiber composite materials that are heated under pressure in large autoclave ovens. The ovens are constructed like massive bank vaults. The AGV would roll right up to the vault; the door would magically roll open; and the vehicle would dutifully roll inside, pick up or drop off its tooling, and then roll out--the door majestically closing behind it.

This facility cost the customer several hundred million dollars to produce. When a company buys a facility like this, it typically contracts a large firm and buys the whole facility from the AGVs to the autoclaves, including the office furniture and the paper clips. Afterwards, the facility is turned over from the contractor to the company. This transaction has great legal and financial ramifications, so there is often an official ceremony when the purchaser takes ownership.

The project management team for the contractor and the aircraft company were fascinated by the AGVs. They would watch the AGVs for hours. The vehicles glided up to the autoclave door, the door rolled open, and the AGVs disappeared inside. Minutes later, the vehicles exited and the door closed. The project management team thought it was the coolest thing in the world.

It is easy to become captivated by the technology you are managing. The tool can become the product, rather than a means to the project's goal. This is especially easy for project managers who typically aren't as close to the technology as the technical staff. You can get blinded by the blinking lights and lose your sense of perspective.

For the change of ownership ceremony, the project managers decided that it would be cool to showcase the great technology the customers were getting. Presidents, vice presidents, and members of the boards of directors for both the customer and the vendors would be present for the ceremony. It would be a great opportunity to show everyone how spiffy this facility was, and how wisely their money was spent.

The project management team recalled how elegantly the vehicles followed the wires in the floor,and the majesty of the whole transportation process. Couldn't the team use the AGVs as part of the ceremony? Couldn't the person who ceremoniously presented the facility to the customer ride one of the vehicles to the ceremony platform? The project management team members imagined a heroic figure, a Captain Composite, who would emphasize the mythic size of their accomplishments in completing this project.

"Can we use your vehicle in the turnover ceremony?" the contractor's management team asked.

"Of course you can," our management exclaimed. "It would be a wonderful marketing opportunity for our products!"

Obviously, this was a change to the project scope. It should have been treated like any other customer request, which requires a clear headed analysis of the costs, benefits, and risks. However, it seemed like a very small change. These small "freebie" changes are often the ones that backfire, sometimes catastrophically.

As a marketing exercise, the benefit was small for the vendor: those scheduled to attend weren't the ones purchasing these systems. On the other hand, it was a very public forum in which something could go wrong. At the very least, a closer scrutiny of the project was called for. But this was such a small change.

All other work on our project stopped two weeks prior to the ceremony. A simple, bulletproof program was created for the AGV. The vehicle would be in the autoclave, the door would roll open, and the AGV would glide over to the reviewing stand carrying Captain Composite. It would continue past the stand, stop, and then back up until it was right in front of where the CEO for the aircraft company would be sitting. At this point, the presentation would be made. It would be epic.

As the date for the presentation approached, the management team became increasingly nervous. Team members would come down and make us run through the AGV program, once, twice, fifty times -- over and over again, they ran the program. It was flawless.

The decision makers on the customer's side were getting nervous as well. They realized that a failure would grant them the kind of notice you don't want from your company's CEO. They started thinking about what they could do to mitigate the risk.

Then the debate began over who would actually ride the AGV. Everyone on the customer's management team seemed to agree that it was a great idea for someone else to be the noble Captain. Their reluctance continued to grow when they realized that a costume would be involved. Mysterious back ailments and vertigo problems began to materialize in the likely candidates.

The tension in the project managers grew greater. They decided that all of the technical people should be sent home. Geeks mingling with august personages just wouldn't do. "No! No!" they cried, "What if something goes wrong? We'll be quiet as church mice, hide under tables in the control room. They won't even know we are there!" No, it was decided. It just wouldn't do.

Somebody decided that it was too risky to have technical types around. This risk was never clearly explained, and no formal risk statement was made. It is always dangerous when you act on your feelings rather than on a lucid analysis.

So it came to pass that on the appointed days all of the technical people who had anything to do with creating the program got on planes and started the long flight home from the factory.

The customer also hired an independent consultant to be in charge of technical direction for the control package for the AGV system. Several hours before the ceremony, the overall project manager decided that he wanted to see the AGV go through its paces just one more time.

The AGV rolled out of the autoclave, glided past the reviewing stand, stopped, and backed up until it was right in front of where the CEO was to sit.

"Can you make it so it doesn't stop and backup? Couldn't it just go straight to where the CEO is going to sit?" asked the overall project manager.

"No problem!" said the consultant.

This is an example of a last-minute change in scope - about as last minute as you can get. Ad hoc changes to the system are always dangerous because they completely bypass all of the process safety that you have built into the system. The correct response in this case should have been "No - there isn't enough time." But the consultant knew that it was a simple change.

The consultant made a change to the program. It didn't work. The consultant made another change. That didn't work either. The overall project manager became increasingly nervous.

"Just put it back the way it was," the overall project manager said nervously.

"No problem!" said the consultant.

It was at this moment that the consultant discovered that he didn't know where the backup for the program was, and he didn't know how to reverse the changes made.

High above the central United States, the technicians were flying home. One after another, their pagers went off. There were no telephones on the airplane. All they could do was look at each other and shrug.

At this point, the overall project manager was apoplectic. Dignitaries, shooters, and others with god-like powers to smite project managers were beginning to file into the reviewing stand with expectant smiles on their faces.

"No problem!" said the consultant.

The consultant told the project manager that the AGVs could be controlled remotely. He would just walk alongside the AGV and drive it. The presentation wouldn't be as cool as the AGV driving itself, but it was still better than nothing. The overall project manager despairingly told the consultant to do it.

As I mentioned before, AGVs are large, self-guided machinery. To warn people of their approach, they typically make a beeping noise:

"Beep!...Beep!...Beep!"

But not in this project. The vendor's hardware engineers had gone to a bar with the customer's engineers one night. Over more than a few beers, the hardware engineers mentioned that for a previous customer, car audio systems with CD players had been installed in the AGVs. When the AGVs were driving normally, they would play music. They would only beep when they were in an error state.

The customer thought this was an outstanding idea. They requested an alteration to install CD players in their AGVs. And so, for many dollars, the AGVs were retrofitted with CD players.

Some time later, the vendor's engineer's sold them another alteration to remove the CD players from all of the vehicles, for many more dollars. It seems that the members of the factory's safety committee couldn't agree on whether the vehicles should play heavy metal or country music. But that's another story.

Of course, every ceremony needs mood music. The overall project manager had requested "that music from 2001: A Space Odyssey." No one could find the soundtrack, and no one knew that the music was "Also Sprach Zarathustra." The overall project manager compromised with "God Bless America" sung by Ethel Mermen. The stage was now set for the turnover ceremony.

It was show time, and all of the dignitaries, shooters, and those with god-like powers were sitting in their appointed chairs with expectant smiles on their faces.

The lights in the factory went out. A spotlight shone on the massive, vault-like door of the autoclave -- the door majestically rolled aside. The dulcet tones of Ethel Mermen came wafting from inside, hushing the crowd. The AGV rolled out of the autoclave and on it rode Captain Composite!

Captain Composite turned out to be the most junior member of the project management staff, who had been reluctantly coerced into the role, proving once and for all that "it" really does roll all the way downhill. He was sitting on a large wooden throne, which had been spray painted gold, and he was wearing a skin tight purple velour jump suit with a black mask. On his chest were a couple of foot-high letters in green: CC. He was carrying a large, ceremonial golden key to the facility that he was to present to the CEO of the company.

At the last moment before the door rolled aside, fifty pounds of dry ice had been dropped into a bucket of water at Captain Composite's feet. Thick, impressive smoke billowed from the bucket, shrouding the purple-clad hero. Alongside the vehicle, attempting to look very inconspicuous, was the consultant.

It is always a good idea when running a test to make the test environment as close to the deliverable conditions as possible. The AGV had never been run with the dry ice before. This was yet another example of a small, last-minute change.

The consultant drove the vehicle forward. Suddenly, the vehicle (and Ethel Mermen) stopped! Red emergency lights started flashing all around the vehicle as the alarm sounded.

"God bless Americ-...Beep!...Beep!...Beep!"

The consultant frantically pushed buttons on the control box, resetting the emergency stop. The AGV started with a jerk, rolled forward two feet, only to stop again!

"God bless Americ-...Beep!...Beep!...Beep!"

The expectant smiles were disappearing from the faces in the crowd and were quickly being replaced with worried frowns.

Once again the consultant frantically pushed buttons. Again, the AGV started with a jerk and rolled forward, then stopped. Captain Composite looked down from his golden throne in obvious alarm.

"God bless Americ-...Beep!...Beep!...Beep!"

Worry was replaced with anger on the faces of the dignitaries. What was wrong with this multimillion dollar AGV system, anyway? The only representative from the vendor -- the president -- later stated, "I looked around for a hole to crawl into, but none was available."

The flashing emergency lights lit the dense fog with an eerie red glow. Suddenly, the consultant realized the AGV has side-collision photo-eye detectors. The dense fog from the twenty pounds of dry ice was blocking the photo-eye!

The consultant lunged for the bucket and kicked it off the AGV. The bucket and fifty pounds of smoking dry ice went skittering everywhere.

The consultant reset the emergency stop. The dulcet tones of Ethel Mermen resumed. The AGV rolled up to the reviewing stand, and a much shaken Captain Composite presented the ceremonial gold key to the stern visage of the CEO.

Well, I don't have to tell you that the AGVs were put underneath the project management microscope after this little faux pas. Word came down from upper management to "straighten out this AGV mess right away!" The AGV system was viewed as a Jonah, and for months afterward, anything that went wrong in the facility was blamed on the AGV system.

* * * * *

What lessons can be gleaned from this fiasco? First of all, no one involved thought to perform a risk assessment, even though they were done regularly on the project. Why did they deviate from their process? Because this demonstration was not considered an important part of the project.

The most important thing was the function of the product - moving the tail section of an airplane. The demonstration was a frivolity, an afterthought that had nothing to do with the real project. Because it was a sideshow, there was no reason to employ risk management. Besides, the system was working fine! Nobody considered the possible impact of the last-minute changes that were made.

In all of the articles and essays about risk management that I have ever read, there has never been a discussion of what happens when upper management, the people who can squash you like a bug, witness a failure. The focus is always on the successful outcome of the project, as measured in terms of the core project requirements. There was no requirement in this project to transport purple pajama-clad faux super heroes. It was unimportant to the project. However, this little sideshow had the potential for a huge impact on the project's life.

A wise project manager knows that you can deliver all of the project requirements on time and on budget, but if upper management comes to believe that you are a bozo, you will end up managing ice procurement in Antarctica. While this demo had little relevance to the overall project, it was highly visible. In fact, it was likely to be the only visibility the decision makers in the customer's company would have. They could probably have survived delivering three months late. They were not going to survive a pratfall in front of the CEO.

Because no explicit risk assessment was performed, the people involved easily confused probability and impact. For example, consider the decision to send all technicians home. There was a high probability that the technicians would be visible to the dignitaries when they toured the facility. So what? They were unlikely to be noticed. They would have blended in with the rest of the furniture.

There were several last-minute additions to the requirements of the demo. The assessment of the relevance and potential impact of these additions was made by those without the technical resources to make an informed decision. A full dress rehearsal could have identified these problems, but it was eliminated because of the inconvenience of obtaining the necessary materials. The project team had a false sense of confidence and a lack of appreciation for the risks that the changes they were considering would entail.

The project managers were overly enamored with the technology they were integrating. To them, it was almost magical. They knew that it would look really cool to the dignitaries. This enthusiasm caused them to overlook the technology's limitations and risks.

They ignored their instincts repeatedly. Their nervousness as the appointed date came closer should have been a clue to them that they were taking a huge personal risk, even if it wasn't a risk to the project itself. It should have served as a wake-up call to use the processes that were making the rest of the project so successful.

However, the biggest lesson concerns the risk of doing this little project at all. Everyone involved was wrapped up in how cool the technology was. No one thought about the possibility of screwing up in front of all of those dignitaries. The whole ceremony added absolutely nothing to the event. The building could have been just as easily turned over without the withering scrutiny of all of those executives turned on us. Captain Composite was the classic career limiting event.

First Law of Demos: If the demo goes well, they will never remember you. If the demo goes poorly, they will never forget.

Postscript: Time passes and heals all wounds. They finally got the system accepted by the customer. About six months after the Captain Composite fiasco, a new overall project manager approached the vendor. They were going to have "family day" at the facility, where the general public, press, and family members of the workers would be invited to tour the facility.

"We want to install a dance floor on one of the AGVs, and have cloggers dancing while the AGV rolls through the building," smiled the new overall project manager.

The cruel, hard lessons of Captain Composite were remembered. A stern letter was written to the company officials saying that what they did with their vehicles was their business, that the project management team wouldn't recommend it, and they would have nothing to do with it. The new overall project manager sulked for awhile and then went ahead with the show.

And for the next week while they worked on the system, gaily clad cloggers danced on the vehicle as it rolled majestically through the factory.