The Road To Production Quality

My product is ready on CD. It's better than the previous release in respect to both quality and functionality perspectives. Does that mean it's ready to be released as a production product? How do I know when it's really ready?

This challenge faces every product, every release of every product. Ever hear of a rocket launch failing because of a software error? What about telephone service being interrupted by software problems? Yes on both accounts, but to be fair, the success rate in these two sectors has generally been good.

There are a number of factors to consider. How critical is the application? If it's a manned space mission launch, the application is pretty important. Almost perfect is not necessarily good enough. You'll want to be sure that when you release the software that every conceivable path of execution has been exercised. If not, production release, and hence the mission, simply has to be delayed.

A critical application could also provide a reason for releasing sooner. If I've cut a new telephone switch into operation, only to find that it's resetting twice a week, I've got some pretty big liabilities. If someone tells me that there's a better release available that has run through its verification tests in good shape with respect to the high priority problems, but needs work on some of the medium priority stuff, if I'm the telco, I'm going to say ship it. I can't afford to keep booting my customers off their phones every third day or so. I rather them find out that Call Display is occasionally not working, and find other annoyances that aren't as likely to result in a law suit. Although this might backfire if a higher quality competitive product is ready to roll.

So there are a number of factors that come into play. There's no simple answer, as it depends on the application and the state of the current production version.

Process Metrics
How can you make the best judgment for your products in your company? I recommend that you start by getting a handle on your process and the underlying metrics. Let's start with this line of questioning.

If you fix 100 more problems, how many of those fixes are going to fail to fix the problems? How many failures will you detect before release? Hopefully your verification process will help you to catch a very high percentage of the fix failures before they go out the door. But let's continue.

How many are going to break something else - that is, fix the problem but break another piece of functionality? How many of those failures will you detect before release? Not quite as high a percentage, I would imagine. Now let's continue this line of questioning.


About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.