The Road To Production Quality

My product is ready on CD. It's better than the previous release in respect to both quality and functionality perspectives. Does that mean it's ready to be released as a production product? How do I know when it's really ready?

This challenge faces every product, every release of every product. Ever hear of a rocket launch failing because of a software error? What about telephone service being interrupted by software problems? Yes on both accounts, but to be fair, the success rate in these two sectors has generally been good.

There are a number of factors to consider. How critical is the application? If it's a manned space mission launch, the application is pretty important. Almost perfect is not necessarily good enough. You'll want to be sure that when you release the software that every conceivable path of execution has been exercised. If not, production release, and hence the mission, simply has to be delayed.

A critical application could also provide a reason for releasing sooner. If I've cut a new telephone switch into operation, only to find that it's resetting twice a week, I've got some pretty big liabilities. If someone tells me that there's a better release available that has run through its verification tests in good shape with respect to the high priority problems, but needs work on some of the medium priority stuff, if I'm the telco, I'm going to say ship it. I can't afford to keep booting my customers off their phones every third day or so. I rather them find out that Call Display is occasionally not working, and find other annoyances that aren't as likely to result in a law suit. Although this might backfire if a higher quality competitive product is ready to roll.

So there are a number of factors that come into play. There's no simple answer, as it depends on the application and the state of the current production version.

Process Metrics
How can you make the best judgment for your products in your company? I recommend that you start by getting a handle on your process and the underlying metrics. Let's start with this line of questioning.

If you fix 100 more problems, how many of those fixes are going to fail to fix the problems? How many failures will you detect before release? Hopefully your verification process will help you to catch a very high percentage of the fix failures before they go out the door. But let's continue.

How many are going to break something else - that is, fix the problem but break another piece of functionality? How many of those failures will you detect before release? Not quite as high a percentage, I would imagine. Now let's continue this line of questioning.


About the author

Joe Farah's picture Joe Farah

Joe Farah is the President and CEO of Neuma Technology and is a regular contributor to the CM Journal. Prior to co-founding Neuma in 1990 and directing the development of CM+, Joe was Director of Software Architecture and Technology at Mitel, and in the 1970s a Development Manager at Nortel (Bell-Northern Research) where he developed the Program Library System (PLS) still heavily in use by Nortel's largest projects. A software developer since the late 1960s, Joe holds a B.A.Sc. degree in Engineering Science from the University of Toronto. You can contact Joe at

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!