The Road To Production Quality

buttons on a "remote" is a lot more important than testing the "alternate input" button, simply because On and Off are used more frequently.

Beta Test Program and Easy Upgrades

If you can easily and quickly provide upgrades, perhaps even automatic upgrades, you can cover a multitude of problems. Microsoft's service packs are an example. Sure there are lots of problems to be fixed, but the fixes are easily applied, at least when they're not bundled with non-upward compatible functionality. By service pack 3 or so, the resulting product is solid.

Finally, work with your customer base to establish a good beta test period. Your beta period will expose your product to use cases you were unable to identify on your own. Start your beta period with an in-house alpha period, if applicable, so that you are the first guinea pig. Don't release anything to your customers, even for beta testing, if you're not happy with it.

To Sum Up

So, to sum up, here are my suggestions:

• Track and understand your problem fix and verification metrics
• Compare your product release to the previously available and or competitive releases.
• Work to increase your use case coverage
• Work with your customer when customization is a key component
• Automate upgrades
• Beta test plan

And make sure your CM tool environment is supporting these activities. It should provide you with problem tracking and test case management. It should provide you with problem arrival rate and fix metrics, and test run coverage. It should allow you to easily compare your new release to the one that the customer currently has, or to the one in production.

The road is long, with many a winding turn... and you'll always be second guessing your decisions. Be conservative or agressive. When you do release a product, just make sure that your product support team is ready for the challange. They are part of the product too.