A QA manager is often faced with measuring the impossible. Here is a simple, post-ship metric to help judge the test effort's effectiveness. The Quality Barometer method uses the bug counts found during testing, calculates a percentage, and then uses that percentage as the defect target number that can be tolerated after shipment.
First-order measurement can help you understand what's going on, make decisions, and improve results. Observation, modeling, and simple data gathering are things that you can implement in your work group without a big measurement program or big funding. Start by modeling your system and working out on paper how different measures will affect your system. Then involve your team, expand your model, and try some simple data gathering. This approach to measurement is one more tool in your toolkit, and it will move your organization toward better quality.
The theory underlying a measurement must take into account at least nine factors. This article defines these nine factors (e.g., the scope of the measurement, the scale of the instrument, and the variation of measurements made with the instrument) and applies them to a few examples.