e-Talk Radio: Paulk, Mark, 28 November 2000

organizations.

Back in 1991 there was a guide, ISO 9000, Part 3, that was written for interpreting ISO 9001, which is the international standard for quality management systems. ISO 9001 is used in a number of different environments, manufacturing, services, et cetera. And it has a section, a clause, on statistical techniques. The guide for interpreting that in the software arena had the section entitled "Metrics."

Carol: Right.

Mark: So instead of statistical techniques, we'll just talk about measuring something. And then in the text it talked about the fact that in the software field, at least at that time, we don't really know how to measure quality. And so if you measure something, if you measure anything, you will in essence satisfy this particular clause of ISO 9001. And in fact that was a -- probably a reasonable way of capturing the state of the practice in measurement back in the 1880s -- 1980s -- excuse me -- time frame. And so we have as a community, grown in our use of measurement thanks to folks such as yourself, Dick …….., Norm Fenton and a whole bunch of other measurement gurus around the world who have given us better and better insights about how to do measurement effectively. But it's been a slow process. And it's been growing over the years.

Carol: And the sampling that you took in 1990 with 90 percent Level 1, have some of those people dropped off? Has your sample size increased? Today's sample, when you say that we've got more than 50 percent that have moved beyond kind of utter chaos, is that still the same samples you had back then, or have we had people added on and people dropped off? What are the demographics? Have you seen any changes in the sample size?

Mark: Well that certainly has been one of the fascinating things about how the process movement in the software world has grown over the past decade or so. In 1990 when we were talking about process improvement, we were fundamentally talking to government contractors, to DoD contractors. The SEI is a federally funded research and development center. We're sponsored out of the U.S. Department of Defense. Our sponsor is the Under Secretary of Defense for Acquisition Technology and Logistics, so we have a primary focus on the military and the military contracting communities. And so the initial adopters of our ideas were in essence the DoD contracting community. And back at that time, I don't recall the exact numbers, but there were probably about a hundred organizations represented in the first set of data points that we looked at back in 1990 that were 90 percent Level 1. Today when we look at it, we're talking at an order of magnitude, more organizations. Some time ago, about three or four years ago, the percentage of DoD and DoD contracting organizations dropped below 50 percent of the organizations reporting data to the SEI.

Carol: Right.

Mark: The majority are now banks, insurance agencies, aerospace companies, telecommunications, companies that are building software intensive systems. And those are the ones who are picking up the ideas that we captured in the CMM and has moved far beyond the DoD contracting community.

Back in 1990 time frame, when we had a workshop for folks interested in software process improvement, we got at our first one about 30 to 40 folks come into that workshop. There were about 50 or 60 at the next one. There were about 80 at the third one. At the fourth one, there were 411 people -

Carol: