What is interesting about our industry's cavalier approach to quality is the astounding level of consumer tolerance for "bugs" within software applications. Such errors simply would not be tolerated in products created in other industries. This willingness of the software consumer to overlook imperfection is due, in large part, to the extraordinary level of innovation traditionally energizing the software market. Recent efforts to bring a new level of quality assurance to software manufacturing have resulted in legislative junkets that have, not surprisingly, been killed by strong opposition from industry giants. These same abandoned attempts at quality legislation have also failed to rouse the ire of the average software consumer. Bugs are accepted, even expected, when they exist in products that become obsolete just as soon as further innovations are discovered. Even popular software products with a wide public acceptance go through many iterations within a short time.
A New Industry
The software business is still so new that it has not yet come to maturity as an industry. Consumer computing is only two decades old—a span similar to that which separated World War I aviation from the Wright Brothers' first flight—and it changes at a rate rarely seen in industrial sectors. Consumers willing to forgive imperfections in exchange for innovative products fuel the change, and keep the industry stalled in a creative cycle that is both flawed and nonproductive. As a result, the software industry has failed to establish a production process that converts human intelligence into computer instructions with the same efficiency as electronics companies produce televisions or appliance companies produce refrigerators.
Many parallels can be drawn between early twentieth-century industry and today's software industry. Only now is the software industry undergoing the preconsumer attention to quality that the automotive industry underwent before World War I, or that the consumer electronics industry (televisions and automated consumer appliances such as refrigerators and washing machines) struggled through just after World War II. Before, the hapless buyer of a Model T could expect any number of breakdowns, from the engine to the upholstery. Consumers accepted the situation due to the lack of choice presented them. They accepted lower quality because industrial manufacturing techniques, while flawed, produced goods that they could afford and that improved their lives, and that were still new and innovative. The same was true following World War II, as enthusiastic consumers lacking market choice purchased inefficient refrigerators and snowy televisions—not because these products worked well, but because they were readily available, relatively new technology, and affordable.
Today, the same situation burdens the software sector. The imperfect software that provides computers with intelligence and functionality is purchased because it is affordable and readily available, and has enjoyed the "forgiveness" that comes with being a relatively new technology. However, while other industries mastered the production process—bringing to it automatic error prevention techniques as a means of creating higher quality products with more affordability and greater abundance—there are those who doubt that the same methods can be applied to the software industry. To them, the necessary rapid evolution of software suggests that a stable production process, an assembly line model, is unattainable within the software industry. Where is the production line when you produce one thing only one time? How do you ensure quality when one product is as different from the next as a television is different from a refrigerator?The Software Production Line
Solving this quandary requires rethinking the concept of a "production line." There is indeed a production line in software manufacturing, but not one that repeatedly makes multiple copies of the same thing. The software production line exists in the transition of raw ideas into software products that provide intelligence to computers. Producing software does not require a traditional rigid production line-one of machines making machines—but necessitates a sophisticated and flexible framework capable of adapting to the many variances encountered from one piece of software to the next. In software, intelligence creating intelligent tools is, in and of itself, a production process.
The industry lost many years of productivity believing that there is no way to produce software in the same way other durable goods are manufactured. But with the innovation of various software development processes (such as waterfall and iterative methods) and with the introduction of certain quality control methods (such as ISO 9001 and SEI CMM certification), many in the industry began believing in the possibility of developing software in a controlled production process. Still, the problems with these approaches are threefold: 1) they are costly to implement, 2) they create vast amounts of documentation to simply certify that a company follows a written procedure for writing software, and 3) they require enormous amounts of human labor to maintain and verify. Even so, the processes do not guarantee that the end product is error free, but only that the human elements involved in the process are regulated in the hopes of reducing inefficiency. Thus, the "benefits" of development certification are only lost time and money wasted.
But these certification processes do point to the solution despite their bureaucratic shell. SEI CMM Level 5, the highest level of certification possible in that model, maintains that error prevention is the pinnacle of the software development process. I believe that this is backwards: Error prevention—the processes and practices of preventing bugs before they occur—must be the foundation of any development process rather than the pinnacle. Without error prevention (more specifically, without automatic error prevention), bugs can increase to the point that a project is so overburdened that it must be completely overhauled or abandoned altogether.
Elements of Error Prevention
The methodology of error prevention encompasses each of the software development techniques pictured below in figure 1. The techniques are equally important, and each of them will fit well in any development process. The more techniques implemented in a project, the closer that project will come to achieving a comprehensive and coherent error prevention program.
Figure 1. Error prevention comprises many software development techniques
But manually implementing these software development techniques does little to ensure their effectiveness. You see, a great chasm exists between simply pointing at key requirements, as many certification processes do, and actually implementing them as a coherent solution. What's more, certain development methodologies (unit testing, for example) require a great deal of manual labor to implement. For the software industry, automatic error prevention can reduce necessary labor and simultaneously incorporate error prevention within a number of development methodologies. The automation of error prevention can solve the problem of quality in the software industry.
Effectively automating error prevention techniques, however, is no easy task. Software testing is not the same thing as running microchips or toasters through a preprogrammed tester at a certain point on an assembly line. Because each software product created is different, automatic error prevention tools in the software industry must be more sophisticated than their traditional production line counterparts. Effective automatic error prevention tools must adapt to any number of test subjects, and to be truly automatic, they must do so without human intervention. To achieve such versatility and automation, automatic error prevention tools must be able to recognize just what they are testing and to determine how to carry out the test. In short, automatic error prevention tools must be intelligent themselves.
Developing tools with such intelligence is more complicated, by several orders of magnitude, than simply preprogramming static error prevention equipment on a traditional assembly line. The difficulty in building automatic error prevention tools may explain why quality has historically been absent from the software creation process. Such tools simply did not exist, and without them automatic error prevention seemed expensive and difficult, if not impossible to achieve.Intelligent Tools
The good news is that these difficulties have finally been overcome, and those concerned with quality can stop talking of automatic error prevention and begin implementing it. Within the last three or four years, new tools have emerged that are intelligent enough to exercise a new piece of software—to bend it, break it, and subject it to realistic and comprehensive test environments—and that are flexible enough to extend the same sort of testing to any number of software products.
Costs and Benefits of Unit Testing
Automatic error prevention requires development practices that are extremely difficult to execute without huge expenditures of manual labor and time. Consider unit testing. Unit testing isolates errors by testing small units of code as they are created, instead of testing finished applications. This strategy has the advantage of errors that are easy to find because the amount of code under test is significantly smaller, leading to further advantages of increased test coverage and the overall number of errors fixed.
Unfortunately, these advantages are costly. When performed manually, unit testing requires a huge investment in time and labor to:
- Design and build test harnesses that will run the class (Note: The harness itself must be thoroughly tested and declared bug-free, so as not to sabotage the source code)
- Design and create stubs that simulate external resources referenced by the class under test
- Design and build appropriate test cases that will test as much of the code as possible. Test cases should include white box test cases (to test application construction) and black box test cases (to test application functionality)
- Perform regression testing to ensure no new errors were introduced into the code by modifications or corrections
The difficulties inherent in executing these steps illustrate why unit testing is an underutilized feature of software development. The full benefit of unit testing can only be achieved by automating the process.
Automating Coding Standards
Coding standards is another effective development technique that is underutilized because of the difficulty of implementing it manually. Without automation, there is little that can be done to enforce coding standards within a development group—some developers will consistently break standard rules by using individual coding styles, and others will simply forget to check their code against standards before checking it into a source repository. The only way to ensure that a development group uses coding standards in a consistent manner is to use a static analysis tool that automatically checks their code against industry accepted coding practices, as well as those of an individual company, group, or other coding rules. Despite the availability of tools that automate these difficult tasks, automatic error prevention does not enjoy total acceptance. This advance is still considered so complex and unrealistic that many in the software industry have not seriously evaluated it. This is unfortunate—these tools are available and they must be utilized in order to completely cure the crippling industry-wide "bugs are the way it is" mentality.A Call for Change
Automatic error prevention must become a rallying cry to bring software development to the same level of consumer confidence and quality assurance enjoyed in traditional consumer industries. The recent advances in automatic error prevention are revolutionary, and will revolutionize companies that embrace the technology. At this point in the evolution of software development, to fail to embrace automated error prevention technology may very well undermine a business's growth and longevity. This is an industry that must no longer simply accept glitches and defects, but must, and can, do everything to prevent them.
