unacceptable levels. Transitioning to model-based testing required that the modeling team process a significant inventory of existing specifications and product announcements, resolving ambiguities in pre-existing functionality as well as new functions and rules for the pending releases.
Again, there is a clear and rapid trend toward high software reliability when the model-based testing processes are applied. Defects not only declined significantly, but major defects such as invalid configurations were eliminated entirely.
Implementing Model-Based Testing
Before you make the leap into model-based testing, be prepared to address significant obstacles. The introduction of formal software engineering practices into general business enterprises creates cultural and institutional impediments and affects the roles of test engineers.
Ironically, the most difficult implementation issues often fall within the QA organization. Many traditional QA professionals are deeply suspicious of test cases designed by software tools, believing that the test cases they manually design are superior to those produced using software algorithms. Time and time again, we've come across QA professionals who are reluctant to accept a new approach to test design.
If this is the case at your organization, you should be prepared to counter resistance with facts. Several case studies have been run to compare the functional variation coverage (the quantitative coverage metric for stuck-at-one test algorithms, which are used in Caliber-RBT). As expected, the automated test design provides 100 percent functional coverage. Manually designed test cases generally provide 60 to70 percent coverage.
Adopting model-based testing practices may also require significant procedural changes, staffing adjustments, and training efforts since the process demands a different skill set and new team interactions. For example, test engineers must be competent at creating complex logic models-a very different mental process from writing test cases. The model-based test process is less concerned about "breaking" the code than it is about ensuring complete functional understanding in the specifications. The test engineer must now focus on accurately modeling the requirements rather than attempting to conceive of test cases that will find defects. Test engineers must also separate the process of test case design from those of test case implementation and execution. Traditionally, these tasks are blurred or run together, creating the illusion that traditional test approaches take less time.
Model-based testing forces the testing group to be more directly involved in the requirements process and also opens testing products (the generated test scripts) to other groups on the team. This is often seen as contrary to the traditional view of testing as an independent and relatively isolated process.
Perhaps the most important prerequisite to the implementation of model-based testing is to elevate the discussion of software testing to the senior management level-the place where the repercussions of software failure are eventually felt most. An organization that accepts the dramatic risks of software failure will better embrace the processes for reducing risks. When this happens, software testing can transition from the stepchild of software development into the significant role it deserves within an organization.