Review By: Robin F. Goldsmith
06/23/2010In "Best Practices for the Formal Software Testing Process," Rodger Drabick presents a highly structured set of narrative-format checklists for tasks and subtasks in a formal system testing process, where separate organizational functions communicate largely via respective written plans. Each identically formatted section of the book describes tasks in terms of successively decomposed Input-Processing-Output (IPO) diagrams.

The book is structured like a set of nested menus based upon a series of IPO diagrams. Level 1's diagram describes five sub-processes: 1.1 Extract Test Information from Program Plans; 1.2 Create Test Plans; 1.3 Create Test Design, Test Cases, Test SW, and Test Procedures; 1.4 Perform Formal Test; and 1.5 Update Test Documentation. For each sub-process, Drabick describes inputs, immediate feedback, feedback from earlier process steps, process flow, outputs, and feedback from later process steps. He devotes a separate chapter with identically structured descriptions of each of the sub-sub-processes (e.g., 1.1.1-1.1.4). He also diagrams the next level down for each (e.g., 1.1.1.1-1.1.1.5) but does not repeat the input-feedback narrative descriptions. Appendix C repeats these in a checklist questionnaire format.

Drabick's formal world involves documented requirements, program management plans, software development plans, software quality assurance plans, configuration management plans, and test plans, each prepared by separate specialized organizational units. A number of the tasks involve pre- and post-testing meetings with these groups, reviewing documents, preparing testing reports for the others to review, and updating the testing documentation upon completion. Appendix B provides templates for each of these documents. Chapter 8 describes ways Drabick feels the formal tasks could be tailored to less formal test environments, such as by skipping or consolidating particular tasks. Other appendices briefly summarize the CMM method and some test execution practices.

The book's structure is its greatest strength and its greatest weakness. The identically formatted, nested IPOs will be helpful for those seeking a reference checklist of all the tasks and subtasks to do in a formal testing process. On the other hand, this orderliness means that every section looks alike. I counted twelve single-level and thirty-five multi-level IPO diagrams, each with the same set of six input-feedback narrative sections. It became a big blur. Nothing stood out as important, and I quickly lost any sense of where I was in the overall structure. The similar-appearing but uncoordinated separate numbering schemes for chapter sections and IPO diagrams further obscured any sense of "You are here."

The book is not an outline of how to do any of the prescribed tasks. Only the process flow sections give even a hint of what testers actually may encounter carrying out these numerous tasks. I sort of doubt there are a lot of people seeking such checklists, except those who already do accept and follow formal (and probably well-documented) processes. I seriously doubt even they would read through the full narrative to extract the list of tasks; and it's highly unlikely other typical testers would read through to Chapter 8's somewhat naïve suggestions for tailoring the numerous tasks to less formal environments.