Understanding the Logic of System Testing


fail testing status.

Concluding a Feature Fail Status
The feature fail criterion is defined as, "If any of the test cases fails, then the feature fails testing." According to the Modus Ponens rule, this means that each failed test case provides grounds for the valid conclusion that the feature has failed testing. As a feature can fail on more than one of its test cases, after finding the first defect a tester should continue feature testing and execute all of its test cases. After that, the tester should report all instances of the feature failure by submitting defect reports, where each defect report should be a valid argument that includes the evidence supporting the feature fail status.

On the other hand, if a given test case passed testing, the Modus Ponens rule does not apply, and there are no grounds for any conclusion at this point, i.e., the feature has neither passed nor failed testing. Finally, only when all of the feature’s test cases have been executed should it be decided whether there are grounds for the feature pass status as discussed in the next section.

Concluding a Feature Pass Status
Obviously, if the feature has already failed, the pass status cannot have grounds. However, if none of the test cases failed, then the Disjunctive Syllogism rule can be applied. According to this rule, the fact that none of the test cases failed provides grounds for a valid conclusion: the feature passed testing. To support this claim, evidence is provided–test case execution results. However, this conclusion should not be confused with the claim that the feature implementation has no bugs, which we know is impossible to prove. The conclusion means only that the feature did not fail on the executed test cases that were presented as evidence supporting the conclusion.


Copi, I., and C. Cohen. Introduction to Logic. 11th ed. Prentice-Hall, 2002.

  1. Bloch, E. Proof and Fundamentals. Boston: Birkhauser, 2000.
  2. Rodgers, N. Learning to Reason. An Introduction to Logic, Sets, and Relations. John Willey & Sons, 2000.
  3. Meyers, G. The Art of Software Testing. John Wiley & Sons, 1979.
  4. Kit, E. Software Testing In the Real World. Addison-Wesley, 1995.

The Sarbanes-Oxley Act .    

About the author

Yuri Chernak's picture Yuri Chernak

Yuri Chernak, Ph.D, is the president and principal consultant of Valley Forge Consulting, Inc. Yuri has worked for a number of major financial firms in New York, leading QA governance committees in IT and helping clients improve their software requirements and software testing practices. Yuri is a pioneer in implementing a new discipline—aspect-oriented requirements engineering—for financial applications on Wall Street. He is a member of the IEEE Computer Society, has been a speaker at several international conferences in the US and Canada, and has published papers in the IEEE publications and other professional journals. Contact Yuri at ychernak@yahoo.com.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!