- consistent with the original results. Clearly if these conditions cannot be met the test is, at best, doubtful. Repeatability and reproducibility is sometimes thought to be an outcome from all the other characteristics of a quality test, but should not be treated as the indicator of quality software testing as it is theoretically possible to achieve repeatability and reproducibility with an invalid test.
- Impartiality and objectivity. Test results must be impartial, ie not biased towards any particular result. Also the results must not be colored by the feelings or opinions of the testers. It is possible for a test to be inherently partial by having a bias toward a particular outcome; this should have been detected during its validation and is one reason for doing validation. Of course tests can be biased by the testers due to pressure applied by the client, or by management.
In the case of tests performed within a development organization there is a need for management to allow the testing to be performed in a way which minimizes any tester bias. In this situation management needs to ensure that it is receiving effective testing and therefore a true picture of the quality of the product. It needs to ensure that it is not inadvertently applying pressure to the test staff which results in less than ideal test outcomes. The use of development staff to test a product which they have developed is to be avoided as it is very difficult to ensure effective testing in these circumstances.
The achievement of testing which displays all of the above characteristics requires the application of considerable management and technical effort. This will not occur by chance. Quality testing cannot occur just by good luck; it does require good management. Reliable test results require a combination of an appropriate management system and technically competent personnel using appropriately validated test methods,
4. How do you ensure reliable test results and quality testing?
The answer to how quality testing is achieved is simply that you perform the tests in a testing laboratory which meets the requirements of ISO/IEC 17025, General requirements for the competence of testing and calibration laboratories. This international standard specifies general requirements for the competence of testing laboratories regardless of the technical discipline in which they operate. It has been adopted as an Australian Standard, AS ISO/IEC 17025, without change.
This standard reflects the lessons learned from fifty five years of international experience in the operation of many thousands of testing laboratories in many technical disciplines. Operation of a testing facility in accordance with this standard ensures that the testing characteristics described above are being addressed.
A common initial reaction from IT industry personnel is “Software testing is different!” or “There is no commonality between software testing and the more traditional test disciplines such as chemical testing, mechanical testing or electrical testing!”. It may be a surprise to many to learn that the similarities far outweigh the differences. Any differences lie mainly in the technical test techniques applied. On the surface there are significant differences between software testing and chemical testing, for example, or between electrical testing and microbiological testing, yet ISO/IEC 17025 still applies, and works.
ISO/IEC 17025 works in the various areas of testing because it is a high level document which does not specify the detail of discipline specific test procedures; rather it defines a generic infrastructure which is essential for all testing regardless of discipline. It provides for specific interpretation in the various technical disciplines. As a result of differing needs between disciplines some aspects may have stronger impact in some disciplines than others,