- it is operating in accordance with ISO/IEC 17025, ie it has the necessary basic management and technical controls in place to enable it to produce reliable test results
- demonstrate to software developers, suppliers and users that it has the necessary systems in place to produce reliable test results
- demonstrate that it not only has systems which enable it to perform effective testing, but that it is sufficiently confident in its procedures that it has been prepared to have them scrutinised by an independent third party accreditation body
- support marketing initiatives by software developers and suppliers to enhance the market perception of software products
- increase confidence of purchasers and users in the reliability of the software they purchase
- complement and support general software quality initiatives and life cycle models
7. The assessment process
Accreditation of software testing facilities provides management with increased confidence in testing, reduces risk resulting from inadequate testing, supports due diligence initiatives, and increases the quality of the delivered product. Some of this comes from the knowledge that the laboratory has undergone the process of assessment against ISO/IEC 17025.
Assessment generally comprises an initial “advisory” visit to the laboratory by the accreditation body‘s staff officer. The purpose of this is to gain an understanding of the test facility’s operation, to inform the laboratory about the assessment process and to identify any obvious, major, deficiencies in its compliance with the standard which need to be addressed . This visit can occur either before or aft er a desktop review of the laboratory’s management system documentation. The laboratory’s documentation is reviewed against the requirements of ISO/IEC 17025. Much of this review will be done prior to the formal on-site assessment. Implementation of documented procedures will be confirmed during the formal on-site assessment.
The initial on-site assessment will take place after the laboratory has addressed any issues identified during the advisory visit or the review of documentation. It would typically take several days depending on the scope of the test facility’s operation. It is done by a team comprising a staff officer from the laboratory accreditation body and usually at least two technical assessors. The staff officer’s role is to coordinate the assessment and to audit the management requirements. The role of the technical assessors is to review staff and procedures against the technical requirements. This is somewhat of a simplification as, in reality, the roles do have some overlaps, eg both will need to examine test records and reports.
The technical assessors are drawn from test laboratories, industry, regulatory bodies, academia, etc. They are peers of the staff of the laboratory being reviewed, in the sense that they are involved in similar activities. For example, a recent assessment by NATA was performed by a team of three comprising the NATA staff officer, a generalist software “tester” from another laboratory and someone from the regulatory body involved, who also had current “hands-on” testing experience.
Some readers will no doubt react negatively to the thought of being assessed by someone from another test facility. The usual response is “They’re a competitor!” or “They’ll find out all our secrets”. The NATA staff officer’s role is to ensure that assessors do not access commercially sensitive information. Usually such information is irrelevant to the assessment process. Also assessors are subject to confidentiality agreements. In extreme cases procedures can be implemented to ensure that assessors do not take away any hard or soft copy information from the laboratory premises. NATA has been using this approach successfully for over fifty years. Care is taken to minimize potential problems. However laboratories find that the benefits of discussing testing