| 1. | The work, responsibilities, and goals of the laboratory
|
| | Many factors affect the necessary elements and requirements of a quality management programme. These factors include:
|
| | i) | The type of testing done;
|
| | ii) | The use of the test results;
|
| | iii) | The impact of a questionable or erroneous result;
|
| | iv) | The tolerance level of risk and liability;
|
| | v) | Client needs (e.g. sensitivity and specificity of the test method, costs, turnaround time);
|
| | vi) | The role of the laboratory in legal work or in regulatory programmes;
|
| | vii) | The role of the laboratory in assisting with, confirming, and/or overseeing the work of other laboratories; and
|
| | viii) | The business goals of the laboratory, including the need for any third party recognition and/or accreditation.
|
| 2. | Standards, guides, and references
|
| | It is recommended that the laboratory choose reputable and accepted standards and guides to assist in designing the quality management programme. The OIE standard on this subject is a useful guideline (7). For laboratories seeking accreditation, the use of ISO/IEC 17025 (4) or the OIE standard (7) will be essential. Further information on standards may be obtained from the national standards body of each country, from the International Laboratory Accreditation Cooperation (ILAC), and from accreditation bodies (e.g. the National Association of Testing Authorities [NATA], Australia; the American Association for Laboratory Accreditation [A2LA], United States of America; the United Kingdom Accreditation Service [UKAS], United Kingdom; and the Standards Council of Canada [SCC], Canada). Technical and international organisations such as the AOAC International (formerly the Association of Official Analytical Chemists) and the ISO publish useful references, guides, and/or standards that supplement the general requirements of ISO/IEC 17025. ISO International Standard 9001 (5), a general standard for quality management systems and one of the many standards in the group commonly termed the 'ISO 9000 series', is not usable for accreditation, as conformity with its requirements does not necessarily ensure or imply technical competence (see Section 3. below). While a laboratory may implement a quality management system meeting the requirements of ISO 9001, registration or certification is used to indicate conformity with this standard. ISO/IEC 17025 contains a tabular comparison by clause numbers of itself, ISO 9001 (1994), and ISO 9002 (1994). Both of these latter documents have been replaced by the 2000 version of 9001 (5).
|
| 3. | Accreditation
|
| | If the laboratory has determined that it needs formal recognition of its quality management programme, then third party verification of its conformity with the selected standard(s) will be necessary. ILAC has published specific requirements and guides for laboratories and accreditation bodies. The terms accreditation, "a procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks" (2), and laboratory accreditation, "the formal recognition of the competence of a laboratory to carry out specific tests or specific types of tests" (2), are specific to ILAC requirements and to the use of ISO/IEC 17025 or the OIE standard as the basis for the accreditation. The use of competence is significant: it means much more than having and following documented procedures. Having competence also means that the laboratory:
|
| | i) | Has technically valid and validated test methods, procedures, and specifications that are documented in accordance with the requirements of the selected standard(s) and/or guidelines;
|
| | ii) | Has adequate qualified personnel who understand the science behind the procedures;
|
| | iii) | Has correct and adequate equipment;
|
| | iv) | Has adequate facilities and environmental control;
|
| | v) | Has procedures and specifications that ensure accurate and reliable results;
|
| | vi) | Can foresee technical needs and problems;
|
| | vii) | Can cope with and prevent technical problems that may arise;
|
| | viii) | Can accurately estimate and control the uncertainty in testing; and
|
| | ix) | Can demonstrate proficiency to conduct the test methods used.
|
| 4. | Selection of an accreditation body
|
| | In order for accreditation to facilitate the acceptance of the laboratory's test results for trade, the accreditation must be recognised by the international community. Therefore, the accreditation body should be recognised as competent to accredit laboratories. Programmes for the recognition of accreditation bodies are, in the ILAC scheme, based on the requirements of ISO/IEC International Standard 17011 (3). One may obtain information on recognised accreditation bodies from the organisations that recognise them, such as the National Cooperation for Laboratory Accreditation (NACLA), the Asia-Pacific Laboratory Accreditation Cooperation (APLAC), the Interamerican Accreditation Cooperation (IAAC), and the European Co-operation for Accreditation (EA).
|
| 5. | Determination of the scope of the quality management programme and/or of the laboratory's accreditation
|
| | The quality management programme should ideally cover all areas of activity affecting all testing that is done at the laboratory. However, for the purpose of accreditation, the laboratory should determine the scope of testing to be included in the accreditation. Factors that might affect the laboratory's choice of scope of accreditation include:
|
| | i) | The availability and cost of necessary personnel, facilities and equipment;
|
| | ii) | The cost of environmental monitoring against the possibility of cross contamination;
|
| | iii) | The deadline for accreditation;
|
| | iv) | The impact of the test results;
|
| | v) | The number of tests done;
|
| | vi) | Whether the testing done is routine or non-routine;
|
| | vii) | Whether any part of testing is subcontracted out;
|
| | viii) | The quality assurance necessary for materials, reagents and media;
|
| | ix) | The availability of reference standards (e.g. standardised reagents, internal quality control samples, reference cultures);
|
| | x) | The availability of proficiency testing;
|
| | xi) | The availability, from reputable sources, of standard and/or fully validated test methods;
|
| | xii) | The evaluation and validation of test methods to be done; and
|
| | xiii) | The technical complexity of the method (s).
|
| | Accreditation bodies also accredit the providers and operators of proficiency testing programmes, and may require the use of an accredited provider in order to issue the laboratory a certificate of accreditation.
|
| 6. | Test methods
|
| | ISO/IEC 17025 requires the use of appropriate test methods and has requirements for selection, development, and validation. The OIE document (7) also provides requirements for selection and validation.
|
| | This Terrestrial Manual provides recommendations on the selection of test methods for trade and diagnostic purposes in the chapters on specific diseases. In addition, a list of tests for international trade is provided. As stated in the introduction to this list, the prescribed tests that are listed are those that are required by the OIE Terrestrial Animal Health Code. These tests are considered to be adequately validated to give reliable results to qualify animals for international movement. Also listed are alternative tests that are suitable for the diagnosis of disease within a local setting, but that may have had limited validation. These tests are generally serological tests.
|
| | In the veterinary profession, other standard (methods published in international, regional, or national standards) or fully validated methods (methods having undergone a full collaborative study and that are published or issued by an authoritative technical body such as the AOAC International), while preferable to use, may not be available. Many veterinary laboratories develop or modify methods, and most of these laboratories have test programmes that use non-standard methods, or a combination of standard and non-standard methods. In veterinary laboratories, even with the use of standard methods, some in-house evaluation, optimisation, and/or validation generally must be done to ensure valid results.
|
| | Clients and laboratory staff must have a clear understanding of what performance can be expected from a test. Many veterinary testing laboratories will therefore need to demonstrate competence in the development, adaptation, and validation of test methods.
|
| | This Terrestrial Manual provides more detailed and specific guidance on test selection, optimisation, standardisation, and validation in Chapter I.1.3. The following items discuss test method issues that are of most interest to those involved in the quality management of the laboratory.
|
| | a) | Selection of the test method
|
| | | Valid results begin with the selection of an appropriate test method for the diagnostic issues at hand. Considerations for the selection of a test method include:
|
| | | i) | International acceptance;
|
| | | ii) | Scientific acceptance;
|
| | | iii) | Method not outdated;
|
| | | iv) | Performance characteristics (e.g. analytical and diagnostic sensitivity and specificity, repeatability, reproducibility, isolation rate, lower limit of detection, precision, trueness, and uncertainty);
|
| | | v) | Behaviour in species and population of interest;
|
| | | vi) | Resources and time available for development, adaptation, and/or evaluation;
|
| | | vii) | Performance time and turnaround time;
|
| | | viii) | Type of sample (e.g. serum, tissue) and its expected quality or state on arrival at the laboratory;
|
| | | ix) | Analyte (e.g. antibody, antigen);
|
| | | x) | Resources and technology of the laboratory;
|
| | | xi) | Nature of the intended use (e.g. export, import, surveillance, screening, confirmatory, individual animal use, herd use);
|
| | | xii) | Client expectations;
|
| | | xiii) | Safety factors;
|
| | | xiv) | Number of tests to be done;
|
| | | xv) | Cost of test, per sample;
|
| | | xvi) | Existence of reference standards, including reference materials; and
|
| | | xvi) | Availability of proficiency testing schemes.
|
| | b) | Optimisation and standardisation of the test method
|
| | | Once the method has been selected, it must be set up at the laboratory. Whether the method was developed in-house or imported from an outside source, generally some additional optimisation is necessary. Optimisation is a series of experiments and subsequent data analysis. Optimisation establishes critical specifications and performance standards for the test process and for use in monitoring the correct performance of the test. Optimisation should ensure that a method is brought under statistical control. Optimisation should also determine:
|
| | | i) | Critical specifications for equipment and instruments;
|
| | | ii) | Critical specifications for reagents (e.g. chemicals, biologicals);
|
| | | iii) | Critical specifications for reference standards, reference materials, and internal controls;
|
| | | iv) | Robustness (if applicable);
|
| | | v) | Critical control points and acceptable ranges, attributes or behaviour at critical control points, using statistically acceptable procedures;
|
| | | vi) | The quality control activities necessary to monitor critical control points;
|
| | | vii) | The type, number, range, frequency, and/or arrangement of test run controls needed;
|
| | | viii) | The requirements for control behaviour for the non-subjective acceptance or rejection of test results;
|
| | | ix) | The elements of a fixed, documented test method for use by laboratory staff; and
|
| | | x) | The level of technical competence required of those who carry out and/or interpret the test.
|
| | c) | Validation of the test method
|
| | | Validation further evaluates the test for its fitness for a given use. Validation establishes performance characteristics for the test method, such as sensitivity, specificity, and isolation rate; and diagnostic parameters such as positive/negative cut-off, and titre of interest or significance. Validation should be done using an optimised, documented, and fixed procedure. Depending on logistical and risk factors, validation may involve any number of activities and amount of data, with subsequent data analysis using appropriate statistics. Test validation is covered in Chapter I.1.3. Principles of validation of diagnostic assays for infectious diseases, and Chapter I.1.4. Validation and quality control of polymerase chain reaction methods used for the diagnosis of infectious diseases.
|
| | | Validation activities might include:
|
| | | i) | Field and/or epidemiological studies;
|
| | | ii) | Comparison with other methods, preferably standard methods;
|
| | | iii) | Comparison with reference standards (if available);
|
| | | iv) | Collaborative studies with other laboratories using the same documented method, and including the exchange of samples, preferably of undisclosed composition or titre. It is preferable that these be issued by a qualified conducting laboratory that organises the study and evaluates the results provided by the participants;
|
| | | v) | Reproduction of data from an accepted standard method, or from a reputable publication;
|
| | | vi) | Experimental infection studies; and
|
| | | vii) | Analysis of internal quality control data.
|
| | | ISO/IEC 17025 recognises that "Validation is always a balance between costs, risks, and technical possibilities." It also recognises that there are many cases in which quantities such as accuracy and precision can only be given in a simplified way.
|
| | d) | Uncertainty
|
| | | Laboratories should be able to estimate the uncertainty of the test methods as performed in the laboratory. This includes methods used by the laboratory to calibrate equipment (4). ISO/IEC 17025 requirements concerning this topic are stated in sections 5.1.2, 5.4.4k, 5.4.6.2, 5.4.6.3 and 5.10.3.1c of that document.
|
| | | The determination of measurement uncertainty (MU) is not new to some areas of measurement sciences. However, the application of the principles of MU to laboratories for the life sciences is new. Most of the work to date regarding MU is for areas of testing other than the life sciences, and much of the work has been theoretical. However, as accreditation becomes more important, applications are being developed for the other areas. It is important to note that MU does not imply doubt about the validity of a test result or other measurement, nor is it equivalent to error, as it may be applied to all test results derived from a particular procedure. It may be viewed as a quantitative expression of reliability, and is commonly expressed as a number after a +/- sign (i.e. the true value lies within the stated range, as MU is expressed as a range). Standard deviation and confidence interval are examples of the expression of MU. An example of the use of standard deviation to express uncertainty is the allowed limits on the test run controls for an enzyme-linked immunosorbent assay, commonly expressed as +/- n SD.
|
| | | Although the determination and expression of MU has not been standardised for veterinary testing laboratories (or medical, food, or environmental), some sound guidance exists.
|
| | | MU must be estimated in the laboratory for each method included in the scope of accreditation. This can be estimated by a series of tests on control samples. MU can also be estimated using published characteristics (6), but the laboratory must first demonstrate acceptable performance with the method. Government agencies may also set goals for MU values for official methods (e.g. Health Canada). Reputable technical organisations, including accreditation bodies (e.g. AOAC International, ISO, NATA, A2LA, SCC, UKAS, Eurachem, and the Co-Operation on International Traceability in Analytical Chemistry [CITAC]) teach courses and/or provide guidance on MU for laboratories seeking accreditation. Codex Alimentarius, which specifies standards for food testing, has taken the approach that it is not necessary for a laboratory to take a further estimate of MU if the laboratory complies with Codex principles regarding quality: i.e. that the laboratory is accredited to ISO/IEC 17025, and therefore uses validated methods (e.g. knows applicable parameters such as sensitivity and specificity, as well as the confidence interval around such parameters), participates in proficiency testing programmes and collaborative studies, and uses appropriate internal quality control procedures.
|
| | | The requirement for "use of appropriate internal quality control procedures" implies that the laboratory must use quality control procedures that cover all major sources of uncertainty. There is no requirement to cover each component separately. Components can be estimated with experiments in the laboratory (Type A estimates), or from other sources (reference materials, calibration certificates, etc.) (Type B estimates). A traditional control sample procedure, run many times by all analysts and over all shifts, usually covers all the major sources of uncertainty except perhaps sample preparation. The variation of the control samples can be used as an estimate of those combined sources of uncertainty.
|
| | | ISO/IEC 17025 requires the laboratory to identify all major sources of uncertainty, and to obtain reliable estimates of MU. Laboratories may wish to establish acceptable specifications, criteria, and/or ranges at critical control points for each component. Where appropriate, laboratories can implement appropriate quality control at the critical points associated with each source, or seek to reduce the size of a component. Sources of uncertainty include sampling, storage conditions, sample effects, extraction and recovery, reagent quality, reference material purity, volumetric manipulations, environmental conditions, contamination, equipment effects, analyst or operator bias, and other unknown or random effects. The laboratory would also be expected to account for any known systematic error. (See also Section 6.b. steps i-vii.) Systematic errors (bias) must either be corrected by changes in the method, adjusted mathematically, or have the bias noted in the report statement. If an adjustment is made to the procedure, there may or may not be a need to reassess uncertainty. If there is an adjustment made to correct for bias, then a new source of uncertainty is introduced (the uncertainty of the correction). This must be added to the MU estimate.
|
| | | There are three principal ways to estimate MU:
|
| | | 1. | The components approach (or 'bottom-up' approach), where all the sources of uncertainty are identified, reasonable estimates are made for each component, a mathematical model is developed that links the components, and the variations are combined using rules for the propagation of error (1).
|
| | | 2. | The control sample approach (or 'top-down' approach), where measurements on a stable control material are used to estimate the combined variation of many components. Variation from additional sources needs to be added.
|
| | | 3. | The method characteristics approach, where performance data from a valid collaborative study are used as combined uncertainties (other sources may need to be added). Laboratories must meet defined criteria for bias and repeatability for the MU estimates to be valid. These should be larger than would be obtained by competent laboratories using their own control samples or components model.
|
| | e) | Implementation and use of the test method
|
| | | Analysts should be able to demonstrate proficiency in using the test method prior to producing reported results, and on an ongoing basis.
|
| | | The laboratory should ensure, using appropriate and documented project management, records keeping, data management, and archiving procedures, that it can recreate at need all events relating to test selection, development, optimisation, standardisation, validation, implementation, and use. This includes quality control and quality assurance activities.
|
| 7. | Strategic planning
|
| | Continuous improvement is essential. It is recommended that the laboratory be knowledgeable of and stay current with the standards and methods used to demonstrate laboratory competence and to establish and maintain technical validity. The methods by which this may be accomplished include:
|
| | i) | Attendance at conferences;
|
| | ii) | Participation in local and international organisations;
|
| | iii) | Participation in writing national and international standards (e.g. participation on ILAC and ISO committees);
|
| | iv) | Consulting publications;
|
| | v) | Visits to other laboratories;
|
| | vi) | Conducting research;
|
| | vii) | Participation in cooperative programmes (e.g. Inter-American Institute for Cooperation in Agriculture);
|
| | vii) | Exchange of procedures, methods, reagents, samples, personnel, and ideas; and
|
| | ix) | Wherever possible, accreditation by a third party that is itself recognised as competent to issue the accreditation.
|
| 1. | American National Standards Institute (1997). ANSI/NCSL Z540-2-1997, US Guide to the Expression of Uncertainty in Measurement, First Edition. American National Standards Institute, 1819 L Street, NW, Washington, DC 20036, USA.
|
| 2. | ISO/IEC International Standard 17000 (2005). Conformity Assessment - General Vocabulary. International Organisation for Standardisation (ISO), ISO Central Secretariat, 1 rue de Varembé, Case Postale 56, CH - 1211, Geneva 20, Switzerland. (ISO/IEC International Standard 17000 replaces ISO/IEC Guide 2 (1996). Standardisation and Related Activities - General Vocabulary.)
|
| 3. | ISO/IEC International Standard 17011 (2003). General Requirements for Bodies Providing Assessment and Accreditation of Conformity Assessment Bodies- General Requirements for Operation and Recognition. International Organisation for Standardisation (ISO), ISO Central Secretariat, 1 rue de Varembé, Case Postale 56, CH - 1211, Geneva 20, Switzerland. (ISO/IEC International Standard 17011 replaces ISO/IEC Guide 58 (1993). Calibration and Testing Laboratory Accreditation Systems - General Requirements for Operation and Recognition.)
|
| 4. | ISO/IEC International Standard 17025 (1999). General Requirements for the Competence of Testing and Calibration Laboratories. International Organisation for Standardisation (ISO), ISO Central Secretariat, 1 rue de Varembé, Case Postale 56, CH - 1211, Geneva 20, Switzerland.
|
| 5. | ISO International Standard 9001 (2000). Quality management systems - Requirements. International Organization for Standardization (ISO), ISO Central Secretariat, 1 rue de Varembé, Case Postale 56, CH - 1211, Geneva 20, Switzerland.
|
| 6. | ISO DTS 21748 (draft). Guide to the use of repeatability, reproducibility and trueness estimates in measurement uncertainty estimation. International Organisation for Standardisation (ISO), ISO Central Secretariat, 1 rue de Varembé, Case Postale 56, CH - 1211, Geneva 20, Switzerland.
|
| 7. | Office International des Epizooties (2000). Standard for Management and Technical Requirements for Laboratories Conducting Tests for Infectious Animal Diseases. In: OIE Quality Standard and Guidelines for Veterinary Laboratories: Infectious Diseases (2002). Office International des Epizooties (OIE), 12 rue de Prony, 75017 Paris, France, 1-31.
|