Laboratory administration
Quality
External quality assessment and proficiency testing

Author: Jacqueline Hall, Ph.D.

Revised: 20 November 2017, last major update November 2017

Copyright: (c) 2002-2017, PathologyOutlines.com, Inc.

PubMed Search: External quality assessment and proficiency testing [title]

Cite this page: Hall, J. External quality assessment and proficiency testing . PathologyOutlines.com website. http://pathologyoutlines.com/topic/qualityassessmentproficiencytesting.html. Accessed January 23rd, 2018.
Essential features
  • External quality assessment (EQA) is a system for objectively checking laboratory performance using an external agency or facility
  • Proficiency testing (PT) is one method for achieving this
    • PT is educational and promotes continuous learning; it supports laboratories in implementing new biomarkers into clinical testing as well as maintaining quality for existing biomarkers
    • PT fulfills a different function from laboratory accreditation although the two are complementary; participation in PT is usually required for laboratory accreditation
Quality in pathology matters
  • Clinical implementation of precision medicine means that biomarkers and new technologies for their detection are increasingly used in pathology
  • Biomarkers assist in diagnosis and inform patient treatment as modern targeted therapies require highly accurate biomarkers to identify which patients should receive which treatment
  • Methods may include:
  • Rigorous quality management using internationally recognized standards is mandatory
  • In addition to the potential impact on patients, errors can be costly to the healthcare system, leading to the retesting and incorrect use of medical resources
    • For example, several studies have examined the economic burden of inaccurate estrogen receptor (ER) testing; data suggests that errors in ER status occurred in approximately 12,025 patients in the U.S. during 2012, costing nearly $1 billion (Value Health 2015;18:541)
Adopting new biomarkers
What is external quality assessment and proficiency testing?
  • External quality assessment (EQA) is a practical tool for reviewing the performance of biomarkers and peer comparison of biomarker testing in diagnostic laboratories
  • It allows laboratories to obtain an independent external assessment of their current process
  • EQA is a way for laboratories to compare their results to an external source
  • Another name, often used interchangeably, is proficiency testing (PT); however, PT is only one method of EQA
  • World Health Organisation (WHO) defines EQA as a system for objectively checking the laboratory's performance, using an external agency or facility; they identify three common processes:
    1. Proficiency testing (PT): an external provider sends unknown samples for testing to a set of laboratories and the results of all laboratories are analyzed, compared and reported to the laboratories
    2. Rechecking or retesting: slides that have been read are rechecked by a reference laboratory; samples that have been analyzed are retested, allowing for interlaboratory comparison
    3. On site evaluation: usually done when it is difficult to conduct traditional proficiency testing or to use the rechecking / retesting method
How does PT work?
  • PT can be arranged in different ways and can differ on how the test results are compared and scored
  • For most of the biomarkers, laboratories report the results to the organization that provides PT challenges
  • Results are compared to either designated "true value" or "consensus score / results"
    • Reasons for using one or the other method in evaluation of laboratory performance may be complex and often depend on the type of the assay used for the biomarker
      • Different approaches may be used for qualitative markers for which reference standards are well defined and readily available (e.g. blood glucose testing)
      • Descriptive methodologies would require designated descriptive true value for which no reference standards could be developed and do not exist (e.g. ER and PR testing in breast cancer)
  • Although not possible for all biomarkers, it is highly desirable for PT organizations to use a known, qualified reference sample that is very similar to clinical samples
    • This is provided by the external third party for their challenges
      • Since this is often not available, a second best option is artificial samples ; for example ctDNA mutation testing using synthetically prepared DNA spiked into plasma
    • These reference samples are first qualified by the PT provider to confirm their biomarker status, then are sent to the participating laboratory for testing
    • Laboratories run these samples through their normal diagnostic process, as they would standard clinical samples
    • Organizations providing PT usually report to the participant whether the laboratory passed or failed the challenge
    • They may or may not provide feedback to the laboratory on possible technical issues that could have influenced the suboptimal or poor results or suggestions for improvements
  • It is also important to consider what or how the readout of the test is performed
    • If the readout process is automated (e.g. glucose testing), there will be no need and no possibility for central review
      • Participating laboratories will send their results to the PT provider
    • If the readout is interpretive, for example done by pathologists (e.g. ER and PR for breast cancer), the IHC results include both elements of the slide staining and the pathologist's readout
    • Slides may be sent to the PT provider without the local pathologist's readout thereby only assessing the slide staining component
    • Laboratories could also report the pathologist's readout results only without sending the slides
    • They also could send the slides back and provide the local pathologist's readout
    • Therefore, both protocol performance (the quality of IHC slide staining) and the readout performance (the quality of pathologist's readout) impact final results of IHC testing or both could be the subject of PT evaluation
  • In practical terms, using IHC biomarkers as an example, PT is usually performed in two different ways:
    1. PT with central sample review
      • IHC slides are reviewed in the participating laboratory and the results are reported to the PT provider
      • When the IHC slides are sent back to PT provider, the PT provider performs a central review of the slides
      • This central review is usually done by experienced pathologists and technologists who reach a consensus “true” result
      • Independent providers verify the results from the laboratories by comparison to the validated “true” central review result and feed back discrepancies to the laboratory
    2. PT with consensus scoring
      • Similar to above, except the PT provider does not perform a central review
      • Laboratories run the test slides and perform the readout locally
      • Laboratories do not need to return the IHC slides to the PT provider since no central review will be performed
      • PT providers use a process to determine the consensus score and designate this as the designated correct result or truth set
      • This method is less objective and can occasionally be problematic, especially when many laboratories return an incorrect result
      • In this approach, the laboratories gain insight on how similar they are to other participants: designated as "concordance"
  • WHO recently released guidelines for national EQA schemes - laboratories are highly encouraged to join EQA schemes that follow the WHO best practice guidelines (WHO: WHO Manual for Organizing a National External Quality Assessment Programme for Health Laboratories and Other Testing Sites [Accessed 5 November 2017])
  • Benefits of participating in PT
    • PT programs support laboratories by providing valuable information (WHO: Overview of External Quality Assessment (EQA) [Accessed 5 November 2017]) which:
      • Allows laboratory self checking and the chance to take appropriate corrective action, if needed
      • Comparison of performance to other laboratories
      • Provides early warning system for possible problems with tests, processes or operations
      • Provides insights into test results among different test sites
      • Facilitates continuous improvement and can highlight areas needing attention
      • Helps identify training needs
    • PT helps to assure the recipients of the test results (such as doctors, patients and health authorities) that the laboratory can produce reliable test results
    • Data collected from PT could be very helpful in evaluation of:
      • Performance of protocol methods
      • Materials and equipment
      • Where applicable (e.g. IHC and FISH), performance of pathologists' or technologists' readouts
      • Impact of training
    • Additionally, PT participation creates:
      • Network for communication
      • Forum for discussion of quality issues
      • Source for conducting continuing education activities
    • Organizations that provide PT can also:
      • Engage in providing their participating laboratories more clarity
      • Provide support during a time of new assay validation
      • Offer verification prior to offering results in a clinical setting
    EQA is educational and promotes continuous learning
    • Quality assessment is a critical aspect of laboratory quality management
      • Laboratories which hold an accreditation such as CAP or ISO 15189 are required to participate in an appropriate PT, where available
      • Passing PT is not a replacement for laboratory accreditation as PT and accreditation fulfill different functions
        • Laboratories test thousands of samples per week, every week of the year
        • Continuous followup of laboratory performance over time is needed as many factors can influence laboratory operations:
          • New reagents
          • New methods
          • Change in staff etc.
        • In this regard, PT is different from accreditation because it is a continuous process, whereas accreditation reflects an assessment of standards in laboratories at a specific time
    • PT is mandatory in some countries and not in others
      • This may change as there is an increasing awareness of the benefits of participating in PT
      • Even in jurisdictions that do not mandate PT, laboratories should consider voluntary participation and join PT programs offered locally, nationally or internationally for the biomarker tests they offer because of the many benefits gained from the participation
    • Different PT providers offer a range of programs and laboratories can now easily participate in PT programs from other countries if local providers do not cover the biomarkers of interest
    • To easily find information about different PT providers, visit the IQN Path webpage
    Getting involved in promoting quality
    • PT providers often need expert pathologists, laboratory technologists and scientists to participate as PT contributors
    • Ways to contribute:
      • Provide samples for testing
      • Participate in expert readout assessments (e.g. IHC biomarkers)
      • Provide feedback to PT provider or inquire about development of challenges for new biomarkers
      • Help analyze reported results or help prepare summary reports of different challenges
    • Many important discussions occur at the results assessment stage (the definition of what best practice should be when faced with the range of practicalities), providing learning opportunities for volunteers
    • PT providers are positioned to review the results of different laboratories and compare the results to employed methodologies; this enables PT providers to learn about possible technical issues not previously recognized
    • Not for profit PT providers amass a large amount of expertise
    • Several PT providers aim to publish results of PT challenges when results provide new evidence that can help improve patient safety and provide scientific basis for future developments
    Board review question #1
    What are the 3 main advantages of laboratory participation in EQA?

    1. EQA provides a mechanism for laboratory self checking
    2. EQA is an opportunity for continuous learning by testing laboratories
    3. EQA provides a substitute for accreditation
    4. EQA provides a substitute to establishing a laboratory quality management system
    5. EQA provides support during a time of new assay validation and verification prior to offering a test in a clinical setting
    Board review answer #1
    A. B. and E. are correct; the three main advantages of laboratory participation are:

    • EQA provides a mechanism for laboratory self checking
    • EEQA is an opportunity for continuous learning by testing laboratories
    • EQA provides support during a time of new assay validation and verification prior to offering a test in a clinical setting