Home > Laboratory Administration & Management of Pathology Practices > Validating a new quantitative assay

Laboratory Administration & Management of Pathology Practices

Quality management

Validating a new quantitative assay


Editorial Board Member: Lewis A. Hassell, M.D.
Deputy Editor-in-Chief: Patricia Tsang, M.D., M.B.A.
Felix Leung, Ph.D.

Last author update: 24 February 2023
Last staff update: 24 February 2023

Copyright: 2022-2024, PathologyOutlines.com, Inc.

PubMed Search: Validating a new assay

Felix Leung, Ph.D.
Table of Contents
Definition / general | Essential features | Terminology | Overview | Precision | Bias and accuracy | Linearity | Reference interval | Other studies | Analytical validation tools | Board review style question #1 | Board review style answer #1 | Board review style question #2 | Board review style answer #2
Cite this page: Leung F. Validating a new quantitative assay. PathologyOutlines.com website. https://www.pathologyoutlines.com/topic/managementlabvalidatingnewassay.html. Accessed December 25th, 2024.
Definition / general
  • Proper analytical validation of a new assay is essential prior to patient testing
  • Using Clinical Laboratory Standards Institute (CLSI) guidelines as a framework for designing new assay validations is highly recommended
Essential features
  • Minimum requirements for analytical validation of a quantitative assay include studies for precision, bias, accuracy, linearity and reference interval
  • Required validation studies each have an associated CLSI standard which outlines how to design, carry out and evaluate the specific analytical aspect (CLSI: Method Evaluation Standards [Accessed 1 December 2022])
  • Statistical software, such as EP Evaluator and Analyse-it, are useful for analyzing the validation data
Terminology
  • Total allowable error (TAE): the amount of error that is allowable without invalidating the interpretation of a test result
  • Precision: closeness of agreement between values obtained by replicate measurements
  • Bias: estimate of a systematic measurement error
  • Linearity: ability to provide results that are directly proportional to the concentration / amount of analyte in the test sample within a given range
Overview
  • Precision, bias, accuracy, linearity and reference interval studies are typically the minimum requirements
  • Establish performance limits for validation studies a priori using appropriate total allowable error goals
  • Factors to consider for each validation study:
    • Sample: what type of samples to use?
    • Target values: what concentrations are relevant?
    • N (number): how many samples / replicates are needed?
    • Limits: what are the performance limits?
Precision
  • Precision studies establish variance within replicate measurements and the likelihood / magnitude of random errors
  • Refer to CLSI EP05-A3 for designing precision studies:
    • Sample: quality control (QC) material, optimally from a third party vendor
    • Target values: at least 2 levels of QC near medical decision limit(s)
    • N: 20 x 2 x 2 format consisting of 2 runs of duplicate measurements daily for 20 days for each level of QC
    • Limits: total observed variance should not typically exceed 33% of the total allowable error
  • For the 20 x 2 x 2 format, CLSI recommends calculating the coefficient of variation (%CV) via 2 way nested analysis of variance (ANOVA) (see Analytical validation tools below)
  • Reference: CLSI: EP05-A3 - Evaluation of Precision of Quantitative Measurement Procedures [Accessed 1 December 2022]
Bias and accuracy
  • Bias studies establish agreement to a predicate assay and the magnitude of systematic error
  • Refer to CLSI EP09-A3 for designing bias studies:
    • Sample: patient samples with known values as reference standard for comparison
    • Target values: preferably an even distribution across the analytical measuring range (AMR)
    • N: minimum of 40 patient sample singleton measurements, preferably tested over several days
    • Limits: mean bias (absolute or relative) should not typically exceed half of the total allowable error
  • Additional analyses to perform for bias studies:
    • Regression analyses are typically performed to assess absolute or proportional bias in the form of a regression line
    • Differences (absolute or relative) are typically plotted on Bland-Altman plots to assess bias trends across the analytical measuring range
    • Consider inclusion of higher metrological order samples such as proficiency testing materials or certified reference material to establish bias against a reference method
  • Reference: CLSI: EP09-A3 - Measurement Procedure Comparison and Bias Estimation Using Patient Samples [Accessed 1 December 2022]
Linearity
  • Linearity studies verify vendor quoted analytical measuring range and reportable limits to be implemented with the assay
  • Refer to CLSI EP06 for designing linearity studies:
    • Sample: patient derived sample pools preferable (see below for preparation steps)
    • Target values: preferably equally spaced concentrations across the analytical measuring range bounded by sample pools at or near the low and high limits
    • N: minimum of 5 sample pools, each tested in duplicate
    • Limits: various schema are used to evaluate linearity, though CLSI recommends bias limits be used for linearity
  • Linearity samples are ideally matrix matched, patient derived sample pools
    • Typical preparation involves serial dilutions of a high sample pool with a low sample pool
    • If patient derived sample pools are not attainable, consider alternatives:
      • Vendor provided material (linearity, calibrator, quality control, etc.)
      • High sample pool diluted with vendor diluent
  • Reference: CLSI: EP06 - Evaluation of Linearity of Quantitative Measurement Procedures [Accessed 1 December 2022]
Reference interval
Other studies
Analytical validation tools
  • Statistical programs / software are useful tools to aid the analyses and calculations required for analytical validation studies based on CLSI standards
    • EP Evaluator (Data Innovations LLC) is a standalone software
    • Analyse-it (Analyse-it Software, Ltd.) is an Excel add on software
Board review style question #1


Using the provided dataset for precision studies for a hemoglobin A1c assay, which of the following statements is true?

  1. A positive shift in both levels is observed on day 14
  2. Level 1 quality control shows more consistency and stable performance compared to level 2
  3. Quality control material is inappropriate for precision studies
  4. The total allowable error of 6% is too stringent and should be widened
Board review style answer #1
A. A positive shift in both levels is observed on day 14. Overall, precision studies show adequate performance for both levels of quality control material in the 20 x 2 x 2 format and are acceptable. Level 2 quality control shows a smaller CV and more consistency than level 1. A total allowable error of 6% is too wide as a precision goal for hemoglobin a1c, which has relatively small absolute values.

Comment Here

Reference: Validating a new quantitative assay
Board review style question #2


Using the provided dataset for method comparison studies for a hemoglobin A1c assay, which of the following statements is true?

  1. The data shows acceptable linearity
  2. The result suggests analytical errors due to carryover
  3. There is an increasing negative bias of the new method with increasing HbA1c concentrations
  4. There is an insufficient number of data points for the analysis to be conclusive
Board review style answer #2
C. While the average bias of -1.5% is within the total allowable error, it is clear that there is an increasing negative bias of the new method with increasing HbA1c concentrations (current method is negatively biased against new method). The bias hovers around 6% starting around 6.0% A1c units and greater, which encompasses the medical decision limit of 6.5% A1c units. This bias renders the method comparison result not acceptable for patient testing.

Comment Here

Reference: Validating a new quantitative assay

Back to top
Home > Laboratory Administration & Management of Pathology Practices > Validating a new quantitative assay
Image 01 Image 02