Vol. 17 Issue 9
The Molecular Edge
Performance of Lab Developed Tests
Editor's note: This is the final of a two-part article. Part one, published in the July issue, examined validating the LDT.
Laboratory developed tests (LDTs) are a vital component of patient care, but key factorsassay performance and personnel competencymust be addressed to ensure safe and effective use.
A series of experiments are designed to establish assay performance characteristics. The requisite equipment and reagents must be assembled. Typical specimens include mock samples prepared to mimic the plausible range of input materials. A natural patient specimen also must be included to assure that expected results are obtained as judged against a legitimate gold standard comparator (such as another laboratory test or clinical diagnosis). Credible specimens are distributed in proficiency surveys among large numbers of testing laboratories. Discrepancies are investigated and procedures modified as needed to optimize performance.
Additionally, pre-analytic variables and administrative issues must be addressed (Table).
A validation report summarizes the findings of the validation study and serves as evidence of assay performance for clients, inspectors and laboratory colleagues. A frequent question is how many specimens must be tested to validate an assay. There is no right answer since so many variables affect this number; likewise, the minimum acceptable performance characteristics vary widely depending on the clinical situation.
For example, a reverse transcription polymerase chain reaction (rtPCR) assay that will be used to screen the blood supply for HIV needs to be nearly 100 percent sensitive since the danger of a false-negative result is so high, whereas an assay for Chlamydia trachomatis DNA is deemed quite useful even if it is only 90 percent sensitive since that is still better than the 65 percent sensitivity of culture-based methods.
It takes technical expertise and medical judgment to decide when an assay is ready for clinical implementation. The decision to move forward should be based on solid evidence that you would feel comfortable justifying to an inspector, laboratorian, clinician or patient. The laboratory director is responsible for vetting the validation process by overseeing the work and signing the validation report and procedure manual.
Training of personnel and competency assessment is part of the initial and ongoing oversight of any laboratory test. Once a test is introduced into clinical practice, proficiency tests monitor ongoing performance. Laboratory personnel must continue to monitor the medical literature for new data relating to validity of the assays on their menu. They also should be prepared to be inspected at any time and produce data supporting assay validity.
Consultation services must be provided on issues related to diagnosis and management of patients as well as weighing the risks and benefits to patients. The clinical consultant in a laboratory is typically a pathologist with experience in the relevant technologies and their medical applications.
Clinical utility is a process encompassing and extending beyond clinical validity to include such factors as cost-effectiveness and impact on the larger healthcare system. Utility is typically established after a test becomes used routinely. For example, quantitative rtPCR for BCR-ABL1 is now being used to help diagnose and monitor chronic myelogenous leukemia patients, although it is not clear if this new test will complement or replace fluorescence in situ hybridization (FISH) tests for BCR-ABL1.
Laboratory professionals have a long track record of successful assay validation under the regulatory oversight of the Centers for Medicare and Medicaid Services (CMS). The laws established under CLIA provide a strong framework for safe and effective laboratory practices, including assay validation. The inspection checklists provided freely by the College of American Pathologist's Laboratory Accreditation Program provide further guidance to help develop and implement new assays in a way that meets the high standards expected of us by our patients and clinician colleagues.
Dr. Gulley is director of Molecular Pathology at the University of North Carolina at Chapel Hill. She serves as chair of the Strategic Planning Committee in the Association for Molecular Pathology.
1. College of American Pathologists Inspection Checklists (www.cap.org)
2. Association for Molecular Pathology statement. Recommendations for in-house development and operation of molecular diagnostic tests. Am J Clin Pathol 1999;111:449-463.
3. Clinical and Laboratory Standards Institute (www.clsi.org)
4. American College of Medical Genetics Standards and Guidelines for Clinical Genetics Laboratories (www.acmg.net)
5. Forbes BA. Introducing a molecular test into the clinical microbiology laboratory: Development, evaluation and validation. Arch Pathol Lab Med 2003;127:1106.
6. Wolff DJ, et al. Guidance for fluorescence in situ hybridization testing in hematologic disorders. J Molec Diagn 2007;9:134.
7. Evaluation of Genomic Applications in Practice and Prevention (www.egappreviews.org)
8. ACCE (www.cdc.gov/genomics/gtesting/acce.htm)
9. U.S. System of Oversight of Genetic Testing: SACGHS's Draft Response to the Charge of the Secretary of HHS (www4.od.nih.gov/oba/sacghs/public_comments.htm).
Table: Issues to Address for Assay Performance
- acceptable specimen types
- patient preparation
- specimen collection
- criteria for rejection