Advocacy - Comment Letter
Subject: Docket No. FDA-2025-N-4203
Dear Sir/Madam:
The Association for Diagnostics & Laboratory Medicine (ADLM) supports the Food and Drug Administration’s (FDA’s) efforts to develop an appropriate regulatory framework for medical devices that utilize artificial intelligence (AI). We offer the following comments to assist the agency in this endeavor.
Real-world Performance of Clinical Laboratory Testing
Clinical laboratories across the United States annually report billions of test results critical to the diagnosis and treatment of patients. Laboratories implement robust quality management systems that monitor the accuracy and reproducibility of each test system to ensure these tests continually meet performance standards. While there are notable differences between laboratory testing and AI, the framework of the quality systems utilized for laboratory testing offer instructive lessons for AI oversight.
Clinical Laboratory Testing Quality Assurance and the Correlation to AI
Laboratory tests undergo a robust validation process to characterize test performance and ensure the laboratory test is safe for clinical use prior to use in patient care or deployment in a new clinical setting. The minimum requirements for FDA-authorized laboratory tests under CLIA are to:
Similarly, all AI tools that are integrated with the performance of clinical laboratory tests and have the capacity to influence the test result or its interpretation, must undergo a comparable evaluation. The specific details of the verification study may differ, however, based on the type of AI, how it is being used, its error modalities, and the risk of harm.
Under CLIA, laboratory tests are monitored to ensure ongoing performance using frequent evaluations of quality control (QC) specimens with pre-defined acceptance criteria (at least daily); periodic evaluations for shifts or drifts in quality control materials or patient test results; and a comparison of test results to other laboratories (i.e. external quality assessment). The corresponding need for monitoring AI tools will vary, depending on the potential sources of shifts and drifts over time, the frequency of use, and the magnitude of the potential harm.
External Quality Assessment of AI medical devices
An important part of laboratory quality systems is external evaluation. Within this structure, laboratories evaluate blinded specimens and send the results to a third-party that assesses the accuracy of the lab’s performance. This mode of assessment could be particularly helpful for AI tools that adapt over time and for tools for which it is cost-prohibitive to regularly evaluate against actual patient outcomes. We suggest that an ongoing external quality assessment system be utilized to ensure the accuracy of these results, similar to the proficiency testing program that exists under CLIA.
Validation and Regulation of AI Algorithms
ADLM suggests that AI algorithms used in the performance of laboratory testing be subject to validation or verification protocols established by the clinical laboratory to ensure acceptable performance regarding the intended use of the AI system. While such AI algorithms have important method-specific considerations, they fit well within the overall framework for laboratory quality systems and oversight. Like any clinical laboratory test system, the use of these algorithms beyond the validated protocols should be restricted.
ADLM recognizes several key challenges surrounding the validation and verification of AI algorithms in laboratory medicine, including:
ADLM believes validation requirements for laboratory AI systems should be generated through expert consensus of clinical laboratorians and laboratory informatics professionals to establish best practices for quality practices that ensure patient safety and efficacy. The guidance for validation and verification requirements of AI systems utilized in laboratory medicine should be FDA developed within a risk-based framework where the degree of oversight necessary is dependent on the AI tool type and is proportional to risk severity.
Additional discussion is warranted to better understand the boundaries of the existing FDA medical device framework, as it pertains to software functions, and laboratory developed testing services which include software functionality, which are legally exempt from FDA oversight.”
Oversight of AI tools for Ordering and Interpreting Laboratory Tests
Clinical laboratorians should be included in the oversight of AI algorithms used in ordering and interpreting laboratory tests. Laboratorians understand the nuances of the performance and reporting of clinical laboratory testing, which is essential to mitigating the introduction of variables that may cause deviation from the expected performance specifications.
For example, while laboratory test results are accurate, they are often not harmonized or equivalent across testing methods. This means that different medical devices measuring the same clinical analyte often generate different results. Clinically, these differences are considered and accounted for by the collaborative efforts of laboratorians and other healthcare providers. Outside of this framework, where AI systems currently operate, these differences (without mitigation efforts) may introduce biases in AI tools.
There are ongoing efforts to better harmonize laboratory test results and reporting—as was done with cholesterol for cardiovascular disease risk and hemoglobin A1c for diabetes—to ensure the algorithms are accurate and meaningful. The Centers for Disease Control and Prevention, with congressional support, has initiated this effort. However, in the current environment laboratory content expertise is essential to appropriately account for these differences in the development and monitoring of AI tools.
We look forward to working with you on this important issue. If you have any questions, please email Vince Stine, PhD, ADLM’s Senior Director of Government and Global Affairs, at [email protected], or Evan Fortman, MPA, ADLM’s Manager of Government Affairs at [email protected].
Sincerely,
Paul J. Jannetto, Ph.D., DABCC, FAACC
President, ADLM