“We’re still wrapping our heads around how we validate these systems, how we get them implemented and deployed,” says committee chair Matthew Hanna, MD, vice chair of pathology informatics at the University of Pittsburgh Medical Center. “In the same vein, we all know we need to be monitoring these systems over time.”

Not Long ago, Dr. Hanna and a number of coauthors wrote in a CAP concept paper that the evidence available to develop formal guidelnes for evaluating model performance was lacking (Hanna MG, et al. Arch Pathol Lab Med. 2024;148[10]:e335-e361). At the time, he recalls, he and his coauthors felt the number of peer-reviewed clinical validation studies on machine learning applications in use in clinical practice wasn’t sufficient.