AI-Initiated Recalls After Screening Mammography Demonstrate Higher PPV for Breast Cancer

Artificial intelligence (AI) may significantly bolster the effectiveness of recalls after women have undergone screening mammography exams, according to new research involving nearly 55,000 women.

For the study, recently published in Radiology, researchers compared mammography recalls and the positive predictive value (PPV) of those recalls for two reviewing radiologists and an AI computer-aided detection (CAD) software (Insight MMG AI version 1.1.6, Lunit). There were 1,348 recall cases and 5,489 cases flagged for consensus discussion that were drawn from 54,991 women (median age of 55) who had screening mammography, according to the study.

After consensus discussions, the researchers found that 14.8 percent of cases flagged by one radiologist (263/1,858) proceeded to recall in comparison to 4.6 percent of cases flagged by the AI CAD software (86/1,886). However, the study authors noted that AI CAD recalls had a positive predictive value (PPV) of 22 percent (19/86) for breast cancer in contrast to a 3.4 percent PPV for recalls (9/263) initiated by one radiologist.

Similarly, the researchers pointed out an 18.6 percent higher number of recalls when mammograms were flagged by two radiologists (57.2 percent, 360/629) in comparison to recalls initiated by one radiologist and the AI CAD software (38.6 percent, 244/632). Yet the study authors determined that PPV for the recalls by one radiologist and AI CAD was 90 percent higher than the recalls involving cases flagged by two radiologists (25 percent vs. 2.5 percent).

“The most important finding was that the consensus discussion recalled a smaller proportion of participants if their digital mammograms were initially flagged by AI CAD compared with flagged for suspicion of breast cancer by AI CAD versus by a radiologist … Of note, among the participants whom the consensus discussion decided to recall, the cancer yield was several times higher when the examinations had been initially flagged by AI CAD,” wrote lead study author Karin E. Dembrower, M.D., Ph.D., who is affiliated with the Department of Oncology-Pathology at the Karolinska Institute and the Department of Radiology at the Capio Sankt Gorans Hospital in Stockholm, Sweden, and colleagues.

In a subsequent sensitivity analysis of cases involving invasive breast cancer, the study authors saw similar results with a fourfold higher PPV for recalls from AI CAD alone (13 percent) versus recalls initiated by one radiologist (3.07 percent).

Comparing recalls for cases flagged by two radiologists in comparison to those initiated by one radiologist and AI CAD software, the study authors noted an eightfold higher PPV when AI CAD was involved in the recall decision (17.9 percent vs. 2.26 percent).

“This suggests a differential reliance on decision support related to whether that originated from AI CAD or from a fellow radiologist. The observed behavior may attenuate and underestimate the potential benefits of AI CAD in screening programs,” wrote lead study author Karin E. Dembrower, M.D., Ph.D., who is affiliated with the Department of Oncology-Pathology at the Karolinska Institute and the Department of Radiology at the Capio Sankt Gorans Hospital in Stockholm, Sweden,

(Editor’s note: For related content, see “Multicenter Mammography Study Shows Greater Than 10 Percent Increase in Breast Cancer Detection with Adjunctive AI,” “New Mammography Studies Assess Image-Based AI Risk Models and Breast Arterial Calcification Detection” and “Study: Mammography AI Leads to 29 Percent Increase in Breast Cancer Detection.”)

Beyond the inherent limitations of a single-center study, the authors acknowledged that the results with their two-radiologist consensus discussion may not be applicable to single-reader settings. The researchers also pointed out that more subtle signs on mammograms may be more of a factor with lower radiologist recalls as opposed to a lack of trust in AI CAD interpretation.