Can AI enhance a general radiologist’s detection of breast cancer on mammograms to an equivalent level of breast imaging specialists?
For the prospective study, recently published in Radiology, researchers evaluated the use of a multi-stage AI-powered workflow (ProFound Pro version 2.x, DeepHealth) for mammography interpretation by 60 general radiologists (median of 19 years of experience) and 35 breast imaging specialists (median of 11 years of experience). The study authors also noted the use of a Safeguard Review that flags non-recalled exams identified as suspicious by AI to subsequent assessment by breast imaging experts.
The researchers found that the cancer detection rate (CDR) for general radiologists rose from a standard-of-care baseline of 3.76 per 1000 exams to 4.99 per 1000 with AI, a CDR that was statistically similar to that of breast imaging specialists with AI (4.76 per 1000 exams).
"(The) findings highlight the potential clinical utility of this AI-driven workflow to deliver standardized outcomes, which could extend specialist-level care to the approximately 70% of women whose screening mammograms are interpreted by general radiologists, thereby mitigating the current shortfall of breast imaging specialists," wrote lead study author Matthew P. McCabe, PhD, a clinical data scientist at DeepHealth, and colleagues.
Can AI Improve the PPV of Mammography Recalls for General Radiologists?
While the study authors noted a 14.79 percent increase in recall rate with the use of AI for general radiologists (increasing from 9.06 to 10.4 percent), they also pointed out a statistically significant 15.09 percent increase in the positive predictive value (PPV) of recalls with AI (from 3.38 to 3.89 percent).
“(The) PPV of recalls offers a lens through which to assess the overall effectiveness, directly quantifying the yield of true cancers per recall and weighing the “cost” of recalls against cancer detection benefits. Our results show an improved PPV of recalls for general radiologists to a level comparable to that of specialists when using the AI workflow. These findings suggest the AI-driven workflow may yield a greater proportion of true cancer detections, thereby enhancing the screening benefit,” added McCabe and colleagues.
The study authors noted that the use of the AI-driven workflow showed no statistically significant difference in the CDR and recall PPV for breast imaging specialists.
Three Key Takeaways
• AI may help close the performance gap between general radiologists and breast imaging specialists. With the AI-driven workflow, general radiologists' cancer detection rate rose from 3.76 to 4.99 per 1,000 exams, statistically comparable to breast imaging specialists' AI-assisted performance (4.76 per 1,000). This suggests such tools could extend specialist-level screening quality to the roughly 70 percent of mammograms currently read by general radiologists.
• The AI workflow improved recall quality, not just recall quantity. While recall rates rose modestly (9.06 percent to 10.4 percent), the positive predictive value of those recalls also increased significantly (3.38 percent to 3.89 percent), meaning a greater proportion of recalled patients actually had cancer. This addresses a common concern that AI-assisted screening might simply drive more unnecessary callbacks.
• Benefits appear concentrated among general radiologists, not breast imaging specialists. The AI workflow showed no significant change in CDR or recall PPV for breast imaging specialists, suggesting its clinical value lies specifically in standardizing performance for non-specialist readers with potential downstream implications for health equity in populations more likely to be screened by general radiologists.
Health Equity in Mammography Screening: Can AI Have an Impact?
In an accompanying editorial, Simone Schiaffino, MD, and Andrea Cozzi, MD, raised concerns about the efficiency of the Safeguard Review utilized in the study, they noted the potential of the study findings in possibly helping to address health equity challenges with access to breast imaging specialists.
“In a real-world unabridged implementation, the AI-driven workflow raised the screening performance of generalist radiologists to specialist-equivalent levels, with a favorable shift in the yield of recalls and a plausible equity dividend for the Medicare-eligible and racially diverse populations served,” noted Dr. Schiaffino, the leader of the breast imaging group of the Centro di Senologia della Svizzera Italiano at Ente Ospedaliero Cantonale in Lugano, Switzerland, and Dr. Cozzi, a radiology resident and postdoctoral researcher at the aforementioned institution.
Final Notes
(Editor’s note: For related content, see “Mammogram Interpretation and AI Automation Bias: What New Research Reveals,” “Mammography Study Shows Dynamic Changes of AI Risk Scores Years Before Breast Cancer Diagnosis” and “Mammography News: FDA Clears New Functionalities for DeepHealth’s AI-Powered Breast Suite.”)
In regard to study limitations, the study authors acknowledged that reviewing radiologists were not blinded to parts of the AI-enabled workflow and suggested that awareness of the Safeguard Review may have influenced recall rates for the reviewing radiologists. The researchers also noted the lack of assessment of interval cancer and false-negative rates due to unavailable data.
