New research yields mixed results in a comparison of artificial intelligence (AI)-only false positives and false positives with radiologist evaluations for digital breast tomosynthesis (DBT) exams.
For the retrospective study, recently published in the American Journal of Roentgenology, researchers compared false positives with radiologist evaluations to AI-only false positive findings with Transpara v1.7.1 (Screenpoint Medical) in a review of data from 2,977 women (mean age of 58) who had a total of 3,183 DBT screening exams.
While the study authors noted the same false positive rate of 10 percent for AI and unassisted radiologist interpretation of DBT screening exams, they found that AI-only false-positive exams yielded a nearly threefold higher total of false-positive findings (932) in comparison to radiologist false-positive assessments (315).
As the researchers pointed out, 40 percent of the AI-only false-positive findings were benign calcifications whereas the most common radiologist false-positive interpretations involved mass presentations (47 percent).
“This mismatch between AI output and radiologist judgment may slow workflows and increase interpretive burden, especially when AI flags multiple nonactionable findings on a single examination. AI in this current form may hinder rather than help screening performance. Future algorithm improvements — including reducing superfluous markings and incorporating prior examinations to allow assessment of stability —are critical for making AI tools more clinically translatable and beneficial in practice,” noted lead study author Tara Shahrvini, M.D., MBA, an internal medicine resident at the David Geffen School of Medicine at the University of California, Los Angeles (UCLA), and colleagues.
Could a concordance of AI and radiologist findings rein in unnecessary recalls? The study authors noted a 13 percent overlap of false positives with AI and radiologist assessment. However, out of the 71 overlap exams with false-positive DBT findings, the researchers noted that only 46 exams had at least one concordant finding. That said, 39 percent of those concordant finding cases proceeded to biopsy and 44 percent of those biopsies yielded high-risk lesions, according to the researchers.
“These findings suggest that concordant flagged false-positive findings may represent more clinically meaningful false-positive results, signifying an enriched subset of actionable abnormalities. … Although this estimate is exploratory based on findings identified retrospectively as concordant, it supports future research evaluating whether selectively focusing on concordant findings — rather than acting promptly on all AI- and radiologist-flagged findings — could reduce unnecessary recalls without compromising diagnostic yield,” suggested Shahrvini and colleagues.
Three Key Takeaways
- AI-only findings may inflate false positives. While AI and radiologists had the same overall false-positive exam rate (10 percent), AI generated nearly three times more individual false-positive findings, often from benign calcifications, which may slow workflow and increase interpretive burden.
- Concordant false positives may be more clinically meaningful. Overlap between AI and radiologist false positives was limited (13 percent), but concordant cases showed higher biopsy rates and a substantial proportion of high-risk lesions, suggesting potential value in focusing on shared findings to reduce unnecessary recalls.
- AI performed differently in women with dense breasts. AI-only false positives were less common in women with dense breasts (24 percent vs. 57 percent for radiologists), indicating AI may better distinguish overlapping tissue from suspicious findings in this population.
The researchers also found that AI-only false positives occurred in 24 percent of women with dense breasts in contrast to 57 percent for radiologist-only false positives.
“AI-only false positives were less commonly associated with dense breasts, suggesting that AI may better differentiate overlapping dense tissue from suspicious findings,” posited Shahvrini and colleagues.
(Editor’s note: For related content, see “Study Shows Higher Recall PPV and Lower False-Positive Recall Rate with Combination of DBT and Synthesized Mammography,” “Large Mammography Study Shows Mixed Results with AI in Breast Cancer Screening” and “Digital Breast Tomosynthesis Study Assesses Impact of Architectural Distortion on Malignancy Rates.”)
Beyond the inherent limitations of a single-center retrospective study, the authors acknowledged the use of one mammography vendor and one AI tool. They also conceded a lack of consecutive case assessment and lack of evaluation for interobserver agreement.
