Comparative Study Evaluates AI Products for Detecting Tuberculosis on Chest X-Rays

Five out of 12 artificial intelligence (AI) products demonstrated higher than an 86 percent area under the receiver operating characteristic curve (AUC) for detecting tuberculosis (TB) on X-rays in a South African patient population with a high TB and human immunodeficiency virus (HIV) burden.

For the comparative study, recently published in the Lancet Digital Health, researchers assessed 12 computer-aided detection (CAD) software products for diagnosing TB on chest radiographs in 774 study participants, including 258 people who had bacteriologically positive TB. Twenty-six percent of the cohort had a history of TB and 18 percent of the study participants were HIV-positive, according to the study.

Insight CXR (Lunit) and Nexus CXR (Nexus) demonstrated the highest AUCs (90.2 percent and 89.7 percent respectively) for TB detection followed by qXR (Qure.ai) at 87.8 percent, JF CXR-2 (JF Healthcare) at 86.5 percent and ChestEye (Oxipit) at 86.1 percent.

Assessing the AI software modalities at 90 percent sensitivity thresholds and 70 percent specificity thresholds, the study authors noted similar results. At approximately 90 percent sensitivity, Insight CXR and Nexus CXR offered the highest specificity rates at 67.7 percent and 67.1 percent respectively. JF CXR-2 (62.7 percent), qXR (62.3 percent) and ChestEye (61.3 percent) had the next highest specificity rates at 90 percent sensitivity.

The researchers added that Insight CXR and Nexus CXR exhibited the highest sensitivity rates (89.5 percent and 88.8 percent) at the 70 percent threshold for specificity. They were followed by qXR (86.8 percent), JF-CXR-2 (86.4 percent) and ChestEye (86 percent), according to the study authors.

“Lunit, Nexus, JF CXR-2, and qXR maintained high sensitivity over a wider range of thresholds, resulting in more individuals being triaged by X-ray,” wrote lead study author Zhi Zhen Qin, MSc, who is affiliated with the Stop TB Partnership in Geneva, Switzerland, and the Department of Infectious Disease and Tropical Medicine at the Heidelberg University Hospital in Heidelberg, Germany, and colleagues.

While pointing out that HIV can impact how abnormalities are interpreted on chest radiographs, the researchers noted lower AUCs for all the reviewed AI products in this patient population but no statistically significant differences in comparison to those without HIV. However, they did notice key differences with respect to age.

“Sensitivity was statistically significantly higher in people aged 15–34 years and people aged 35–54 years than in people older than 55 years. Specificity was statistically significantly higher in people aged 15–34 years than in people aged 35–54 years and people older than 55 years,” pointed out Qin and colleagues.

(Editor’s note: For related content, see “Adjunctive AI Powered Tool for Tuberculosis Detection Gets FDA’s Breakthrough Device Designation,” “Study: Deep Learning System is Comparable to Radiologist Assessment of Chest X-Rays for Tuberculosis” and “Study Finds Four Out of Seven AI Algorithms Offer Better Lung Nodule Detection on X-Rays than Radiologists.”)

In regard to study limitations, the authors acknowledged possible inaccuracies with sub-analysis findings due to patient self-reporting of demographic, lifestyle, and clinical data. The researchers acknowledged that inclusion in the cohort was limited to those with symptoms of tuberculosis and abnormal X-rays. They also cautioned against extrapolation of the study findings to children as the study excluded patients under 15 years of age.