Technical refinements improve software’s sensitivity, but definitions of true and false remain tricky
Computer aided-detection can boost the accuracy of chest radiography for lung cancer, especially for some inexperienced readers.
Although chest CT is the de facto imaging standard for lung cancer diagnosis, many hospitals still rely on digital or analog chest radiography for such diagnoses. Their confidence may be easier to justify when the radiographs are read with the help of CAD, as demonstrated by a clinical trial by radiologist Dr. Dorith Shaham and colleagues at the Hadassah-Hebrew University Medical Center in Jerusalem. Results were presented at the 2007 RSNA meeting.
Shaham’s retrospective study involved 39 nodules greater than 5 mm that had been previously confirmed in 36 digital chest radiographs from health facilities in Europe and the U.S. Three radiologists with various levels of expertise-a resident, a general radiologist, and a thoracic radiologist-read the cases randomly and marked apparent lung masses and nodules greater than 5 mm. Readers relied on a prototype detection system that generated CAD marks for all DR images.
Using the CAD system, the general radiologist detected 23 nodules compared with 20 nodules without CAD. The thoracic radiologist detected 22 nodules compared with 20 nodules without CAD, and the resident detected 18 nodules with CAD assistance and 14 without CAD, a statistically significant increase of 10.3% ( p <0.046). The increased detection was inversely proportional to the expertise of the reader.
This study, however, did not assess the detection rate of true-positive CAD marks by each reader. A subsequent review of similar data that included this parameter found that CAD actually improved the detection rate of the thoracic radiologist to a greater extent, Shaham said.
“The more experienced reader benefited more from the use of CAD,” she said.
In a separate study, Feng Li, Ph.D. and colleagues at the University of Chicago sought to establish the true rate of detection of a digital radiography system versus the “accidental” markings that can be attributed to this technology when used in lung cancer detection.
They retrospectively applied CAD to 34 digital chest radiographs with malignant nodules not mentioned in their respective reports. They defined CAD markings as true detections only when their center was within the area of a lesion boundary. Detection was deemed accidental if the center point of the CAD marking was not within the lesion boundary, even if the lesion was located completely or partially within the 5-cm circle provided by the marking.
Based on these criteria, the investigators found the reported sensitivity of the CAD system can vary by as much as 100% depending on the precise definition of true versus false marks. Without refinement, CAD found 211 marks, with a sensitivity of only 35% and an average of 5.9 false positives per radiograph. By considering only lesions located completely within the circles, the sensitivity for cancer rose to 47% with 5.7 false-positive marks per image. And by assessing lesions located at least partially within the circles, the sensitivity increased to 59% with 5.6 false-positive marks per image. Specificity changed only marginally as lesions farther away from the center of the markings were included.
“In evaluating CAD systems, it is important to understand how their apparent accuracy can be influenced by the specific criteria that are used to determine sensitivity and specificity,” Li said.