Breast CAD succeeds where human readers may falter

Despite the increasing availability of commercial computer-assisted detection packages for breast screening, radiologists have yet to agree on their relevance. Studies examining the use of breast CAD in this arena should focus on the technology's realistic potential, rather than expecting the impossible, according to speakers at CARS 2005.

Human observers should outclass computers as diagnostic readers, given their combination of speed and intuition, said Prof. Dr. Ulrich Bick, a professor of radiology at Charite, Humboldt University of Berlin. The flesh-and-blood radiologist falters, however, in reproducibility and thoroughness.

"If you show the same images to a radiologist on several occasions, he or she may come to a different judgment," Bick said. "They may have a bad day and miss a lesion that they spotted the day before. This is not true for the computer. It is never tired; it will perform at the same rate every day."

Installing a CAD package will not correct every human error, he said. If a human observer has already spotted a malignant lesion but has interpreted it as benign, a CAD prompt is unlikely to reverse that decision. Alternatively, a computer may fail to flag a cancer that the radiologist has noted. The danger is that radiologists might question their own judgment and accept the computer's incorrect analysis.

"The computer only really helps in situations where the radiologist does not initially detect the cancer," Bick said.

Introduction of CAD should be accompanied by appropriate training so that readers' performance does improve, Bick said. Without training, technological skeptics may simply disregard every prompt, regardless of their likely accuracy. At the other extreme, less experienced readers may use CAD as a crutch and accept every mark at face value.

Many radiologists are confused by published data on the value of CAD in breast screening, said Dr. Robert Nishikawa, an associate professor of radiology at the University of Chicago. This is hardly surprising, given that two high-profile studies of breast CAD in screening reported apparently conflicting results. A prospective analysis of 12,000 mammograms first brought cheer to the breast CAD camp by showing that the software boosted breast cancer detection by almost 20%.¹ This study was followed by a retrospective analysis of more than 115,000 screening mammograms that concluded that CAD software made little difference to radiologists' performance.²

Although radiologists in the first study claimed to have found far more cancers with CAD, it is unlikely that software would generate such a dramatic improvement in daily practice, Nishikawa said. Many of the "missed" cancers in that study were calcifications, but radiologists are far more likely to overlook masses. Participants in the study made an initial diagnosis from films alone, then looked again at the study with CAD marks and made a second diagnosis. It is possible they paid less attention to calcifications on the first read because they knew the computer would pick them up.

Measuring the clinical effectiveness of CAD will always be problematic, Nishikawa said. Evaluation is highly sensitive to the method used. Wide variance in the number of cancers present each year makes it difficult to assess any difference when CAD is added to daily practice. Variability in reader performance can also mask the software's true impact.

"The real benefit of CAD is not to increase the cancer detection rate but to find cancers earlier," he said.

Breast radiologists at the Elizabeth Wende Breast Clinic in Rochester, New York, have confirmed this finding. They tested its prowess on a form of cancer that is notoriously difficult to diagnose.

The pathological appearance of invasive lobular carcinoma (ILC) makes it far harder to detect mammographically than most other forms of breast cancer, said Dr. Stamatia Destounis, a radiologist at the clinic. The cancer may be mammographically occult or visible on only one of two views. Characteristic signs of ILC include asymmetric density, architectural distortion, and vague palpable thickening.

Destounis and her colleagues reviewed 81 biopsy-prove consecutive cases of ILC collected over an 18-month period. They ran breast CAD software on all sets of mammograms used to diagnose the cancer and on the immediately prior screening set when available (31 cases). CAD correctly marked the cancer in 53 cases (65%) in the year of diagnosis and in eight (26%) of the available priors (see figure). These eight marks highlighted what should have been actionable signs; that is, true misses, Destounis said.

"Breast CAD may improve radiologists' sensitivity, specifically for ILC, and aid in finding these cancers at a smaller size," she said. "So, potentially, CAD may be instrumental in reducing the number of ILC cases with positive lymph node involvement and increasing long-term patient survival."

TOUGHEST TRIAL

Debates over breast CAD's role are also ongoing in Europe, where staffing shortages are focusing attention on double-reading without two radiologists. The use of an automated prompt system holds appeal for organizers of large-scale, population-based breast screening services. While radiologists are in short supply, screening programs must serve an increasing number of eligible women as the population ages. Many regional and national programs are also extending their services to younger and older women. Finding sufficient trained personnel to provide a double-reading service is a challenge.

Prof. Fiona Gilbert, a professor of radiology at the University of Aberdeen, would like to see a Europe-wide randomized controlled trial to prove the case for CAD as a second reader.

"What we would really like to do is a very robust evaluation of CAD in a prospective setting," Gilbert said. "We believe that we should randomize patients to be double-read, as is standard practice in the U.K., against a single reader using CAD."

All U.K. women between the ages of 50 and 70 are now invited for a two-view mammography exam every three years. About 1.5 million women present for screening every year at one of 95 separate sites. Attendance rates stand at 70% for first-time invitees, with 80% of women returning on a regular basis.

Approximately 300 radiologists who have undergone at least six months of dedicated training in breast imaging report at least 5000 mammograms each year. Two readers view the mammograms independently and use either arbitration or consensus to settle disputed calls. Quality assurance guidelines dictate that fewer than 7% of first-time attendees should be recalled. For women returning for rescreening, the upper limit for recall drops to 5%.

"We have very strict criteria," Gilbert said. "If your screening unit is operating above these rates, then questions are asked."

The breast CAD study design should permit the use of arbitration so that the single reader using CAD could benefit from a radiologist's opinion if required.

"If another radiologist is allowed to look at the few cases that the single reader with CAD wants to bring back, we can actually reduce our recall rates," she said.

The issue of recall parameters could pose a problem for organizers of a Europe-wide trial. Expected recall rates in the Netherlands, for example, are several percentage points lower than the accepted U.K. upper limit.

Paula Gould is a contributing editor for Diagnostic Imaging Europe.

References

1. Freer TW, Ulissey MJ. Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology 2001;220(3):781-786.

2. Gur D, Sumkin JH, Rockette HE, et al. Changes in breast cancer detection and mammography recall rates after the introduction of a computer-aided detection system. J Nat Cancer Institute 2004;96(3):185-190.