Hologic execs assail CAD study as flawed

A study asserting that mammography CAD does more harm than good is deeply flawed in design and conclusions, according to executives at Hologic. The study, which appears today in The New England Journal of Medicine, concludes that CAD, when used in screening mammography, causes significantly more callbacks and biopsies without increased cancer detection.

Andy Smith, Ph.D., Hologic manager of imaging sciences, described the study as more statistical than clinical.

"It is one thing to do these complex clinical studies with statisticians and algorithms, where people argue until the end of time whether the study was done properly or not. But you only need to talk to radiologists in clinical practice to discover the real value of CAD," he told DI SCAN.

Hologic has a lot riding on this debate. Its subsidiary, R2 Technology, is the premier provider of mammography CAD, having installed more than 2500 CAD products worldwide. The company's ImageChecker was approved by the FDA in 1998 for film-based mammography and in 2001 for digital mammography. Other vendors, including iCAD and Kodak, also provide CAD, but the CAD sites included in the NEJM article all used ImageChecker. This technology was the preeminent CAD product at the time of the study.

Dr. Joshua Fenton, an assistant professor of family and community medicine at the University of California, Davis, Health System, and colleagues at eight other U.S. medical institutions wrote that using CAD in screening mammography causes significantly lower overall accuracy and a significantly higher rate of false positives, leading to more recalls and biopsies. They called for larger studies to judge more precisely whether the benefits of routine use of CAD outweigh its harms.

Hologic was tipped off to the impending publication earlier this week and began gathering data that might blunt its effect on CAD.

"There have been so many positive studies, at least 30. In fact, I stopped counting," said Jim Culley, director of strategic projects at Hologic.

Company executives yesterday reached out with those data and their argument about the value of CAD to consumer-oriented media, including wire services and newspapers. All reported the NEJM study, leading to a $4 fall in Hologic's share price at the opening bell of trading. The price had made up some of that ground by late morning but remained down $2 at about $59 per share.

Mammography CAD boosts revenues directly and indirectly for sites using it. Medicare adds $20 for the use of CAD to mammography reimbursements of between $100 and $150. Providers also promote the technology to patients and referring physicians as an example of their technological prowess. If the NEJM article creates a general impression that CAD mammography is harmful, it could pull the rug out from under the facilities using this technology as a marketing tool and reduce demand from those who have yet to buy it.

"I think there is a real risk there," Smith said. "But it is not clear that the radiologists and professionals in the field will be fooled by this paper and change what they are doing."

The flaws in the study are the strengths in Hologic's argument that CAD is a valuable clinical tool, according to the company. Culley noted that 36 of the 43 sites participating in the survey reported data not obtained with CAD and that the seven sites with experience implemented the technology during the study period. This produced a lopsided comparison: 398,159 interpretations without CAD and 31,186 with CAD. Most important, the CAD-related data were obtained early in the learning curve, when mammographers were still figuring out how to use CAD.

Data presented by the researchers indicate that at least one site provided data during just its initial two months of working with the technology. The most experienced site operated CAD for just 25 months. Average experience was just seven months, even though the study period ran from 1998 to 2002, Culley said.

The availability of data before and after CAD implementation led to a comparison of specificity, 90.2% before and 87.2% after, that put CAD in an unfavorable light. Adding to the authors' diminished view of CAD was an increase in recall rates from 10.1% before CAD implementation to 13.2% after. Biopsies per 1000 screening mammograms increased from 14.7 before CAD to 17.6 after.

Rather than conclude that the use of CAD is associated with reduced accuracy in screening mammography, as the Fenton group did, the data suggest to Smith and Culley that mammographers need to exert care not to overinterpret results during the early adoption of CAD.

They are not alone in this concern. In an editorial accompanying the NEJM published results, Dr. Ferris M. Hall, a Harvard professor of radiology on staff at Beth Israel Deaconess Medical Center in Boston, questioned the researchers' failure to consider the time needed to adjust to CAD, noting that mammographers initially exposed to this technology may be unduly influenced by the suspicious lesions marked on each mammogram. Hall further noted that the adjustment to CAD can take weeks to years.

Smith raised a related point about the technology itself, asserting that CAD mammography has improved substantially in the five years since the survey was conducted, markedly reducing the number of false positives. He further noted that the Fenton study looked only at screen-film CAD - digital mammography became commercially available only in 2000 - and 90% of digital mammography systems today ship with CAD.

Fenton and colleagues described as significant in their paper an increase in sensitivity from 80.4% before CAD implementation to 84%. Research published previously in the American Journal of Roentgenology and Radiology, as well as other journals, documented substantial increases in detection rate, particularly of small, invasive cancers.

CAD's strength is its ability to detect the early signs of cancer, said Culley, who noted that the NEJM article documented the ability of this technology to find more cases of ductal carcinoma in situ. The researchers discounted the value of such detections, however, stating that DCIS usually does not progress to the most life-threatening forms of breast cancer.

Culley worries that the most recently published results will have added weight because of the journal in which they were published. In the accompanying editorial, Hall opined that the results would not end the use of CAD in screening mammography, but "they constitute a substantial hit to this technology."

Unfortunately, if the technology takes a hit, so will women's healthcare, Smith said.

"If you have ever talked to a radiologist who found a cancer through CAD that he or she had missed, you realize that the impact of CAD is quite important," he said. "These cases are not just anecdotal. They come up all the time."

Editor's note: This story replaces an earlier version.