Mary Roddie, MD

 Ideal computer-aided detection tool combines high sensitivity with low false-positive rates 

Minimally invasive CT colonography has been embraced by radiologists and patients alike. As the technique evolves, its use is shifting from specialized academic centers to community hospitals and private practices. That transition is focusing increased attention on reimbursement, clinical efficacy, and interpretation issues. Computer-aided detection for CTC could affect all three.

Comparisons with optical colonoscopy suggest equivalent sensitivity and specificity when CTC is used to detect adenomatous polyps and invasive cancer.1-5 Other studies have found it less effective.6,7

Interpretation of CTC entails a meticulous evaluation of the complete luminal surface of the colonic wall, making interpretation a time-consuming and tiring task. These factors are implicated as major factors behind poor radiologist performance. In the setting of colorectal neoplasia screening, it has been estimated that more than 13,000 images must be interrogated in order to identify a single 1-cm polyp.8

Experts agree that one of the most important factors affecting the diagnostic performance of CTC is radiologist training and experience.9 The learning curve varies among radiologists, and also with the methods used for primary interpretation (2D versus 3D). An important condition for broad acceptance and reimbursement of CTC will be the assurance of high-quality standards of reporting and interpretation and elimination of the current wide interobserver variability in accuracy.

Computer aided detection has proved effective in situations where radiologists must detect small or subtle lesions that occur infrequently, such as screening mammography10 and pulmonary nodule detection.11 Studies using small data sets have shown that CAD may also be effective for CTC.12-14 A 2005 study of screening patients found that an experimental CAD algorithm detected colonic adenomas with an efficacy similar to optical colonoscopy.15 It has been suggested that CAD might improve radiologist sensitivity for polyp detection while reducing expensive interpretation time.

The radiology community has eagerly anticipated the arrival of CAD software for CTC but until recently no commercial applications have been available. In the past year, at least two CAD products have received U.S. Food Drug and Administration approval: Polyp Enhanced Viewing from Siemens Medical Solutions (Figure 1) and ColonCAR (computer-assisted reader) from Medicsight.

Other products have received regulatory approval for use outside the U.S., including V3D-Colon from Viatronix (Figure 2) and CT Colon from Vital Images (Figure 3). Other products are in development. All of these CAD tools have been designed to detect polyp "candidates" and to draw them to the radiologist's attention.

The ideal CAD tool should combine high sensitivity for detection of colorectal polyps and cancer with the lowest possible false-positive rate.

In general, CAD manufacturers must opt for a level on the free response receiver operator characteristic (FROC) curve where optimal sensitivity combines with an acceptable number of false positives. Increasing sensitivity brings with it a higher number of false positives. This is well illustrated in preliminary studies in which stand-alone CAD performance was measured. Sensitivity rates for CAD in detecting polyps varied between 64% and 100%, while the average number of false-positive prompts ranged from two to eight per exam. The CAD system with the best sensitivity in these papers also had the highest number of false-positive prompts per case.

These studies are limited by the fact that they fail to address the interaction of CAD with the radiologist. A 2006 study comparing the performance of CAD with unaided readers probably overestimated the benefit of CAD, since there is no guarantee that readers would have accepted the true-positive prompts or indeed dismissed the false-positive prompts.16

It's possible that experienced readers will perform better with CAD at a higher sensitivity level on the FROC curve, as they are capable of dismissing false-positive prompts quickly and easily. Inexperienced readers, on the other hand, might perform better at a lower sensitivity level with fewer false-positive prompts.

There is more than one way to use CAD. The traditional paradigm is to employ it as a second reader. The radiologist reads the case according to his or her normal practice and afterwards applies the CAD. Each image bearing a CAD prompt is then reviewed and the polyp candidate either accepted or rejected.

This approach has been shown to improve reader accuracy in mammography and for lung nodule detection. It also prolongs interpretation time. With hundreds of images to review in CTC, this process may be too time-consuming. When reading CTC, radiologists may not use CAD as a second reader unless the number of false positives is very low, which may limit the sensitivity.

An alternative is to use the CAD as a concurrent reader, applied before the reading process begins. Polyp candidates are accepted or rejected by radiologists during their initial interrogation of a case. The number of allowed false positives can be slightly higher, as the "irritation factor" of a false-positive finding is likely to be lower if displayed during an initial read.

It is not clear how concurrent read CAD prompts affect reader behavior. Preliminary experience with lung CAD suggests that a second read paradigm results in higher sensitivity for nodule detection, although using CAD as a concurrent reader is more time efficient.17 It may be that concurrent read CAD prompts distract the reader from unmarked pathology or simply that the reader becomes too reliant on the CAD.

No clinical studies have yet been published comparing the two approaches. Until then, users will likely have to choose between faster interpretation time with concurrent read CAD and increased accuracy with the second read method.

CAD has the potential to make CTC reading easier for nonradiologists and thus could be perceived as a threat to the radiology community. Many gastroenterologists believe that they are qualified to interpret CT colonography because of their expertise in conventional optical colonoscopy. Radiologic technologists are also interested in expanding their clinical role. Many have expressed an interest in learning CTC interpretation. Others argue that if CTC ever becomes a major tool in colorectal cancer screening, radiologists will be ill-equipped to report all the exams generated by a population-screening program.

The key opinion leaders in CTC, however, emphasize the need for close collaboration with gastroenterology colleagues. CAD could facilitate this collaboration by enabling efficient triage of patients with polyps or cancer from the imaging department to same-day colonoscopy. Any tool that makes radiologists more efficient and accurate is beneficial to patients and to the reputation of radiology among clinical colleagues.

As we explore the strengths and pitfalls of CAD for CTC, we also should be thinking about future capabilities for the technology.

As CTC with fecal tagging becomes more widely practiced, CAD will need to adapt. Rather than simply detecting polypoid lesions protruding into the lumen, it will need to be able to detect polyps submerged in tagged fluid. CAD algorithms should also be trained to distinguish between residual fecal material and polyps. CAD software that can perform sophisticated analyses of the region of interest to distinguish between a polyp and fecal residue may facilitate the interpretation of minimal or no prep CTC. When there is consensus regarding the polyp size category to report, CAD algorithms should be trained to ignore smaller objects. Alternatively, systems could allow users to select upper and lower limits for polyp detection.

CAD should not only detect lesions but also perform simultaneous automatic measurement. CAD prompts would then be presented to the user in size order, with the largest polyp candidate shown first. This makes great sense in terms of workflow, especially when patient management is determined by the identification of a polyp of 1 cm or greater. There is little point in spending valuable time interrogating three potential tiny polyp candidates in the sigmoid colon only to later find a huge polyp or cancer in the cecum. The latter lesion ideally should be the first candidate to be presented to the radiologist.

Flat adenomas are challenging for CAD to detect, but can be recognized by radiologists as areas of subtle colonic wall thickening. New CAD algorithms that automatically analyze colonic wall thickness and detect unexpected areas of focal thickening could be useful in flat adenoma detection.

Finally, how good is CAD in detection of colon cancer? Most studies have concentrated on the performance of CAD in polyp detection. While this is likely to be the main role for CAD in CTC, we should note that radiologists have been known to overlook invasive cancers. A CAD algorithm optimized for cancer detection with near 100% sensitivity would be an extremely valuable tool for the general radiologist.

CAD for CTC has arrived, but the ideal paradigm for its integration with routine clinical practice has not been established. Initial study results for CAD in CTC appear promising, although its effect on performance and efficiency needs to be proven in large studies. As experience evolves, the role of CAD in CTC will become clearer.

DR. RODDIE is a consultant radiologist at Charing Cross Hospital and the director of radiology at Medicsight plc, a developer of CAD technology for imaging modalities, in London.

1. Fenlon HM, Nunes DP, Schroy PC III et al. A comparison of virtual and conventional colonoscopy for the detection of colorectal polyps. NEJM 1999; 341:1496-1503.

2. Yee J, Akerkar GA, Hung RK, et al. Colorectal neoplasia: performance characteristics of CT colonography for detection in 300 patients. Radiology 2001; 219:685-692.

3. Pineau BC, Paskett ED, Chen GJ, et al. Virtual colonoscopy using oral contrast compared with colonoscopy for the detection of patients with colorectal polyps. Gastroenterology 2003;125:304-310.

4. Pickhardt PJ, Choi JR, Hwang I, et al. Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. NEJM 2003; 349:2191-2200.

5. van Gelder RE, Nio CY, Florie J, et al. Computed tomographic colonography compared with colonoscopy in patients at increased risk for colorectal cancer. Gastroenterology 2004;127:41-48.

6. Cotton PB, Durkalski VL, Pineau BC, et al. Computed tomographic colonography (virtual colonoscopy): A multicenter comparison with standard colonoscopy for detection of colorectal neoplasia. JAMA 2004;291:1713-1719.

7. Rockey DC, Paulson E, Niedzwiecki D, et al. Analysis of air contrast barium enema, computed tomographic colonography, and colonoscopy: prospective comparison. Lancet 2005;365:305-311.

8. Johnson CD, Harmsen WS, Wilson LA, et al. Prospective blinded evaluation of computed tomographic colonography for screen detection of colorectal polyps. Gastroenterology 2003;125:311-319.

9. Soto JA, Barish MA, Yee J. Reader training in CT colonography: how much is enough? Radiology 2005;237:26-27.

10. Warren Burhenne LJ, Wood SA, D'Orsi CJ, et al. Potential contribution of computer-aided detection to the sensitivity of screening mammography. Radiology 2000;215:554-562.

11. Awai K, Murao K, Ozawa A, et al. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. Radiology 2004; 230:347-352.

12. Yoshida H, Masutani Y, MacEneaney P, et al. Computerized detection of colonic polyps at CT colonography on the basis of volumetric features: pilot study. Radiology 2002;222:327-336.

13. Summers RM, Johnson CD, Pusanik LM, Malley JD, Youssef AM, Reed JE. Automated polyp detection at CT colonography: feasibility assessment in a human population. Radiology 2001;219:51-59.

14. Kiss G, Van CJ, Thomeer M, Suetens P, Marchal G. Computer-aided diagnosis in virtual colonography via combination of surface normal and sphere fitting methods. Eur Radiol 2002;12:77-81.

15. Summers RM, Yao J, Pickhardt PJ, et al. Computed tomographic virtual colonoscopy computer-aided polyp detection in a screening population. Gastroenterology 2005;129:1832-1844.

16. Taylor SA, Halligan S, Burling D, et al. Computer-assisted reader software versus expert reviewers for polyp detection on CT colonography. AJR 2006;186:696-702.

17. Beyer F, Zierott L, Stoeckel J, et al. Computer-assisted detection (CAD) of pulmonary nodules at MDCT: Can CAD be used as a concurrent reader? Eur Radiol 2005;15(supplement 1):168.

Articles

Ideal computer-aided detection tool combines high sensitivity with low false-positive rates

CT colonography tools advance in clinical use

Reimbursement for screening CTC is not yet available through Medicare, although private carriers are beginning to reimburse selectively for specific diagnostic indications and, in some areas, for screening. This has somewhat restricted use of CTC in the U.S. Medicare's position is unlikely to change before large clinical trials prove that CTC performs as well in community practice settings as in academic centers of excellence.

Comparisons with optical colonoscopy suggest equivalent sensitivity and specificity when CTC is used to detect adenomatous polyps and invasive cancer.

 Other studies have found it less effective.

Interpretation of CTC entails a meticulous evaluation of the complete luminal surface of the colonic wall, making it a time-consuming and tiring task. These factors are implicated as major reasons behind poor radiologist performance. In the setting of colorectal neoplasia screening, it has been estimated that more than 13,000 images must be interrogated in order to identify a single 1-cm polyp.

Experts agree that one of the most important factors affecting the diagnostic performance of CTC is radiologist training and experience.

 The learning curve varies among radiologists and also with the methods used for primary interpretation (2D versus 3D). An important condition for broad acceptance and reimbursement of CTC will be the assurance of high-quality standards of reporting and interpretation and elimination of the current wide interobserver variability in accuracy.

Computer-aided detection has proved effective in situations where radiologists must detect small or subtle lesions that occur infrequently such as screening mammography

 Studies using small data sets have shown that CAD may also be effective for CTC.

 A 2005 study of screening patients found that an experimental CAD algorithm detected colonic adenomas with an efficacy similar to optical colonoscopy.

 It has been suggested that CAD might improve radiologist sensitivity for polyp detection while reducing expensive interpretation time.

The radiology community has eagerly anticipated the arrival of CAD software for CTC, but, until recently, no commercial applications have been available. In the past year, at least two CAD products have received FDA approval: Polyp Enhanced Viewing from Siemens Medical Solutions (Figure 1) and ColonCAR (computer-assisted reader) from Medicsight.

In addition, other products exist that have received regulatory approval for use outside the U.S., including V3D-Colon from Viatronix (Figure 2) and CT Colon from Vital Images (Figure 3). Other products are in development.

In general, CAD manufacturers must opt for a level on the free response receiver operator characteristic (FROC) curve where optimal sensitivity combines with an acceptable number of false positives. Increasing sensitivity brings with it a higher number of false positives. This is well illustrated in preliminary studies in which standalone CAD performance was measured. Sensitivity rates for CAD in detecting polyps varied between 64% and 100%, while the average number of false-positive prompts ranged from two to eight per exam. The CAD system with the best sensitivity in these papers also had the highest number of false-positive prompts per case.

These studies are limited by the fact that they fail to address the interaction of CAD with the radiologist. A 2006 study comparing the performance of CAD with unaided readers probably overestimated the benefit of CAD, since there is no guarantee that readers would have accepted the true-positive prompts or indeed dismissed the false-positive prompts.

An alternative is to use the CAD as a concurrent reader, applied before the reading process begins. Polyp candidates are accepted or rejected by radiologists during their initial interrogation of a case.

It is not clear how concurrent read CAD prompts affect reader behavior. Preliminary experience with lung CAD suggests that a second read paradigm results in higher sensitivity for nodule detection, although using CAD as a concurrent reader is more time efficient.

 It may be that concurrent read CAD prompts distract the reader from unmarked pathology or simply that the reader becomes too reliant on the CAD.

No clinical studies have yet been published comparing the two approaches. Until they are, users will likely have to choose between faster interpretation time with concurrent read CAD and increased accuracy with the second read method.

CAD has the potential to make CTC reading easier for nonradiologists and thus could be perceived as a threat to the radiology community. Many gastroenterologists believe that they are qualified to interpret CT colonography because of their expertise in conventional optical colonoscopy. Radiologic technologists are also interested in expanding their clinical role. Many have expressed an interest in learning CTC interpretation. Others argue that if CTC ever becomes a major tool in colorectal cancer screening, radiologists will be ill equipped to report all the exams generated by a population-screening program.

Flat adenomas are challenging for CAD to detect, but they can be recognized by radiologists as areas of subtle colonic wall thickening. New CAD algorithms that automatically analyze colonic wall thickness and detect unexpected areas of focal thickening could be useful in flat adenoma detection.

Finally, how good is CAD in detection of colon cancer? Most studies have concentrated on the performance of CAD in polyp detection. While this is likely to be the main role for CAD in CTC, we should note that radiologists have been known to overlook invasive cancers. A CAD algorithm optimized for cancer detection with near-100% sensitivity would be an extremely valuable tool for the general radiologist.

Dr. Roddie is a consultant radiologist at Charing Cross Hospital and director of radiology at Medicsight, a developer of CAD technology for imaging modalities, in London.

1. Fenlon HM, Nunes DP, Schroy PC III, et al. A comparison of virtual and conventional colonoscopy for the detection of colorectal polyps. NEJM 1999;341:1496-1503.

2. Yee J, Akerkar GA, Hung RK, et al. Colorectal neoplasia: performance characteristics of CT colonography for detection in 300 patients. Radiology 2001;219:685-692.

3. Pineau BC, Paskett ED, Chen GJ, et al. Virtual colonoscopy using oral contrast compared with colonoscopy for the detection of patients with colorectal polyps. Gastroenterology 2003;125: 304-310.

4. Pickhardt PJ, Choi JR, Hwang I, et al. Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. NEJM 2003;349:2191-2200.

5. van Gelder RE, Nio CY, Florie J, et al. Computed tomographic colonography compared with colonoscopy in patients at increased risk for colorectal cancer. Gastroenterology 2004; 127:41-48.

8. Johnson CD, Harmsen WS, Wilson LA, et al. Prospective blinded evaluation of computed tomographic colonography for screen detection of colorectal polyps. Gastroenterology 2003; 125:311-319.

11. Awai K, Murao K, Ozawa A, et al. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. Radiology 2004;230:347-352.

14. Kiss G, Van CJ, Thomeer M, et al. Computer-aided diagnosis in virtual colonography via combination of surface normal and sphere fitting methods. Eur Radiol 2002;12:77-81.

Mary Roddie, MD

Articles

CT colonography tools advance in clinical use

Tools for CT colonography advance toward clinical use