Software pulls images based on visual characteristics

Accessing medical images from large databases using their visual characteristics remains an elusive goal in medical imaging, but progress continues. Limited clinical trials have demonstrated feasibility and organizations are promoting the concept, although such systems are far from becoming a clinical reality, according to a recent report.

Content-based image retrieval (CBIR) offers tremendous potential for medical applications that use digital images for diagnosis, treatment planning, and therapy, according to Henning Müller, a researcher in the medical informatics division at University Hospital Geneva in Switzerland and author of an overview on the topic [Int J Med Informatics 2004:73;1-23].

Physicians generally access archived digital images by inputting a patient's ID number, Müller said. A CBIR system pulls out images according to their visual characteristics, such as color or texture. A radiologist struggling to assess an ambiguous image could retrieve a selection of similar-looking images from past cases.

"Looking at these other images, you can then start considering and ruling out possible diagnoses," Müller said. "You will almost definitely find irrelevant cases showing up, but doctors are generally very quick at spotting which information is relevant and which is not."

Müller acknowledges that CBIR will be relatively difficult to integrate into diagnostic radiology. Accurate categorization according to color and textural features is more straightforward with dermatology, pathology, and hematology images, he said. Work in radiology is needed to produce global classifications of images segmented into regions.

Initial trials of software on specific applications amenable to detailed textural classification, notably high-resolution lung CT, show that CBIR has potential for diagnostic work. But the systems' main role is likely to be in teaching and research, where a relatively high false-positive rate is less problematic, Müller said.

"For teaching, it can actually be useful to find visually similar cases that have a different diagnosis. You can then show the subtle differences that differentiate between diagnoses," he said.

Development of effective CBIR systems for radiology will require thorough standardized methods of comparative assessment, Müller said. He recommends that systems be compared for their speed, sensitivity, specificity, accuracy, precision, and recall, with trials based on intended clinical use.

Access to comprehensive reference image databases will be essential for CBIR system evaluation, he said. Some free-to-use image collections are already available over the Internet. The European Federation of Medical Informatics working group on medical image processing has formed an initiative to generate relevant medical image collections. Images included in these open-access databases will be anonymized, though certain clinical information could be relevant.

"For example, lung textures can differ significantly depending on whether you have an older or a younger patient," Müller said.

Reference databases should be regularly updated to ensure CBIR algorithms keep pace with increasingly sophisticated medical images, Müller said. He is testing his own system on a teaching file at University Hospital Geneva. The in-house database contains some lung CT studies acquired with thick slices, as well as recent studies containing hundreds of thin-slice views.

"Images have changed so much over the past 10 years. The level of detail in the textures can be very, very different in exactly the same kind of images acquired just a few years apart," he said. "So the software needs to be adapted periodically to fit the images you are actually producing."

Should CBIR become a standard tool in future PACS, Müller recommends that the software be accessible via an open interface. Radiologists should not be tied to a specific vendor's system and should have the option to add modules or use alternative systems, he said.

"These interfaces are extremely important," he said. "It's similar to the situation in medical imaging when departments were switching to soft copy. Pre-DICOM, hospital departments had to buy their viewing stations from certain vendors because all the viewing formats were proprietary. But it's important to be able to communicate with different systems."