Quality appears in the eye of the beholder

In the continual struggle between humans and machines, it seems that machines hold an edge because of their ability to crunch large databases without tiring and to produce accurate analyses. Dr. Khan M. Siddiqui knows otherwise.

Dr. Khan M. Siddiqui wants to better understand radiologists' interpretation process and use the data to help design intelligent workstations.

As a fellow at the VA Maryland Health Care System in Baltimore, Siddiqui was researching how to optimize image quality when something opened his eyes. He noticed that his perception of good images did not always mesh with the computer's choices, based on mathematical measures such as signal-to-noise ratios.

He found that a model closely correlated with human observer perception called just noticeable differences (JND) was better able to predict image quality than traditional quantitative measures. These findings earned the 32-year-old a 2004 RSNA Fellow Research Trainee Prize and a spot in the proceedings of the 2005 Society of Photo-Optical Instrumentation Engineers Medical Imaging Conference.

Under the direction of Dr. Eliot Siegel, the informatics research group at the VA continually pushes the boundaries of the PACS and informatics knowledge base. The researchers work in a unique collaborative fashion, where ideas gain momentum from other ideas. Siddiqui calls it a spiraling research approach.

"Someone will start off with an idea, and then we'll spin off different studies depending on the data," he said.

The initial pursuit for Siddiqui and colleagues was to find the lowest possible radiation dose for lung nodule detection and to experiment with noise filtering for low-dose images. The spinoff idea came when the computer and the researchers judged distinctly different images as being high quality.

"It was obvious that we needed an objective method to assess image quality as it is perceived by the human visual system," said Siddiqui, now chief of imaging informatics at the VA.

CR Image Quality: PSNR vs. JND

The JNDmetrix visual discrimination model correlates well with radiologists' assessments of image quality, in comparison with traditional metrics such as peak signal-to-noise ratio.

To test the JND visual discrimination model, 11 radiologists examined 80 CT and computed radiography images on 3-megapixel LCD monitors. The images had undergone a range of JPEG compression from lossless up to 60:1.

As expected, the researchers found that measures of image acceptability were inversely proportional to the level of image compression. Normalized reader scores were highly correlated to JND values for both CT (-0.9) and CR (-0.91). Peak signal-to-noise ratios did not correlate as well to human observations, with values of 0.78 for CT and 0.63 for CR. The differences between peak signal-to-noise ratios and JND correlation to normalized reader scores were statistically significant.

"This research can have an enormous impact on how we assess image quality. By showing that visual discrimination models have excellent correlation with radiologists' perceived image quality, we can use JND image analysis to narrow down the variables prior to conducting human observer ROC studies, which can be very time consuming," Siddiqui said.