Siemens 3D technology lets surgeons 'see' inside patients

Prototype could improve efficiency and precision

Augmented reality image guidance is how Siemens executives describe a prototype that enables in situ visualization. The technology features a head-mounted display that allows surgeons to 'see' inside patients. Surgeons using the system will be able to look directly at a surgical site and at the same time see 3D reconstructions from MR or CT, as well as patient vital signs.

Dr. Gregory Rubino, medical director of the Neurosurgical Center of Excellence at Holy Cross Hospital in Silver Spring, MD, is working with Siemens to explore the clinical applications of the new system. He believes it can speed up surgical procedures and at the same time provide surgeons with more useful information.

"What this does is eliminate the step (of looking back and forth between the patient and a nearby monitor), and put the information right in front of you, you can look right into the patient's head," Rubino said.

The prototype employs a head-mounted display equipped with three miniature cameras. Two capture a stereoscopic view of the surgical site; the third is used for viewpoint tracking, in combination with optical sensors that frame the site. A computer superimposes 3D images taken from the patient's own CT or MR data onto the video for viewing on the surgeon's display. The surgeon, looking at the patient, can then see computer models of anatomical structures at the location of the actual structures.

Whether doctors use the device for surgical planning or during actual surgery will depend on the procedure as well as the commercial development of the device.

"There are different options," said Frank Sauer, Ph.D., project manager in Siemens corporate research's imaging and visualization department. "In the beginning, it could just be a tool to better understand where the actual structures are. The other option is to act while you wear it, perform different kinds of interventional procedures."

The system is already sufficiently developed to provide value to surgeons, according to Rubino. He said it would be extremely useful, even at this early stage, for brain biopsies.

"You could use this device, calibrated to the patient's head, and do a frameless biopsy that would take probably as little as 15 minutes," he said.

The next step for Siemens will be to determine in which clinical areas the technology would be most useful--and to chart a development path toward commercialization. Once the pilot applications are selected, and the technology enters clinical trials and exploration, Sauer estimates it would be three years before a commercialized version is available.

"We receive a lot of feedback from people saying 'Oh, that's really neat,' and it is. But it has to be more than that," he said. "So for us, it's very much understanding from the point of the doctor how we can make it a useful and most practical tool."