Standard data sets provide basis for volumetric pixels

Actuality Systems, a start-up visualization firm in Burlington, MA, is seeking the future in a crystal ball of its own making.

Inside this transparent dome, about the size of a basketball, a screen the size of a lollipop spins faster than the eye can detect, catching and reflecting 100 million points of light cast by a high-intensity projector. The light hovers, forming a ghostlike image that can be studied from top, bottom, or sides.

Such a capability could help air traffic controllers predict airplane collisions, pharmaceutical designers see the interaction of drugs and proteins, security guards recognize weapons in luggage. A DICOM-compatible version of this technology, called Perspecta Medical, promises to help physicians as well.

"You can load up a very complex stack of slices, getting coronal, sagittal, and axial views on a computer screen, then circle a spherical region with the cursor-and immediately those data will pop up in spatial 3D on the Perspecta display," said Gregg Favalora, founder and chief technology officer of Actuality Systems.

The National Institutes of Health Clinical Center is looking into this idea, evaluating the Perspecta Spatial 3D system for use in surgical planning and radiation treatment. An early application may be to visualize patient data prior to radio-frequency ablation of liver tumors and to compare the location of the tumor before and during intervention. Ultimately, the system might be integrated into radiation-based regimens that require extraordinary precision, such as intensity-modulated radiation therapy. Other applications may be in such routine procedures as breast biopsy and catheter placement. Before applying Perspecta in such cases, however, researchers are assessing the value of the system, using data from past cases.

"We want to quantify the benefit of looking at things in their natural, spatial 3D state," Favalora said.

Preliminary results have come from studies conducted by the U.S. Navy, which determined that spatial 3D might help operators quickly and accurately avoid airplane collisions. The effect of spatial 3D on speed and accuracy in medical applications is now being examined. Results could be published by the end of this year, he said.

"The results we are getting (from our luminaries) are material and distinct," said Cameron Lewis, president and CEO of Actuality Systems. "They show this technology can have an impact on the outcome of medical procedures."

The Perspecta technology is more than a bit out of the ordinary. To date, true 3D systems have depended on gadgets and special effects, such as goggles that merge stereoscopic data. Alternatively, and much more commonly, 3D reconstructions are rendered on 2D computer screens, relying on shading, texture, and vanishing points to create the illusion of three dimensions. Perspecta uses a different approach. Perspecta displays cause the observer to react. Depth cues cause a person's eyes to swivel inward or outward, when looking at the image from different angles, Favalora said. Lenses in the eyes change focus.

"In spatial 3D, images fill up a volume, but it's not just a sculpture or hologram frozen in time," Favalora said. "It is something the clinician can manipulate using software."

The images are rendered in seconds, allowing operators to segment and magnify regions of the reconstructed object interactively. The Perspecta platform typically requires no special programming, if the software is based on the Open GL standard.

"You just plug in Perspecta," he said. "Any graphics that appear on a flat computer screen float in the Perspecta display."

The idea behind the technology goes back decades to the time when microwave ovens were invented. Back then, military engineers began experimenting with high-intensity lasers built into a room-sized device that fired beams of light into a centrally located, rapidly rotating screen. These devices proved that spatial 3D was possible. But it was not practical, as a single device could take up a small room and cost millions of dollars.

The availability of microdisplay components that automatically focus projected images, expanding and inexpensive computer memory, and the company's capacity to invent proprietary new optics set the foundation for creating Perspecta, according to Favalora. Funding from private and institutional investors provided fuel for its initial development. A $1.7 million grant, obtained last September from the National Institute of Standards and Technology, is funding research to eliminate the spinning screen and dome, which-if successful-would dramatically reduce the cost of making spatial displays.

For the time being, at least, this gee wizardry remains expensive, but not prohibitively so. The company has sold 14 units with prices beginning at $39,995 for a complete system, including hardware and software. Support and maintenance contracts, and the software developer kit (SDK) bump up the price.

The SDK is a good investment, as it provides the tools to make modifications that optimize the software for spatial display, according to Lewis. It is not the means for creating new applications, nor are any necessary.

"We are taking existing applications and improving their outcomes, using Perspecta," he said.

The company is discussing with two medical imaging companies how Perspecta might be integrated into radiological systems. It is also working with medical luminaries, in addition to those at the NIH Clinical Center, to learn how Perspecta should evolve.

Evaluations address three key points. One is processing speed. Engineers at Actuality Systems want to document the performance needed to achieve clinically valuable, interactive images in spatial 3D.

The second is how operators prefer to view a reconstructed object. If the preference is to look down from the top of the dome, the optical quality of the dome will be optimized for that view.

Third, engineers want to refine the size and shape of the Perspecta unit to fit the environment in which it will be used. Design specifications are much different for a spatial display mounted on an ultrasound scanner, for example, than for ones meant to go in a reading room or interventional suite. Currently the Perspecta unit, which generates a 12-inch-diameter image, has a base about 24 inches wide.

Spatial displays may be especially well suited to surgical and radiotherapy planning. At least one Boston hospital is evaluating Perspecta as an aide in planning liver transplants. The company is also getting interest from radiologists and cardiologists. The system might be used in the cardiac cath lab, Lewis said, to merge 3D MR or CT data sets with continuously updated 2D images of the heart. Interventional procedures, such as breast biopsy or the RF ablation of tumors, might also benefit. But the company is not betting on any one area, preferring to explore as many as possible.

"Our objective is to put Perspecta in as many places as it has utility," Lewis said.