The first practical intuitive device for browsing digital images using only hand gestures is being tested in intensive care settings and operating suites. Gestix is a video-based gesture capture and recognition system used to manipulate MR images. It was designed, as a joint effort between the Institute for Medical Informatics at the Washington Hospital Center in Washington, DC, and Ben Gurion University of the Negev in Israel, to address ICU sterility issues and surgical focus control (J Am Med Inform Assoc 2008;15:321-323). "In the ICU, the use of computer keyboard, mouse, and joystick by physicians to access medical records and digital images is a common means of spreading infections," said industrial engineer Juan P. Wachs, Ph.D., currently of the Naval Postgraduate School in Monterey, CA. Wachs said that while other noncontact methods of image manipulation, such as voice control, also offer sterility advantages, these aren't practical in ORs. "The noise level in ORs makes voice control systems problematic," he said. Wachs said Gestix gives surgeons in the OR a means of maintaining focus while achieving rapid interaction with the image. Gestix consists of a Canon VC-C4 camera placed over a flat-screen monitor and an Intel Pentium IV (600MHz, Windows XP) with a Matrox Standard II video-capturing device.

The devise uses a two-layer architecture:

• In the lower level, Gestix provides tracking and motion recognition functions.
• At the higher level, a graphical user interface called Gibson manages image manipulation.

Gesture operations are initiated by a calibration procedure in which a skin color model of the user's hand or glove is constructed. Superimposed over the image of the camera's scene is a rectangular frame called the neutral area. Movements of the hand across the neutral area boundary constitute directional commands.

"When the doctor wishes to browse the image database, the hand is moved rapidly out of the neutral area toward any of four directions, and then back again," Wachs said. When such a movement is detected, the displayed image is moved off the screen and replaced by a neighbor image. To evoke zoom mode, the open palm of the hand is rotated within the neutral area clockwise/counterclockwise (zoom-in/zoom-out). To avoid the tracking of unintentional gestures, the user may enter a "sleep mode" by dropping the hand. To arouse the system the user waves the hand in front of the camera. "The selection of these gestures was intuitive to express the natural feeling of the user," Wachs said. For example, left/right/up/down gestures evoke the actions used to turn pages in a book left or right, or flip notepad pages up and down. The rotation gesture (zoom-in/zoom-out) is similar to that of turning a radio knob to increase or decrease volume.