Voice, Foot Pedal, or Gesture Control in the OR?

Here is the short answer. There are four practical ways for a scrubbed clinician to control a screen: voice commands, foot pedals, draped peripherals, and camera-based gesture control. Voice suits discrete commands in quiet rooms. Pedals suit a few repeated actions. Draped peripherals are cheap and familiar, and they degrade in use. Gesture control suits continuous navigation such as scrolling, zooming, and rotating, which is most of what image review involves. Many rooms end up with a mix.

What are the options for sterile screen control?

Every option is a workaround for the same constraint. The person making decisions from the images cannot touch the computer that holds them. Today the most common answer is a second person: the surgeon narrates and a circulator clicks. The four alternatives below all try to hand control back to the clinician.

How does voice control hold up in an operating room?

Voice is appealing because it needs no hands and no line of sight. For discrete commands it can work well: next series, show the coronal view, start recording.

It struggles in two places. The first is the room itself. Masks, suction, alarms, music, and several people talking make recognition harder, and a misheard command mid-procedure costs more than the click it replaced. The second is the nature of image review. Scrolling a stack to the right slice, adjusting window level until the lesion stands out, rotating a 3D volume: these are continuous adjustments. Saying "down, down, down, stop, up one" is the verbal relay problem again, with a computer in place of the circulator.

Where voice wins: discrete commands, rooms where the clinician's hands are fully occupied, and systems already built into the OR integration platform.

What about foot pedals?

Pedals are proven and dependable. Fluoroscopy, electrosurgery, and microscopes all use them, which is also the problem. The floor is crowded, the clinician cannot look down, and each pedal supports only a handful of mapped commands. A pedal can page forward and back. It will not set window level, jump to a prior study, and rotate a reconstruction.

Where pedals win: one or two high-frequency actions that never change, such as next image and previous image.

Do draped mice and touchscreens solve it?

A sterile cover over a mouse, keyboard, or touchscreen keeps the familiar interface and costs very little. In use, the experience degrades. Clicks misfire through plastic, scroll wheels drag, touch accuracy drops, and the cover becomes one more thing to manage in the field and replace between cases. The device also has to sit within reach, which is not where the big display usually hangs.

Where draping wins: occasional use, tight budgets, and tasks that need text entry.

Where does gesture control fit?

Camera-based gesture control and hand tracking software maps hand movement to the mouse, keyboard, and touch input the viewer already accepts. That makes it a natural fit for the continuous part of image review. The hand moves, the stack scrolls. A pinch and drag rotates the volume. Mapped gestures fire the viewer's own shortcuts for everything else.

AirTouch works with a standard RGB camera, and nothing enters the sterile field. The models are trained for gloved hands. Patented intent detection separates the clinician deliberately controlling the screen from the rest of a busy room, which was the main failure of earlier camera systems.

Where gestures lose: the clinician needs at least one hand free for a moment and a line of sight to the camera. Text entry is slow. Unusual lighting or placement may call for a specialized camera, and through our partnership with Orbbec, AirTouch also takes advantage of 3D depth sensing and IR cameras when a deployment calls for them. See the Orbbec page.

Which should you choose?

These are not exclusive. A room can keep its pedal for next image and add gestures for everything the pedal cannot do.

Is gesture control a medical device?

AirTouch is computer input software. It sends the same mouse and keyboard events a person would, and it does not process, alter, or interpret clinical images. It is not a medical device, and we make no clinical outcome claims for it.

Where to start

The fair test is your room, your viewer, and your gloves. Book a demo and we will test against your environment. To try the interaction on your own first, start a free trial.

Related: touchless image control in the IR suite and cath lab, touchless control in the sterile operating room, and gesture control for medical environments.