In-Car Gesture Control: Why Early Systems Failed

The first generation of in-car gesture control shipped a few fixed gestures for volume and calls, fired on ordinary hand movement, and needed a dedicated infrared sensor in the headliner. Drivers tried it, got a wrong answer twice, and stopped. The problem was never that gestures are a bad idea in a car. It was that those systems reacted to motion instead of intent. Patented intent detection, running on a standard camera, is what changes the outcome.

What did the first generation get wrong?

It could not tell a command from a conversation. People talk with their hands, point at the view, and reach for a coffee. A system that treats every twirl as a volume change becomes something you learn to avoid. That is the failure automotive reviewers wrote about most, and it is the one we hear from engineers who tried the early systems.

The vocabulary was tiny. A handful of fixed gestures, mapped to a handful of functions, none of which a button did not already do better.

It needed special hardware. A dedicated infrared sensor in the roof console meant cost on every unit, a fixed field of view, and a feature that lived in one trim level.

It was built into one screen. The gesture system spoke only to the infotainment stack it shipped with. New displays, new apps, and new suppliers meant starting over.

Why is the problem worth solving anyway?

Screens in vehicles keep getting larger and further away. A passenger display, a rear seat screen, or a head-up projection is not always within a comfortable reach, and touch on a moving road means looking at the glass. Voice handles discrete commands well and struggles with continuous ones: nudge the map, scroll the list, dim the display a little. Those continuous adjustments are where a hand in the air is the natural input. The case is the same one clinicians make about image review, described in voice, foot pedal, or gesture control in the OR.

What does intent detection change?

AirTouch's patented intent detection identifies the person who means to control the screen and stays with them until they finish. Ordinary movement, from gesturing while talking to reaching for the cup holder, is filtered out in most setups. That single change turns gesture control from a party trick into an input method people keep using.

Three other things changed with it:

• The camera is ordinary. AirTouch runs on a standard RGB camera, which many cabins already carry for driver monitoring or occupant sensing. Where a cabin needs depth or works in the dark, through our partnership with Orbbec, AirTouch also takes advantage of 3D depth sensing and IR cameras when a deployment calls for them. See the Orbbec page.

• The input is standard. Gestures become the touch, pointer, and key events the display software already understands, so there is no SDK to port into the infotainment stack.

• The vocabulary is mappable. Fifteen gestures map to any action, and the map can differ by seat, by screen, and by product line.

Where does it belong in a vehicle?

• Passenger and rear seat displays, where reach and shared touch are the issue and the driver is not involved.

• Head-up and holographic displays, which have no surface to touch at all. See gesture control for holographic and head-up displays.

• Showroom and configurator screens, where a buyer explores a model on a large display.

• Commercial cabs and equipment, where operators wear gloves.

Driver-facing use during driving carries safety and regulatory questions that belong to the OEM's own validation. We will say so in the first meeting.

Where to start

Automotive teams usually start with a skeptical engineer and a display on a bench. Book a demo and bring that engineer. Licensing and integration support are described on the OEM solutions page.

Related: add gesture control to a display product without an SDK, gesture control for Android displays, and gesture control and hand tracking software.