Gesture Control for Drive-Thru and Kitchen Displays

Quick service restaurants are automating the drive-thru with voice AI, and voice is the right tool for taking an order. It is the wrong tool for the kitchen display, where a crew member with gloved, greasy, or full hands needs to bump an order, recall it, or pull up a build. Gesture control fills that gap. AirTouch reads hand gestures through a standard RGB camera and sends the touch and key input a kitchen display system already understands, so the crew controls the screen without touching it and without talking over the kitchen.

Why is the kitchen display the beachhead?

The customer-facing lane gets the attention and carries the most risk. A misheard order costs a customer. The crew side is where a small change pays quickly and quietly.

• Hands are gloved and greasy. Bump bars and touchscreens get wiped down, tapped with a knuckle, or ignored until a glove comes off.

• Kitchens are loud. Fryers, hoods, headsets, and a dozen conversations make voice a poor input for the crew, even where it is excellent for the customer.

• Screens are above the line. Kitchen display monitors hang where they can be seen, not where they can be reached.

• Hygiene is the job. A shared touch surface between the grill and the bagging station is a surface everyone would rather not have.

How does gesture complement voice?

Voice takes the order at the speaker post and hands it to the kitchen. From there, the actions are discrete and repetitive: bump, recall, park, next. A raised hand and a mapped gesture do that from the station, with no headset traffic and no contact. Voice for the customer, gesture for the crew, and the existing kitchen display system in the middle unchanged.

AirTouch runs on the computer driving the display. Its output is standard input, so the kitchen display software, the order confirmation screen, or a menu board controller sees the same events it would from a bump bar or a touchscreen. There is no SDK to integrate and no change to the point of sale.

Where does it fit in a restaurant?

• Kitchen display stations. Bump and recall by gesture. Map one gesture per action and keep it simple.

• Expo and bagging. Mark orders complete with hands full of bags.

• Drive-thru order confirmation boards. A customer confirms or corrects from the car window without touching a screen in the weather. The board sits behind glass, which a camera tracks through, within reason.

• Self-order kiosks in the lobby. Covered in the touchless kiosk and best touchless kiosk software.

• Prep and recipe displays. Advance a build sheet with flour on your hands, the commercial version of scrolling recipes with messy hands.

What does the kitchen need?

One camera per station with a clear view of where the crew member stands, delivering 20 frames per second or more, and the Windows, Linux, or Android compute already driving the display. Camera video is processed on that computer and does not leave it. Gloves in food service colors track well, and saturated colors use the Support Color Glove setting. Patented intent detection separates the crew member at the station from people moving behind them in most setups, and a lunch rush is the right time to test it.

What are the limits?

Steam, heat shimmer over a grill, and heavy grease on a lens degrade tracking, so camera placement and a wipeable housing matter. Some stations may call for a specialized camera. Through our partnership with Orbbec, AirTouch also takes advantage of 3D depth sensing and IR cameras when a deployment calls for them. See the Orbbec page. AirTouch is software, so the camera and computer must meet your own requirements for the station they sit in.

Where to start

Start with one station in one store, the one where the bump bar is most abused. Contact us with a photo of the line and the kitchen display system you run. Operators and integrators can start a free trial on a PC with a webcam and drive the display software today.

Related: touchless screens for food plants and warehouses, gesture control for Android displays and kiosks, gesture control for digital signage, and gesture control and hand tracking software.