Gaming Input Compared: Gesture, Mouse, or Gamepad?

For competitive shooters and fighting games, a mouse and a gamepad are still the best inputs you can buy, and nothing in this post argues otherwise. Gesture control is a different kind of controller. AirTouch reads your hands through a standard RGB camera and sends the mouse and keyboard input games already accept, so you bend air, cast spells, fling birds, and steer from the couch with your hands and no device in them. It wins on the living room, the big screen, casual and motion-style games, and the simple pleasure of using the Force. Here is how it stacks up against a joystick, a mouse and keyboard, a programmable mouse, and a gamepad.

What is each input good at?

• Mouse and keyboard. The precision benchmark. Sub-pixel aiming, dozens of bindable keys, and decades of games designed around it. The cost is a desk, a chair, and a posture.

• Programmable mouse. A mouse with extra buttons and on-board macros. Everything a mouse does, plus a dozen shortcuts under the thumb. Still a desk input.

• Gamepad. Analog sticks, triggers, and rumble. Built for the couch and for games that want smooth, bounded movement rather than fast pointing. Aiming is slower than a mouse, which is why console shooters add aim assist.

• Joystick. The right tool for flight and space sims, where the genre is designed around a stick, throttle, and hat switches. Narrow outside that genre.

• Gesture control. Your hand is the pointer and the gesture is the button. Nothing to hold, charge, or hand to the next player. Precision sits below a mouse and above a gamepad for pointing, and its real advantage is where you can play and how it feels.

How does gesture control work in games?

AirTouch gesture control and hand tracking software runs on Windows, Linux, and Android, with macOS on the way, and treats a game like any other application. An open palm summons the cursor, an index and thumb pinch clicks and drags, and 15 gestures map to any key, mouse button, or shortcut, saved per profile, so one profile can be a strategy game and another a rhythm game. Inference runs in about 50 milliseconds. Because the output is standard input, there are no mods, no SDK, and no game-specific support: titles built for a mouse respond to your hands out of the box. In touch mode the display behaves like a touch screen, which suits games designed for tablets. The gestures and mapping guide walks through building a profile.

Where does gesture control beat the others?

• The living room. Plug a laptop into the TV, point the webcam at the couch, and the big screen is playable from across the room with nothing in your hands. A mouse needs a desk; a gamepad needs charging and pairing; a raised hand needs neither.

• Motion-style games. Anything that once wanted a Kinect or a motion controller. Casting, swinging, throwing, and steering map naturally to hand movement, as covered in Kinect alternatives for gesture control.

• Casual and party games. Angry Birds, Candy Crush, puzzle and physics games, and anything where a flick of the fingers is more fun than a click. Players rotate by raising a hand, not by passing a controller.

• Arcades, exhibits, and showrooms. Public machines where a controller would break, walk off, or need wiping. See gesture control for immersive displays.

• Spectacle. Pinching a spaceship out of the air in front of guests is a different experience from pressing a button. That matters more than it sounds.

Where do the traditional inputs win?

• Competitive aiming. A mouse on a hard surface is faster and more precise for a flick headshot than any camera-based pointer, and that gap will not close soon.

• Analog movement. Gamepad sticks give smooth, pressure-sensitive control for driving and third-person movement. AirTouch sends key presses and mouse movement, not analog stick values.

• Very long sessions. Holding a hand up for an hour of raids is tiring. Gesture sessions run best in bursts, which is also how casual games are built.

• Dark rooms. Hand tracking wants light on the player. Some setups may call for a specialized camera.

• Many simultaneous inputs. A keyboard holds eight keys at once. A hand holds one gesture.

Which should you choose?

Can you mix them?

Easily. AirTouch does not take over the mouse; it adds another source of the same input. Keep the keyboard for movement and use a gesture for a spell or a reload, or keep the gamepad for the game and use a raised hand to pause and navigate menus from the couch. Pausing a movie with a raised hand is the same trick, covered in controlling Netflix and YouTube with hand gestures.

Where to start

Start a free trial on the PC you already play on, open a casual game, and fling something. The Windows install guide gets you to a moving cursor in about five minutes, and the camera setup guide covers framing the couch.

Related: gesture control on a laptop, control your computer with hand gestures and a webcam, the Minority Report computer setup, and gesture control and hand tracking software.