Computers

Prototype Microsoft keyboard recognizes hand gestures

Prototype Microsoft keyboard recognizes hand gestures
A user zooms in on a map by pinching their fingers on the keyboard
A user zooms in on a map by pinching their fingers on the keyboard
View 3 Images
Some of the gestures recognized by the keyboard
1/3
Some of the gestures recognized by the keyboard
Each sensor node peeks out from between the regular key caps, tracking the movement of the user's hands as they touch the keyboard or hover above it
2/3
Each sensor node peeks out from between the regular key caps, tracking the movement of the user's hands as they touch the keyboard or hover above it
A user zooms in on a map by pinching their fingers on the keyboard
3/3
A user zooms in on a map by pinching their fingers on the keyboard
View gallery - 3 images

Gesture recognition devices may indeed offer more functionality than is possible using just a keyboard and mouse, but in order to use them, users have to lift their hands up and away from that keyboard. A team at Microsoft Research decided to address that problem, and created a prototype mechanical keyboard that recognizes hand gestures performed on or immediately above the keys.

The Type-Hover-Swipe keyboard features a 16 x 4 array of infrared proximity sensors embedded in a printed circuit board. Each sensor node peeks out from between the regular key caps, tracking the movement of the user's hands as they touch the keyboard or hover above it.

Each sensor node peeks out from between the regular key caps, tracking the movement of the user's hands as they touch the keyboard or hover above it
Each sensor node peeks out from between the regular key caps, tracking the movement of the user's hands as they touch the keyboard or hover above it

Although the array only manages a resolution of 64 pixels, it operates at high frame rates – above 300 hertz. That's enough for it to detect rapid movement, thus allowing the keyboard's machine learning algorithm to recognize a number of both static and dynamic "motion signatures."

These include swiping up, down, left and right; changing applications by hovering the hand in place; pinching to zoom; pointing for index up and down; and even a steering wheel gesture that allows users to control an onscreen car in a racing game.

More details on how the keyboard learns and recognizes gestures are available in the video below. There's currently no word on possible commercialization of the technology.

Source: Microsoft Research

Type-Hover-Swipe in 96 Bytes: A Motion Sensing Mechanical Keyboard

View gallery - 3 images
2 comments
2 comments
Daniel Gregory
From the company that brought you the Kinect, (a camera device able to recognize hand gestures, body movements, which individual is using it, etc) comes a new device which does even less, and is more frustrating to use.
Rann Xeroxx
What's funny is the patent filed by Apple for this tech and gestures when MS has had this type of tech in an actually demo product.
I can see this being very useful for newer 8/9" Windows tablets that do not have room for a touchpad on a detachable keyboard.