Computers

Type in mid-air with a Leap Motion and DexType

Type in mid-air with a Leap Motion and DexType
DexType and Leap Motion combine for gesture-based typing
DexType and Leap Motion combine for gesture-based typing
View 5 Images
DexType and Leap Motion combine for gesture-based typing
1/5
DexType and Leap Motion combine for gesture-based typing
DexType will work on any device that can run the Chrome browser and support the Leap Motion system
2/5
DexType will work on any device that can run the Chrome browser and support the Leap Motion system
The user interface has a grid along the bottom that represents a linear arrangement of a QWERTY keyboard
3/5
The user interface has a grid along the bottom that represents a linear arrangement of a QWERTY keyboard
Separate layout screens are included for numbers and symbols, or a user can draw what's needed in the air and DexType will convert it to an onscreen character
4/5
Separate layout screens are included for numbers and symbols, or a user can draw what's needed in the air and DexType will convert it to an onscreen character
DexType for Android is due to appear by late August
5/5
DexType for Android is due to appear by late August
View gallery - 5 images

The news release announcing the availability of the Leap Motion controller and supporting software isn't even cold yet, and the first applications that make use of its gesture recognition capabilities are already making a break for freedom. Asetniop creator Zack Dennis has joined the fray with an alternative to the physical keyboard he's calling DexType. Essentially a Google Chrome browser plugin, the Dex-typist uses mid-air point and poke gestures to select characters from a strip at the bottom of the screen.

Dennis says that DexType will work on any device that can run the Chrome browser and support the Leap Motion system. The user interface has a grid along the bottom that represents a linear arrangement of a QWERTY keyboard (though other layouts are also supported, including Dvorak, AZERTY and QWERTZ).

Using the power of Leap, a typist selects each letter by gesture-bouncing dots on the top of the desired character. DexType doesn't require user precision, treating keystrokes as groups instead of individual letters and then using an algorithm to determine the most likely output, similar in operation to predictive text in SMS messaging. As such, even if your energetic gesturing appears to produce nothing more than gibberish, DexType could convert it into something more meaningful.

By way of example, Dennis says that if a user bounces on the letters n, x, o, p, and l, DexType will guess that you actually meant to type "hello" and offer that word onscreen as an option. Of course, there are occasions where precision typing is an absolute necessity, such as entering passwords, usernames or acronyms. In such situations, character selection can be confirmed by pressing the onscreen selection down a mite deeper than normal.

Separate layout screens are included for numbers and symbols, or a user can draw what's needed in the air and DexType will convert it to an onscreen character
Separate layout screens are included for numbers and symbols, or a user can draw what's needed in the air and DexType will convert it to an onscreen character

The system currently supports 26 languages and separate layout screens are included for numbers and symbols, or a user can draw what's needed in the air and DexType will convert it to an onscreen character. Punctuation is called up using chorded keying, by simultaneously pressing two specific key regions, for example. Gestures can also be used for simple edits, such as sideways swiping to erase a letter, work or line (depending on the length of the motion).

DexType can also be used for mouse- and keyboard-free web surfing, taking care of clicking and scrolling, onscreen typing, link selection and activation, moving backward and forward through browser history, and so on.

The virtual keyboard is currently free-to-download from the Chrome Web Store. After August 7, DexType will be split into two. One will be an ad-supported, free-to-use version and the other a premium (ad-free) flavor for US$4.99.

The video below demonstrates what the system is capable of. The Android app mentioned at the end isn't expected to be available until the end of August.

Source: DexType

View gallery - 5 images
2 comments
2 comments
Sam Sams
Hmm... watched the vid... and that was soooo slow! Defo pass...
dave be
I got mine recently, along with the SDK for making apps on the Leap. After using my Leap for a few days I've got to say that this kind of thing is really -really- not the natural forte' for it.
Anytime you have a need to hold steady and/or make precise movements the Leap will tend to freak a little bit. It knows where your hands are, and it knows the general orientation they have. ..until it doesn't. Then your pointer will go flitting across the display. At some point I'm sure this will be solved.
Until that happens, you'll not that most of the successful apps and demos don't rely on precision. They work off of movent and change of orientation on your hands. As soon as we figure out some apps that work on that paradigm then the Leap will shine.