Introducing the Gizmag Store

NAVI project turns Kinect into a set of eyes for the visually impaired

By

March 20, 2011

The NAVI project system helps the visually impaired 'see' obstacles

The NAVI project system helps the visually impaired 'see' obstacles

Image Gallery (8 images)

While we've looked at a couple of efforts to upgrade the humble white cane's capabilities, such as the ultrasonic Ultracane and the laser scanning cane, the decidedly low tech white cane is still one of the most commonly used tools to help the visually impaired get around without bumping into things. Now, through their project called NAVI (Navigation Aids for the Visually Impaired), students at Germany's Universität Konstanz have leveraged the 3D imaging capabilities of Microsoft's Kinect camera to detect objects that lie outside a cane's small radius and alert the wearer to the location of obstacles through audio and vibro-tactile feedback.

That's right, I said "wearer" because the system created by Master's students Michael Zöllner and Stephan Huber places the Kinect camera atop the visually impaired person's head thanks to a hard hat, some sugru and a liberal application of duct tape. The image and depth information captured by the Kinect cameras is sent to a Dell laptop mounted in a backpack, which is connected via USB to an Arduino 2009 board glued to a fabric belt worn around the waist.

The vibrotactile wistbelt

The depth information captured by the Kinect camera is processed by software on the laptop and mapped onto three pairs of Arduino LilyPad vibration motors located at the upper and lower left, center and right of the fabric belt. When a potential obstacle is detected, its location is conveyed to the wearer by the vibration of the relevant motor.

A Bluetooth headset also provides audio cues and can be used to provide navigation instructions and read signs using ARToolKit markers placed on walls and doors. The Kinect's depth detection capabilities allows navigation instructions to vary based on the distance to a marker. For example, as the person walks towards a door they will hear "door ahead in 3, 2, 1, pull the door."

The system can read ARToolKit markers placed on walls and doors

The students see their system as having advantages to other point-to-point navigation approaches using GPS – which don't work indoors – and seeing-eye dogs – which must be trained for certain routes, cost a lot of money and get tired.

For their NAVI project, the Universität Konstanz students wrote the software in C# and .NET and used the ManagedOpenNI wrapper for the Kinect and the managed wrapper of the ARToolKitPlus for marker tracking. The voice instructions are synthesized using Microsoft's Speech API and all input streams are glued together using Reactive Extensions for .NET.

Via SlashGear

About the Author
Darren Quick Darren's love of technology started in primary school with a Nintendo Game & Watch Donkey Kong (still functioning) and a Commodore VIC 20 computer (not still functioning). In high school he upgraded to a 286 PC, and he's been following Moore's law ever since. This love of technology continued through a number of university courses and crappy jobs until 2008, when his interests found a home at Gizmag.   All articles by Darren Quick
Tags
3 Comments

Awesome...right up to the point you get a Blue Screen of Death in the middle of the street. Then, not so much...

James Jennings
21st March, 2011 @ 07:02 am PDT

Great project, well done Guys...

I'm just wondering if you could take the kit out of the Kinect box and make something a bit more aesthetic, or is it all fixed to a "motherboard"? (haven't got my hands on a Kinect to hack as yet!)

agulesin
18th April, 2011 @ 07:37 am PDT

Is there any cane which can detect water for blind people

Amardeep Singh
13th October, 2011 @ 09:33 am PDT
Post a Comment

Login with your gizmag account:

Or Login with Facebook:


Related Articles

Just enter your friends and your email address into the form below

For multiple addresses, separate each with a comma




Privacy is safe with us because we have a strict privacy policy.

Looking for something? Search our 26,553 articles