Novel system guides the blind by turning images into music


July 5, 2012

EyeMusic is an experimental system for the blind, that translates visual information into musical "soundscapes"

EyeMusic is an experimental system for the blind, that translates visual information into musical "soundscapes"

Sensory substitution devices work by converting one type of sensory input into another – examples would be systems such as CASBLiP and EYE 21, which allow the blind to “see” by assigning sounds to images. Now, a team of researchers at the Hebrew University of Jerusalem have created a similar wearable device, known as EyeMusic. It “employs pleasant musical scales to convey visual information,” and could one day help the visually impaired more easily perform tasks that the rest of us take for granted.

The device consists of a pair of glasses with an attached video camera, which are wired to a pair of headphones – those headphones transmit sound into the user’s scalp, as opposed to directly into their ears.

The camera makes scans of the scenery in front of the user, with each scan going from left to right. These scans are translated into “soundscapes,” which are short pieces of music played back through the headphones. The higher that a pixel is vertically located in the scanned image, the higher of a musical note will be used to represent it in the soundscape. The farther to the right that a pixel is within the scanned area, the later that it will be heard within the soundscape – an auditory cue marks the beginning of each scan/soundscape.

The brightness of each pixel is translated into how loud its sound is. Different colors are represented by different instruments, with blue becoming a trumpet, red a reggae organ, green a reed, yellow a violin, and white being heard as vocals. Black is represented by silence.

Although some of the sample soundscapes may sound rather ... complex, a group of 18 blind-folded sighted test subjects reportedly learned to use EyeMusic relatively quickly – in some cases, less than half an hour. In a subsequent test, they were able to use the device to locate a white square displayed on a tablet computer.

Down the road, it is hoped that it could be used by the blind in everyday activities such as selecting produce at the grocery store, or even playing video games.

Source: IOS Press

About the Author
Ben Coxworth An experienced freelance writer, videographer and television producer, Ben's interest in all forms of innovation is particularly fanatical when it comes to human-powered transportation, film-making gear, environmentally-friendly technologies and anything that's designed to go underwater. He lives in Edmonton, Alberta, where he spends a lot of time going over the handlebars of his mountain bike, hanging out in off-leash parks, and wishing the Pacific Ocean wasn't so far away. All articles by Ben Coxworth

CASBLiP and EYE21 are not yet generally available. The most wide-spread soundscape-based sensory substitution system for the blind is currently The vOICe (from )

Peter Meijer

Well done! A techy variant of Synethsia ...

Jansen Estrup

I just heard a blue flower! That is awesome.

Post a Comment

Login with your Gizmag account:

Related Articles
Looking for something? Search our articles