Good Thinking

Sign language-to-speech translating gloves take out Microsoft Imagine Cup 2012

Sign language-to-speech translating gloves take out Microsoft Imagine Cup 2012
The EnableTalk gloves feature a variety of sensors that allow the system to recognize signs that are then translated into speech
The EnableTalk gloves feature a variety of sensors that allow the system to recognize signs that are then translated into speech
View 2 Images
The EnableTalk gloves feature a variety of sensors that allow the system to recognize signs that are then translated into speech
1/2
The EnableTalk gloves feature a variety of sensors that allow the system to recognize signs that are then translated into speech
quadSquad team members Stepanov Anton and Osika Maxim demonstrate the EnableTalk system at the 2012 Imagine Cup (Photo: Microsoft)
2/2
quadSquad team members Stepanov Anton and Osika Maxim demonstrate the EnableTalk system at the 2012 Imagine Cup (Photo: Microsoft)

Since beginning in 2003, the Microsoft Imagine Cup has tasked students the world over with developing technology aimed at solving real-world problems. In this, its 10th year, students were asked to build their project around a specific Millennium Development Goal (MDG), with the finals held this month in Sydney, Australia. The winners have just been announced and beating out teams from 75 countries to claim first place (and US$25,000) in the Software Design category was the Ukraine’s quadSquad with their EnableTalk gloves that translate sign language into speech in real time.

Aiming to extend the communication capabilities of those with hearing and speech disabilities after interacting with hearing and speech impaired athletes at their school, the quadSquad team set out to develop a way for those who know sign language to more easily communicate with those who don’t. Their solution includes a hardware component – the gloves fitted with various sensors – and a software component – which translates the hand signals into speech in real time.

quadSquad team members Stepanov Anton and Osika Maxim demonstrate the EnableTalk system at the 2012 Imagine Cup (Photo: Microsoft)
quadSquad team members Stepanov Anton and Osika Maxim demonstrate the EnableTalk system at the 2012 Imagine Cup (Photo: Microsoft)

Although the software was developed under Windows Phone 7, the team was forced to turn to the older Windows Mobile platform for their entry because Windows Phone 7 doesn’t provide developers access to the Bluetooth stack, which is how the gloves communicate wirelessly with a mobile device running the translation software.

The gloves themselves are fitted with an accelerometer, gyroscope, compass, and 15 flex sensors along the fingers, thumb and palm that determine the position of the glove in space. Data from the sensors is relayed to a controller on the back of the glove that is powered by a rechargeable battery. The battery can be recharged via USB, while a small solar panel also helps extend the intervals between charging. Data from the glove is then transmitted via Bluetooth to a mobile device that translates the signs into speech using the Microsoft Speech API and Bing API.

The team anticipates selling each glove for around US$200 if they can get them to mass production. If they do manage to get them to market, they would sell the system with a library of standard gestures. However, with sign language varying greatly around the world, the team says users can teach the system new gestures and modify existing ones.

Congratulations to the quadSquad team as well as the more than 350 students from 75 countries who made it to the finals of the competition.

The team's EnableTalk promo video can be viewed below.

Source: Microsoft Imagine Cup, EnableTalk, TechCrunch

5 comments
5 comments
Alastair Carnegie
Let's go back to Horse and Cart Technology...Why Not? And, is this Gizmo powered by a steam engine? with a back-pack full of glass thermionic valves? that would be much more 'Retro-Funky!'
William Rawls
Tie this into a language translator and you have the ability to speak in any language while only knowing sign language.
Travis Casey
Unfortunately, the article is inaccurate. The gloves do not "translate sign language" -- instead, the user does what is called "finger-spelling", making a different sign for each letter they want to communicate, and the system puts those letters together into words and speaks them.
Someone who has the manual dexterity to finger-spell could just as well type their message, though -- and at that point, any text-to-speech synthesizer can read it out.
Actual sign language is a very different thing from finger-spelling.
Roger Williams
I think the earlier post is incorrect. In the video, they show the sign for sun and it is accurately translatedfrom the sign, not just individual letters as in the "hello" demonstration. Looking at the video, it does appear that it can translate signs, although the word order would still be confusing to a spoken English user since ASL grammar (as well as the grammars of the other signed languages) has its own grammar not based on the spoken language used in the same country.
kellory
My daughter is learning sign language. This is headed in the right direction, but needs to be able to interpret gestures as well. Finger spelling is only a small part of sign language. Much like being able to spell words, but not build sentences. In general, only things without ready gestures get spelled out. Also, Sign language has slang, and short cuts. This is crude and slow, but has potential. God luck getting it on an airline though.