Robotics

Mask-bot takes a new approach to giving robots a human face

Mask-bot takes a new approach to giving robots a human face
Mask-bot uses a small projector to beam the image of a face onto a transparent plastic mask
Mask-bot uses a small projector to beam the image of a face onto a transparent plastic mask
View 7 Images
Mask-bot could be used for video conferences
1/7
Mask-bot could be used for video conferences
Mask-bot displays realistic three-dimensional heads
2/7
Mask-bot displays realistic three-dimensional heads
Mask-bot's projected face is bright enough to be seen in daylight
3/7
Mask-bot's projected face is bright enough to be seen in daylight
Dr. Takaaki Kuratate with Mask-bot
4/7
Dr. Takaaki Kuratate with Mask-bot
Mask-bot can display male or female faces
5/7
Mask-bot can display male or female faces
Mask-bot puts a human face on robotics
6/7
Mask-bot puts a human face on robotics
Mask-bot uses a small projector to beam the image of a face onto a transparent plastic mask
7/7
Mask-bot uses a small projector to beam the image of a face onto a transparent plastic mask
View gallery - 7 images

While great strides have been made in the development of humanoid robots, such as Honda's ASIMO, giving robots a human face with natural expressions and movement has proven a difficult task. While some look to create lifelike faces and expressions with motors under artificial skin replicating the function of facial muscles, German and Japanese researchers have joined forces to come up with a different solution called Mask-bot that sees a 3D image of a human face projected onto the back of a plastic mask.

The Mask-bot displays realistic three-dimensional heads using a projector positioned behind a transparent plastic mask. The projector beams a human face onto the back of the mask to create realistic features that can not only be seen from various angles, including the side, but can also be changed on demand.

Dr. Takaaki Kuratate compares the Mask-bot approach to that used to project faces onto sculptures in Disneyland's Haunted Mansion ride. However, these images are projected from the front whereas the Mask-bot uses rear-projection. This means there is only a 12 cm (4.7 in) gap between the high-compression, x0.25 fish-eye lens with a macro adapter used to project the image and the face mask.

Mask-bot can display male or female faces
Mask-bot can display male or female faces

To ensure the projected image was also bright enough to be viewed in daylight, the team used a projector that is strong and small and they also gave the inside of the plastic mask with a coat of luminous paint.

In developing Mask-bot the team also faced the challenge of projecting a moving image onto the mask instead of just a static photo without requiring a video image of the person speaking. To achieve this they use a program that converts a normal two-dimensional photo into a correctly proportioned projection for the three-dimensional mask. Additional algorithms are then used to provide the facial expressions and voice.

The talking head animation engine developed by Takaaki Kuratate to replicate facial expressions filters an extensive series of face motion data from people collected by a motion capture system. It then selects the facial expressions that best match the specific sound - or phoneme - when it is being spoken. The computer extracts a set of facial coordinates from each of the selected expressions that is can then assign to any new face. Emotion synthesis software is responsible for delivering the visible emotional nuances, to indicate when someone is happy or sad, for example.

Mask-bot's current understanding of the spoken word is limited to listening and making appropriate responses as part of a fixed programming sequence. It can also realistically reproduce content typed on a keyboard in English, Japanese, and soon German. A text-to-speech system converts the text to audio signals, producing a male or female voice, which can then be set to quiet or loud, happy or sad, at the touch of a button.

While the researchers say the technology used in Mask-bot may one day give robots a human face, they anticipate it could appear sooner in avatars for video conference participants.

Mask-bot displays realistic three-dimensional heads
Mask-bot displays realistic three-dimensional heads

"Usually, participants are shown on screen. With Mask-bot, however, you can create a realistic replica of a person that actually sits and speaks with you at the conference table. You can use a generic mask for male and female, or you can provide a custom-made mask for each person," explains Takaaki Kuratate, who says such systems could also be used to provide companionship for older people.

The researchers are already working on Mask-bot 2, in which they aim to see the mask, projector and computer control system all contained inside a mobile robot. Although the first Mask-bot prototype cost just under EUR3,000 (approx. US$4,125), they estimate the successor model should cost around EUR400 (approx. US$550).

Mask-bot was created through a collaboration of the Technical University of Munich's (TUM) Cognition for Technical Systems (CoTeSys) Cluster of Excellence and Japan's National Institute of Advanced Industrial Science and Technology (AIST).

Here's some video of Mask-bot in action.

Mask-bot - life size talking head robot head

View gallery - 7 images
12 comments
12 comments
Jason Catterall
Madame Tussauds were doing something very similar to this in their waxwork museum in London almost 35 years ago. Not exactly new!!!
SuperGreenGuy
\"Donna Noble has left the Library. Donna Noble has been saved.\"
Pat Kelley
Disneyland has had this technology in their haunted house experience since the late 1950s. Of course it wasn\'t interactive or computer-driven, but the effect was better quality. The video shows noticeable lag between the head movement and image tracking, and the \"animation\" looks more like the cheesy animation of the old Speed Racer cartoons, with a moving mouth on a still face image. Most humans are uncomfortable with a human mimic that obviously isn\'t human, so a more practical effect would be to use obviously non-human robots with just the essentials of facial movement to convey appropriate emotion. That\'s much easier.
gizmag_reader
The new (1994) floating Madame Liotta at Disney Haunted Mansion uses a coherent fiber bundle, starting at a fixed LCD projector and ending with a custom ~180 degree short throw lens. The mask is vacuum formed and internally coated to be a high-gain rear projection screen. The original 1963 Liotta Head was of course front projected, using a 16mm projector with a film loop cabinet.
warren52nz
Chinese Robotic Research: Making robots look human to help around the house and keep the elderly company
American Robotic Research: Making four legged robots that can help carry military gear into battle (ala Big Dog)
Hmmmm.......
Australian
Can anyone explain what the fascination is with making robots human-like? I seriously just don\'t get it.
M. Report
A bit bulky; They should develop a mask with a surface which acts as the display.
Gavin Greaves
Robots themselves should not be made to look human it will bring a plethora of problems on human society. There is many documents both fictional and non-fictional on the evolution of robots that\'s show the dangers and pitfalls of making robots a/ more human in looks and actions especially decision making b/ the ability to learn enough to become self aware.
Although we maybe a long way from the path of the Terminator or the Dune Butlerian Jihad I just don\'t know that there are the right checks and balances to stop us falling in that hole while randomnly trying to see how far we can go instead pf having a planned destination for robots in a human world.
I see Asimo as a more perfect humanoid robot in that it is developing only a set of physical functions and deliberately not trying to take on human characteristics. We should never ever build a robot brain.
Tysto
What is this for?! Stop trying to create robots that look human! We already have plenty of people!
It\'s like when film makers make an animated film about people who don\'t do anything that real people can\'t do. I\'m lookin\' at you, Robert Zemeckis....
Dave Andrews
Disney has been doing this for many years to great effect.
Load More