Augmented reality robot fleshes out virtual characters


February 7, 2013

The "virtual humanoid" prototype will allow you to physically interact with a computer-generated character seen through a head-mounted display

The "virtual humanoid" prototype will allow you to physically interact with a computer-generated character seen through a head-mounted display

Image Gallery (10 images)

Different Dimension Inc., a Japanese start-up company, will begin marketing an Augmented Reality robot doll that can take on the appearance of any virtual character. Essentially, a computer-generated image is projected in real-time onto a robot (covered in green screen material) that is viewable through a head-mounted display (HMD). It's a spin-off of a bizarre research project led by Michihiko Shoji at the Yokohama National University and the Japan Science and Technology Agency, originally developed in 2006.

The so-called "virtual humanoid" is an example of "mixed reality" – a technology which takes augmented reality to the next level by giving virtual objects a physical counterpart in the real world. Video images from the user's perspective – taken from the camera mounted to the HMD – are combined with a computer-generated character that moves in synch with the robot. The user can then touch and physically interact with the virtual character through the robot.

The original prototype, called U-Tsu-Shi-O-Mi, presented several challenges. It was a costly life-sized robot, and its green padding could be seen around the edges of thinner virtual characters. Additionally, the system only played back recorded voices, so it could not actually converse with the user. The new version has been shrunk to 60 percent of its original size (a height of 13.7 inches / 35 cm), and is only the top half of the body. It features a slimmer profile, which matches up better with the virtual characters, and the researchers have paid special attention to the texture of the green screen material, so that it feels more natural to the touch.

The new prototype can tilt and turn its neck and move its arms at the shoulders, elbows, and wrists. The servo motors which power each joint contain position sensors which read the joint angles – data which is fed to the software program controlling the virtual character – thereby synchronizing their movements. It's similar to the system adopted by the Qumarion, a puppet which can be posed to animate virtual characters.

The actual software used is called MMDAgent, which creates an interactive 3D anime-style character that can carry on conversations using speech recognition and a synthesized voice.

The research team is exploring possible applications for the technology, including education, training, and video games – ignoring the naughtier uses that might immediately spring to mind. Interested parties will be able to pre-order the virtual humanoid in March. It will be priced between ¥450,000 - ¥500,000 (US$4,800 - $5,300). The company, which will soon launch its website, expects to sell 120 units the first year, followed by 180 and then 270 units in the following years.

You can see the original U-Tsu-Shi-O-Mi and the MMDAgent software in action in the following videos.

Source: Wingover (Japanese) via MyNavi (Japanese)

About the Author
Jason Falconer Jason is a freelance writer based in central Canada with a background in computer graphics. He has written about hundreds of humanoid robots on his website Plastic Pals and is an avid gamer with an unsightly collection of retro consoles, cartridges, and controllers. All articles by Jason Falconer

That all seems like fine research I suppose, and who knows where it may lead, but it seems to me, that if a user doesn't require true physical interaction, working out the software to overlay 'social' interactive characters/images within a head-mounted display (perhaps with force feedback haptic gloves) would be much cheaper, more practical, easier (relatively) and nearly if not totally as effective.


Now that is some technology. Good Job. I am seriously 1 BIG fan of Augmented Reality. After watching this I have realized in coming days we will see much more and much better things than this. My first experience of Augmented Reality was because of AdStuck, in India. A big thanks to them as well.

Post a Comment

Login with your Gizmag account:

Related Articles
Looking for something? Search our articles