Virtual Reality

VR headset that relays facial expressions to the virtual world

VR headset that relays facial expressions to the virtual world
Veeso is claimed to be the world's first VR headset that tracks a user's facial expressions and conveys them in real time to digital avatars
Veeso is claimed to be the world's first VR headset that tracks a user's facial expressions and conveys them in real time to digital avatars
View 3 Images
Veeso is claimed to be the world's first VR headset that tracks a user's facial expressions and conveys them in real time to digital avatars
1/3
Veeso is claimed to be the world's first VR headset that tracks a user's facial expressions and conveys them in real time to digital avatars
The facial tracking works through two infrared cameras inside the headset: one between the eyes, tracking eyebrows, pupils and how open or closed the eyelids are, and the second hanging off the bottom, taking in the jaw, lips and mouth
2/3
The facial tracking works through two infrared cameras inside the headset: one between the eyes, tracking eyebrows, pupils and how open or closed the eyelids are, and the second hanging off the bottom, taking in the jaw, lips and mouth
Veeso is focusing on social apps and games, such as poker and charades, more than traditional gaming
3/3
Veeso is focusing on social apps and games, such as poker and charades, more than traditional gaming
View gallery - 3 images

VR hardware is already capable of tracking your head, your hands, your eyes and in some cases, your feet, but Veeso is claimed to be the first VR headset to capture your face and transmit your expressions – and as a result, your emotions – onto a virtual avatar in real time. With it, the company is emphasizing emotional connections through chat apps and social games like poker.

Like the Samsung Gear VR and Google Cardboard, Veeso is a smartphone-based VR headset, which the company claims is compatible with Android and iOS devices. Unlike those aforementioned headsets, however, Veeso has two infrared cameras mounted on it to capture the wearer's facial expressions.

One of these cameras is located between the eyes to capture pupil movements, eyebrows, and how open or closed the eyelids are, while the second hangs off the bottom of the unit, taking in the jaw, lips and mouth. Together, from what we see in the videos, they seem to do a pretty solid job of covering the whole face and mimicking the facial expressions on a digital avatar in real time.

The facial tracking works through two infrared cameras inside the headset: one between the eyes, tracking eyebrows, pupils and how open or closed the eyelids are, and the second hanging off the bottom, taking in the jaw, lips and mouth
The facial tracking works through two infrared cameras inside the headset: one between the eyes, tracking eyebrows, pupils and how open or closed the eyelids are, and the second hanging off the bottom, taking in the jaw, lips and mouth

In most current VR games, you don't usually see your character's face, and if you do, having them copy your expression probably isn't going to add much to the experience. But if we're going to spend time interacting with other actual people in virtual worlds, be it multiplayer games or social applications, which is where a lot of those involved with VR see the technology heading, then being able to see people's real world facial expressions in real time will be key.

Potential applications suggested by the company so far include a kind of VR video call, where people sit together in virtual rooms and have a chat through the mouths of cartoon frogs and squirrels, and more traditional social games like poker and charades. Veeso says it plans to release at least 10 applications and games for the platform in the first six months.

As interesting as the tech sounds, keep in mind that the best Veeso can do content-wise is use existing Google Cardboard content for smartphones. The best mobile VR library is locked behind the Gear VR's Samsung/Oculus walled garden. With that in mind, the product itself may not end up being as enticing as the general prospect of VR facial tracking.

Veeso is currently seeking funding on Kickstarter in order to begin Phase 2 of the product's development. An Early Bird pledge of US$70 will get you a unit, or if you're a developer or interested tinkerer, you can get your hands (and face) on the headset and dev kit for $80. If all goes to plan, the company says the device will ship in December.

The team demonstrates the Veeso headset in the video below.

Source: Veeso

Veeso: The First Face-Tracking Virtual Reality Headset

View gallery - 3 images
2 comments
2 comments
EddieG
VR hardware is moving forward at a glacial pace. Face tracking is not exactly a breakthrough concept. What is holding this tech back? It is the marketing people who call the shots. If marketing can't conceive a thing, it doesn't get out the door. And marketing has a pretty myopic view of technical issues.
VR needs first to be untied from the ball and chain of the cell phone. After that it must be offered to a much greater demographic than adolescent boys and girls.
abedavid
Coupled with CGI,this device could be to visual editing what the Foley Stage is to sound.