Purchasing new hardware? Read our latest product comparisons

Hand-manipulated objects and transparent displays - the computer desktop of tomorrow?


March 1, 2012

A Kinect-driven prototype desktop environment by the Microsoft Applied Sciences Group allo...

A Kinect-driven prototype desktop environment by the Microsoft Applied Sciences Group allows users to manipulate 3D objects by hand behind a transparent OLED display

Image Gallery (6 images)

A see-through screen, digital 3D objects manipulated by hand, perspective adjustments according to the user's viewing angle - these are the core features of a prototype computer desktop user interface created by Microsoft's Applied Sciences Group. The prototype uses a "unique" Samsung transparent OLED display through which the user can see their own hands to manipulate 3D objects which appear to be behind the screen.

A demo video appears to show a working prototype of a computer markedly different from those we use today. Yes it includes a familiar keyboard and trackpad - but these are placed behind the OLED display. The user simply lifts their hands from these input devices to manipulate on-screen (or more accurately behind-screen) objects, such as selecting a file or window.

The video shows the interface in action with a series of program windows stacked behind one another, with the user selecting the desired program by hand, using the depth of the workspace.

Similar actions are shown to manipulate 3D objects - an exciting prospect not only for gamers, but perhaps also for architects, inventors and engineers working on 3D models. The cherry on the muffin in this respect is the inclusion of head-tracking technology - step to the side to shift your angle of view and your view of the 3D objects on screen will be altered accordingly.

The video certainly poses questions as to the future of human-computer interaction - not necessarily all intentional. Non-touch typists may balk at the idea of a keyboard positioned behind a busy screen (the display may be transparent - that's not to say the information it displays is). Similarly, objects arranged behind one another will necessarily impede view.

But to quibble at the details is rather to miss the point. "This project advances research in current display technologies hoping to provide a more interaction with everyday desktop computing of the future," said Cati Boulanger, a researcher at the Applied Sciences Group. Even with a working prototype, this is a technological what if, not a thou shalt.

See below for a demo video of the prototype in use.

Via Wired UK

About the Author
James Holloway James lives in East London where he punctuates endless tea drinking with freelance writing and meteorological angst. Unlocking Every Extend Extra Extreme’s “Master of Extreme” achievement was the fourth proudest moment of his life.   All articles by James Holloway

Yeah, it all sounds great until you have to hold your arms out in front of you for 8-12 hours in the typical working day for an IT worker.

1st March, 2012 @ 09:18 am PST


I love debbie downers like you. No foresight and only the desire to put down innovation.

Do you think, maybe, that this would not be used by people typing all day and would, instead, be deployed by graphic designers, 3D design engineers, physicists and chemists exploring chemical structures, and a host of other people looking for visualization aids?

Seriously, after seeing this the first and most important thing that came to your mind was, gee, this sucks because IT people who type all day wont use it because of ergonomic restraint?

So sad.

1st March, 2012 @ 12:18 pm PST

The system should use a visor display and ideally will have tactile input gloves as well.

1st March, 2012 @ 12:47 pm PST

Your assumption that only those who type all day would be the ones effected by holding out their arms all day is faulty. In fact, IT technicians would probably rarely, if ever, use this kind of system.

Yes, "graphic designers, 3D design engineers, physicists and chemists exploring chemical structures, and a host of other people looking for visualization aids" would use this system. However even they would be susceptible to the effects of holding their holding out their arms all day. As a short term use, it would probably be of some benefit.

But, as Slowburn commented. it would probably be way more efficient, and less tiring, if the screen were a visor display instead.

2nd March, 2012 @ 09:37 pm PST

Imagine having both this screen AND a normal keyboard and mouse along with a sensor glove and Google Goggles. Then you could decide what you want to use whenever.

I just look forward to the tablet version of these screens - you could use it as a HUD it seems:) Now add the sensors so you can wave a hand to get more info etc and it is a winner:)

3rd March, 2012 @ 08:51 am PST

try reaching behind your monitor for a couple of minute and see how comfy that is. its a bad position to be in for long periods of time.

Denis Klanac
4th March, 2012 @ 09:55 pm PST

i imagine if you used this system, you would orient the monitor a bit closer to you than yours is now.

I also want to point out that research and design is all about creating something (prototype) and testing it to identify problems like the ones being pointed out here. maybe we shouldn't develop any new technology unless it is absolutely perfect from the very beginning?

Andy Piper
11th March, 2012 @ 10:01 pm PDT
Post a Comment

Login with your gizmag account:

Or Login with Facebook:

Related Articles
Looking for something? Search our 31,675 articles