Top 100: The most desirable cars of all time

New software dramatically simplifies addition of objects to photos

By

December 12, 2011

Shadows from set geometry are accurately depicted on added objects

Shadows from set geometry are accurately depicted on added objects

Image Gallery (14 images)

For more than twenty years, the software program Photoshop has been the industry standard for seamlessly manipulating images, especially the removal of unwanted items like blemishes, wires and telephone poles. When it comes to adding something to a photo, however, the process is still rather involved. Now, a team from the University of Illinois at Urbana-Champaign (UIUC), led by PhD candidate Kevin Karsch, is presenting a new algorithm at Siggraph Asia 2011 that promises to radically simplify the procedure of believably incorporating new or "synthetic" objects in still shots and the results are quite impressive.

To appreciate the elegance of the UIUC team's contribution, it's helpful to understand just what an improvement it is over current techniques. Look at any effects-heavy feature film and you'll see instances where computer-generated objects, be they giant robots, dinosaurs or futuristic vehicles, were added to previously-shot footage (also called a background plate in the Biz). In order to sell the illusion that the added objects were there in the first place, technicians typically need to gather copious amounts of on-set data (e.g. lens height and focal length, size of on-set objects and direction/color temperature of visible light sources, etc.) from which they recreate a virtual environment. When this is done poorly, it's a dead giveaway that the added object is fake and was inserted after the original image or footage was shot.

Multiple added objects can interact with one another

The genius of the Karsch team's approach lies in what the user doesn't have to provide in order to create believable additions. (Currently, the software only handles still images, but plans are in the works to tackle film and video sequences before long). Starting with a still photo about which nothing is known, novice users have been able to add virtual subjects in just minutes with results good enough to fool experts. That's great news for graphic artists, but maybe not so thrilling for folks like crime scene investigators who'll soon have to doubt the veracity of every object in frame!

Schematic of work flow

To pull off this digital sleight-of-hand, the process is broken up into intuitive stages, the first being the creation of 3D geometry from 2D data. "The main idea is we want to take our picture and we want to get a 3D model of the scene from it," Karsch explained. "We first apply an automatic estimation procedure which tries to find the boundaries of the scene - it tries to find the walls, the ceiling, the floor." Since that's occasionally inaccurate, the software also includes an interface that allows the user to refine the estimate.

"With two vanishing points, we can recover the focal length of the camera, and then using the rest of the input we can also recover a rough representation of the 3D geometry of the scene: where the floors are at, where the walls are at and where the ceiling is at," Karsch added.

Example of various types of user input to refine results

Next, the Lighting Stage requires the user to identify light sources in the image, whether in frame or not. In the traditional computer graphics approach, lighting sources and characteristics are identified by placing a chrome reference ball in front of the camera. The UIUC team's approach eliminates the need for that and a host of other steps as it calculates a decent representation of the actual lighting conditions.

If the object to be added will land behind another opaque item, the Occluding Object Stage (in which the user basically scribbles on the objects that will block part of the object to be added) comes next. Also called a spectral matting procedure, this generates an object mask which, when applied, creates the illusion that the added object is behind the masked one.

Enabling beginners to generate professional results with minimal time and effort is sure to make the Karsch team's algorithm a big hit in the graphics world. Judging from the sample images provided from the team's paper, the approach adeptly handles just about every lighting condition conceivable, even for transparent and shadowed subjects. For those of you itching to get your hands on it, plans are in the works to create an online version that anyone can use. In the meantime, it seems we'll just have to make do with run-of-the-mill reality!

Source: Kevin Karsch

Check out the video below for a look at the software in action:



About the Author
Randolph Jonsson A native San Franciscan, Randolph attended the U.S. Naval Academy at Annapolis, Maryland before finding his way to the film business. Eventually, he landed a job at George Lucas' Industrial Light + Magic, where he worked on many top-grossing films in both the camera and computer graphics departments. A proud member of MENSA, he's passionate about technology, optimal health, photography, marine biology, writing, world travel and the occasional, well-crafted gin and tonic!   All articles by Randolph Jonsson
10 Comments

Looks amazing but what is the name of this new software? We have the exhibition details and the developers name, but not the product name!

And price? (Holding my breath here)

Janet Haines
13th December, 2011 @ 03:27 am PST

Not gonna lie, but it does look kinda fake.

Each image put into the stills is a 3d model. Of which you can alter easily for lighting and shading and drop ontop of photos.

Not to mention it opens with a video despite the article mentioning they haven't been done yet....

Look keyshot up ;)

Mark Penver
13th December, 2011 @ 05:01 am PST

I don't need this for a second, but it's so amazing that I will definitely buy one as soon as it is released, just for the hell of having fun with it.

Krister Knutars
13th December, 2011 @ 05:57 am PST

Hmmm....so now we can't believe ANYTHING we see. Amazing tech. I'm sure it will be put to nefarious use immediately! LOL

TheRogue1000
13th December, 2011 @ 08:12 am PST

Pretty awesome. Now they just need to make it work for video and make a plug in for Premiere and/or After Effects... Or simply sell the tech to Adobe so they can add it.

Dave Andrews
13th December, 2011 @ 08:30 am PST

...thus making it easier for criminals to fake documents or photographic evidence. Before this is released, they should develop the algorithms to detect inserted objects just as easily.

William H Lanteigne
13th December, 2011 @ 08:40 am PST

Guys in future beware what you see in any picture... digital or not!

Leong Hee Chan
13th December, 2011 @ 11:26 pm PST

First step on the road to criminals producing fake incriminating videos of influential and wealthy people to use for blackmail. Which of course the gullible sorts who'll believe absolutely anything - except for the truth - will believe in spite of proof the videos are fake.

Gregg Eshelman
13th December, 2011 @ 11:43 pm PST

this was demo'd (? leaked?) months ago ..

at least now we know the team responsible for it ..

tkj
14th December, 2011 @ 04:43 pm PST

I have been doing this for quite a while now using two picture editing programmes. See egsample in videos at youtube at https://www.youtube.com/watch?v=6GTrIyvbcK8

Donald Thomas
14th December, 2011 @ 07:26 pm PST
Post a Comment

Login with your gizmag account:

Or Login with Facebook:


Related Articles
Looking for something? Search our 29,852 articles
Recent popular articles in Digital Cameras
Product Comparisons