Science

Digital face-swapping heading for low-budget film-making

Digital face-swapping heading for low-budget film-making
A Harvard computer scientist has created a digital "face transplant" system, that could be used by amateur and low-budget film-makers (Images: Association for Computing Machinery)
A Harvard computer scientist has created a digital "face transplant" system, that could be used by amateur and low-budget film-makers (Images: Association for Computing Machinery)
View 2 Images
A Harvard computer scientist has created a digital "face transplant" system, that could be used by amateur and low-budget film-makers (Images: Association for Computing Machinery)
1/2
A Harvard computer scientist has created a digital "face transplant" system, that could be used by amateur and low-budget film-makers (Images: Association for Computing Machinery)
A Harvard computer scientist has created a digital "face transplant" system, that could be used by amateur and low-budget film-makers (Images: Association for Computing Machinery)
2/2
A Harvard computer scientist has created a digital "face transplant" system, that could be used by amateur and low-budget film-makers (Images: Association for Computing Machinery)

If you've seen the film The Social Network, then you might have wondered about the identical Winklevoss twins - were a real-life pair of twins cast for the roles, or was it a bit of Hollywood magic? Well, it was magic. Although two different actors' bodies were used, their faces both belonged to actor Armie Hammer. After the movie was shot, the body double's face was digitally replaced with Armie's. While such computer-enabled face-swapping trickery has so far been available only to feature film-makers with deep pockets, that could be about to change, thanks to research being conducted at Harvard University.

Computer scientist Kevin Dale has created a digital "face transplant" system, that reportedly works with a single camera, simple lighting, and a regular desktop computer.

The process begins with footage of a supplying "source" actor, and a receiving "target" actor - both of which can be talking. The software proceeds to create 3D models of both of their faces, then subtly alters the pose and speech of the target actor to match those of the source. Needless to say, it works best if both actors are in the same pose, in the same lighting, and saying the same thing.

A Harvard computer scientist has created a digital "face transplant" system, that could be used by amateur and low-budget film-makers (Images: Association for Computing Machinery)
A Harvard computer scientist has created a digital "face transplant" system, that could be used by amateur and low-budget film-makers (Images: Association for Computing Machinery)

On a frame-by-frame basis, the outline of the transplanted face is blended with the skin of the receiving actor - in this way, a flickering seam around the face is avoided when all the frames are run together. Additionally, the software chooses a location on both faces, both for cutting out the one and covering over the other, where the boundary between the two will be the least noticeable.

Ten seconds of finished footage takes about 20 minutes to render, with minimal input from the user.

Not only could Dale's system be used to put one actor's face on another's body, but it could also combine different parts of the same actor's face, shot in different takes of the same scene. If the director liked the way an actor's eyes looked in Take 2, for instance, those eyes could be blended into the actor's face from the otherwise-better Take 4.

Source: New Scientist

10 comments
10 comments
CCTVGUY
This is not new, news and history documentaries have been using this technology for years now.
Serge Richard
Jabelon, The article does not say it\'s a NEW technology, It merely states it\'s now available (heading) for cheaper ... Thus; \"Digital face-swapping heading for low-budget film-making\"
leondk
I had the idea years ago that a computer could digitally replace one body with another one, after all images whether moving or not are just pixels but I had not way of bringing it to reality. Its good to see its beginning to happen. I believe what will eventually see are new movies starring dead actors or actors in an old movie being replaced with other dead actors. Imaging James Dean bringing out a new movie or Jimmy replacing Kurt Russell as Stuntman Mike in the film Deathproof. The possibilities are truly endless, gone with the wind, Die hard with Marlon Brando?
christopher
I\'ve been wondering about this for years - how long before lawsuits turn up because some hacker \"stole\" the image of a famour actor, and used it like this to make a new movie :-)
Dave Andrews
Christopher, that can be done, but not with this software. The \"new\" face isn\'t programmable, it just says what it always said. You could take Brad Pitt\'s face and put it on my body (shudder), but you couldn\'t make it say or do new things. It would take completely different software to make that face say and do whatever you wanted. Frighteningly enough though, in the highest echelons of filmmaking, we\'re just about there.
Serge, you\'re completely right. There\'s no claim here that this is totally new, simply that someone has FINALLY found a way to make it affordable to those of us who don\'t have multi-million dollar budgets or massive programming skills.
Paul Perkins
I wanta look like Olivia Wild...
Robert Hoge
Wonderful. Now they can put anybody\'s face on crooks in surveillance videos. Don\'t we all feel safer now?
Kirill Belousov
Makes me think about \"Total Recall\" 8)))
windykites
Looking at the photos of the two girls, the end results just seem to show the girl with a slightly different hair style. I must be missing the point of this. Robert: This would be good for framing crooks, to nab them. I think that is what you meant.
[Actually, the Source girl's face is on the Target girl's head, placed below the Target girl's existing hair -Ed.]
Robert Hoge
Windykites1: No, actually, I was thinking of how the cops can now frame uncomfortable types they want to get rid of. With thousands of surveillance cameras recording crimes every day, they could put my face on a criminal committing a violent crime on camera at a time when they know I have no alibi, and haul me off to prison for as long as they please. As in The Running Man, just more elegant.