14 ms·
What Makes Tom Hanks Look Like Tom Hanks: Modeling a person from photos
- Jemaclus 11y agoThat's pretty amazing. It won't belong before Tupac really does release a new album...
- vessenes 11y agoThis video is AMAZING and well worth watching. Skip ahead a minute or so if you have a very short attention span. I most enjoyed watching Obama with Bush's facial expressions -- Bush is so expressive that it just looks incredibly wrong on Obama's face -- I wish they would run an Obama speech on Bush's face as well.
- otoburb 11y agoThe last 20 seconds are impressively creepy. I'm thinking this technology demo will be spliced into 5s edits for least one prime time TV show themed around shadowy intelligence agencies by middle of next year. Show writers will be able to incorporate this as a plot enabler for upcoming episodes, assuming they haven't already finished shooting the current seasons.
- LesZedCB 11y agoI don't know if you noticed, but they also ran the voice through some modulation to make Bush's voice sound like Obama as well. It was surprisingly convincing...
- jevinskie 11y agoHonestly I was most impressed by that pairing. The software seemed to "get" Obama's "persona" when while Bush purses his bottom lip over his top lip, Obama had more of a pucker where the lips are merely pressed together. [0] https://www.youtube.com/watch?v=ladqJQLR2bA&feature=youtu.be&t=84 https://www.youtube.com/watch?v=ladqJQLR2bA&feature=youtu.be...
- vessenes 11y agoIt seems like Obama's lip is given a different stiffness than Bush's. I wonder if that comes from software -- e.g. for personality or physical reasons, his lip is more stiff, or something else.
- tetraodonpuffer 11y agohow long before this can be done with the voice as well?
- Nav_Panel 11y agoYou might find the first answer interesting here: http://dsp.stackexchange.com/questions/7833/how-to-mimic-copy-fake-someones-voice http://dsp.stackexchange.com/questions/7833/how-to-mimic-cop... Looks like there's some interesting research on Voice Conversion out there, but I'm not knowledgeable enough to comment on how close we are to seeing it as an actual product. This pdf (from 10 years ago) seems interesting, though. Hmmm: http://nlp.lsi.upc.edu/papers/thesis_helenca.pdf http://nlp.lsi.upc.edu/papers/thesis_helenca.pdf
- jkaunisv1 11y agoThere's a Scottish company that does something like this but it's boutique and requires a large corpus. They recreated Roger Ebert's voice after he lost his jaw but it was only possible because of his vast recorded collection of movie reviews to draw on for phonemes.
- jordache 11y agoi don't get the W vs Obama split screen. The voice was changed slightly to sound like neither individuals... The Obama 3D face fails horribly when it tried to snicker a little bit... This is slightly better than FallOut 4 quality facial animation.
- mistercow 11y ago> The voice was changed slightly to sound like neither individuals I think the audio is just low quality. It wouldn't make any sense for them to do that.
- eyuelt 11y agoI think Obama just never makes that face so it looked really weird.
- stephensonsco 11y agoI think the point is the assembling of random photos/videos into a convincing representation of textures and animation (without devoted hardware, bringing the person in for a scan, etc), not that 'this is bleeding edge visuals'.
- daveguy 11y agoHoly crap. They cross from the uncanny valley right into an Orwell-Stephenson character -- disturbing for its realism.
- msielski 11y ago"...they all came to the realization that what made this place a success was not the collision-avoidance algorithms or the bouncer daemons or any of that other stuff. It was Juanita's faces." Neal Stephenson, Snow Crash. I think in our world as well, VR will really take off when technology like this is applied.
- kingkawn 11y agoIt's amazing how unexpressive the eyes themselves are despite the otherwise uncanny matching of facial expressions. This tech is incredible
- awl130 11y agoi can only describe the biggest thing missing in very unscientific terms: there is no light, or life, in their eyes. whatever that means.
- coke12 11y agoThe eyes look dull because they're using very simple rendering techniques. I'd guess that they could make the eyes much more realistic by adding some simple optical effects like specularity or diffraction (http://www.graphics.cornell.edu/~westin/misc/fresnel.jpg http://www.graphics.cornell.edu/~westin/misc/fresnel.jpg). One interesting thing that I noticed at 1:45 is that visible teeth can really change a person's appearance -- Bush doesn't seem to use his teeth at all when he talks.
- awl130 11y agothe pupils, or my perception of the pupils, don't seem to be focusing or moving naturally.
- rzzzt 11y agoIn some cases, movement of the eyelids when blinking is tracked and reflected in the model, but the texture is not changed to a closed eye quickly enough.
- profinger 11y agoThat becomes really obvious in people like Daniel Craig who has blue eyes that appear to glow.
- graham1776 11y agoDoes anyone think this maybe a new way to immortality? Instead of just photos, soon we will have models that use video input of our family members, superimpose them over AI+visual models(like this), and we will be able to have conversations with them, even though they aren't real?
- danso 11y agoHave you ever watched the show, Black Mirror? That's exactly the subject of one of its episodes: https://www.youtube.com/watch?v=mzSIQxc_KqE https://www.youtube.com/watch?v=mzSIQxc_KqE Also, I think you're thinking too limited...I think if this were perfected, and the physical technology put in place, most people will opt to look like Tom Hanks (or someone else as famous and attractive) than to look like themselves.
- graham1776 11y agoNever seen it, but woah. You're right.
- joshschreuder 11y agoIt makes you wonder about whether one day there will be a law against looking like Tom Hanks, or any celebrity. It would end up making crimes difficult to solve when the suspect is identified as Tom Hanks!
- JabavuAdams 11y agoThis is a killer app for VR. Imagine creating your Oculus avatar from your Facebook selfie collection.
- ollifi 11y agoReposting this paper from earlier this year. Bit similar but with different technique I believe. http://people.mpi-inf.mpg.de/~mzollhoef/Papers/SGASIA2015_RR/paper.pdf http://people.mpi-inf.mpg.de/~mzollhoef/Papers/SGASIA2015_RR... https://www.youtube.com/watch?v=eXVspNUeiWw https://www.youtube.com/watch?v=eXVspNUeiWw
- gotchange 11y agoAmazing! Does this mean that we finally will have (hyper)realistic faces for the FIFA video game series?
- pearjuice 11y agoThe faces in the FIFA video games are already rendered with much more detail than this. This paper is not about detail but about "reconstruct[ing] a controllable model of a person from a large photo collection that captures his or her persona". It is a very specific technique to get to a realistic interactive facial model, not one to get to a very detailed realistic interactive facial model.
- mirajshah 11y agoI wonder if the rendering complexity is low enough to work in games? (~real time) Even with the prior knowledge of well-known celebrity expressions, these faces do seem much more expressive than those of current video game characters who have been modeled with motion capture.
- chadzawistowski 11y agoDoes it need to work in real time? The only person present while a game is running is usually the gamer. L.A. Noire did a great job with ahead-of-time facial modeling based on real people.
- anonymfus 11y agoAvatars in multiplayer games.
- yzh 11y agoVery amazing results and fantastic work! I noticed that all the examples are all front faces with at most 20 to 30 degree angel, but no profiles. I haven't read the paper, but does this mean it only reconstructs a 2D texture but not the 3D structure?
- dalbin 11y agoReal question : Is it really "amazing", "hyper-realistic" ? I have some kind of prosopagnosia, I recognise people in video but I absolutely don't recognise modelled faces, its just looks like lo-poly models on Nintendo 64 for me.
- grubles 11y agoPerhaps because the people in the video are only floating faces without a skull, neck, or hair?
- ericflo 11y agoFunny that they seem to have switched the paper's title and figures from Kevin Spacey to Tom Hanks http://arxiv.org/abs/1506.00752 http://arxiv.org/abs/1506.00752
- deGravity 11y agoI talked to the author about this - apparently Kevin Spacey's lawyers asked them to change it.
- ericflo 11y agoI can imagine why - seems this will be an interesting legal/ethical question for our society in the next decade.
- Pyxl101 11y agoI'm not a lawyer but I would guess that there is no reason they need to change it. As long as they're not profiting from the likenesses of the celebrities they're emulating, then the paper is likely to be considered free speech that does not violate their privacy or publicity rights. See the legality of fair-use of someone's likeness. Some states might have different laws.
- joshkpeterson 11y agoWait till this is on dating sites so you can see people's mannerisms before you meet them.
- dennisgorelik 11y agoTop comment on YouTube: "This is going to revolutionize the Tom Hanks porn industry."
- xixixao 11y agoI think part of the effect is that they create a texture from photos - today's top games don't do this, they scan and model the face, so it looks higher-fidelity but less realistic (while this is very low fidelity). Mafia 1 used a similar technique to achieve really nice results, given how low-poly the models were back then.