14 ms·
DeepFaceLab: A tool that utilizes ML to replace faces in videos
- s_Hogg 7y agoWhat are the non-malicious uses for something like this? Haven't actually got around to pondering that yet.
- boilerupnc 7y agoFirst Person Artist Museum experiences [1] [1] https://www.dezeen.com/2019/05/24/salvador-dali-deepfake-dali-musuem-florida/ https://www.dezeen.com/2019/05/24/salvador-dali-deepfake-dal... [EDITED]
- m12k 7y agoPersonalized meme gif generation? Send your friend a gif with their face on Bruce Lee to tell them they are kicking ass
- mc32 7y ago“Joke” apps, so you can put yourself in a video as a hero heroine, villain villainess. Put yourself as a singer or whatever on a music video, etc. Punking your friends too. In a productive context in filmmaking you can have fun with doubles and stuntmen and stunt women. Or if you have the rights to license the face and likeness of an actor actress and they can be in five films, ten films in a year (forgetting about brand dilution). John Wayne can be in a new Western movie,
- ebg13 7y ago> Punking your friends too. "Punking" does not qualify as non-malicious.
- mc32 7y agoThat depends on your relationship and liberties you take.
- scarygliders 7y agoIt does if you and your friends share the same sense of humour.
- Oras 7y agoalready there and its called Zao https://twitter.com/AllanXia/status/1168049059413643265 https://twitter.com/AllanXia/status/1168049059413643265
- godelski 7y agoWait, this is a one shot? The poster is cherry picking results, right?
- sct202 7y agoHe posted a ton of video examples if you scroll down thread on Twitter, and some are better than others. It's shocking how accessible the technology is now that a phone app is letting you do faceswaps on short video clips with one picture and basically no effort on the part of the user.
- Mic92 7y agoSeems useful for memes?
- amq 7y agoIt could be useful for film production where you could replace the stuntman's face with the real actor's one.
- guerrilla 7y agoAnd replace the low paid actor's face with a high paid actor's face...
- novaRom 7y agoSavings! You also can produce movies then in low-payment countries.
- AnIdiotOnTheNet 7y agoThis has been routine in films since at least Jurassic Park (1993).
- bsenftner 7y agoDoing exactly this, replacing stunt doubles, is what I wrote and acquired a global patent back in '08. I tried to create a personalized advertising agency: put people into the advertisements themselves, driving the new car, wearing the new fashions, taking the luxury vacation. Problem was, I started my effort in '04, had a working pipeline by '08 - but none believed it was possible. Even when I'd demonstrate right in front of them, investors thought it was a trick. I eventually went bankrupt and quit. Today, we have deepfakes, where we could have had personalized adverting a decade ago.
- larryjustin 7y agoThat is neat and all but do we want that?
- bsenftner 7y agoFor the fashion industry, it becomes a fantasy visualization platform - see yourself in all manners of clothing, hair and makeup.
- nelsonic 7y agoAgree that this feels like a tool for malicious intent. But, maybe reviving dead celebrities or family members? Either way this tech seems inevitable, nerds showing off their skills to other nerds. Next question: does it make CCTV footage obsolete as a form of permissible evidence in court?
- phs318u 7y agoIf you had hardware that generated per frame signatures using PK encryption? Private keys and hardware can be compromised but at least it raises the bar and weeds out casually malicious actors. Having said that, I have no idea if such devices exist nor how they might work.
- michaelt 7y agoPoint your signature-per-frame camera at a TV screen showing a faked video. Instant signed fake video.
- antisemiotic 7y agoPorn.
- dominotw 7y agohaha obvious answer.
- TulliusCicero 7y agoPutting your own face (or a friend's face) into a movie or TV show. Or going the other way, use a real video of your friend and replace their face with a famous actor.
- DMcVeigh 7y agoAn interesting use could be to anonymise people in public videos. You could use randomly generated faces (which other emerging tech can produce) and effectively remove peoples recognizable features by "replacing" them.
- cj 7y agoThat’s interesting. I wonder if that would be sufficient to forgo the typical release forms required when filming in certain public places.
- chris5745 7y agoReminds me of the anonymizing suits in “A Scanner Darkly”.
- drusepth 7y agoActually, you could probably get exactly that effect if you modify the script to pull from a pool of source faces per frame instead of a single face.
- BurningFrog 7y agoAn alternative to blurring could be useful. But who should they look like? Maybe that spot could be sold...
- filoleg 7y agoDoesn't have to be an existing person at all: https://thispersondoesnotexist.com/ https://thispersondoesnotexist.com/ I like your idea about selling spots though, definitely something I have never thought of before.
- rohan_shah 7y agoPoliticians and celebrities are all about their faces. They wouldn't allow for anonymizing their footage even though the 'bad' guys will want to exactly use their faces on fake videos for exactly the same reasons.
- derivagral 7y agoOn a network like FB that has your photos, mix your own face into the advertisement! The video can have the pre-computed target, and the network can generate yours from all that free photo storage/tagging...
- zawerf 7y agoMy initial reaction is that this counts as malicious. But thinking about it some more, I at least don't mind clothing ads using me as the model. (Assuming there's opt in and all) It's better than mentally swapping out the skin tone and body type to see if it was really the clothes that looked good or the attractive model.
- tyingq 7y agoStanding in for me for many tedious videoconference meetings :)
- krasin 7y agoPrivacy. A commercial video shot in public today is usually required to blur faces of people who didn't give explicit consent. Replacing faces with stubs is a nice alternative, as long as the target face is from a person with a consent. I am not a lawyer, though.
- gchadwick 7y ago> Replacing faces with stubs is a nice alternative, as long as the target face is from a person with a consent. Why use a real face? You could generate a fake one using a GAN. You may even be able to build a fully automatic anonymize tool that can identify the broad characteristics of each face present (skin tone, age, male/female etc) then use that to generate replacement faces.
- jobigoud 7y agoMaybe it could be coupled with GAN generated faces.
- golergka 7y agoHabituating people to the fact that this exists, therefore lessening the impact of malicious uses.
- mkr-hn 7y agoI could see someone using it to update home movies after transitioning. Dysphoria is probably not great for reminiscing.
- divbyzer0 7y agoLocalisation of films\tv shows\educational material. Often English language films are poorly dubbed for local (mostly non-English speaking) markets. Having a local/recognisable actor filling the role would be a big improvment.
- mc32 7y agoThat’s a little more complicated. Now you’re replacing a face and body and either having them voice dialogue separately (dubbing themselves) or you’re putting a local face to a foreign film (and subbing?) I can see them modifying the actual original actor to speak in a local language however to sync with the localized dialogue. That said, your proposition is also interesting. It raises some questions though. Imagine an indie French film where the protagonist (or whoever) gets replaced by a Hollywood actor. Or an indie Angolan film which gets the treatment. There would be lots of purists (for many reasons) protesting this kind of option.
- sakopov 7y agoI recently watched a Deep Fake video of The Shining with Jim Carrey's face over Jack Nicholson's. It was so damn good you couldn't even tell it wasn't actually him playing the role. This was all fun and games, but one thing that immediately comes to mind is movies staring people who are no longer alive.
- tashoecraft 7y agoI’ve been telling people that there will now always be new movies with Tom Hanks. Even after he has passed, there’s so much screen time to use and audio recordings that he could forever be in new movies.
- BurningFrog 7y agoImagine if when you click up a movie/show on Netflix, you get to pick which actors play the main parts.
- Angostura 7y agoGrand Moff Tarkin says hello.
- scarmig 7y agoFamous actors could license their face for use. This could even last past their death, allowing them to star in movies decades after their flesh death. If an actor dies while filming a movie, it can be required in the contract that they allow their face to be generated for scenes that haven't been filmed yet. They could also use this to use a younger face as they age.
- fawak 7y agoI'm pretty sure I read somewhere that Stan Lee's face had been scanned like crazy before his death in order to keep the cameos going.
- JohnJamesRambo 7y agoI hope they don’t do that. I’d just find it more sad and creepy than enjoyable.
- bitL 7y agoAs a part of a pro movie making suite like AfterEffects or DaVinci Resolve - you can take somebody else to act in a scene and then replace with physique of your chosen actor, e.g. you do stunts, your actor can't appear at a given geographic location at a given time, somebody had a facial injury/scars you want to get rid of etc. Plenty of valid uses. Then you could e.g. do a render in Blender and replace "avatars" with real human faces, avoiding "uncanny valley" in your animation. Travel industry can have uses as well - these days many places offer video recordings of their clients doing some dangerous activity; for those that can't do them yet still want the video they could offer pre-baked scenes where taking a selfie could kill somebody and then replace their face/body with some green screen footage of their client using this app and some pose estimator/body switcher. Or just show you how much would you enjoy some beach property in some hot location with a video featuring you etc. This would make it useful for custom ads featuring you as well.
- martin-adams 7y agoStorytelling of historical characters where we have common knowledge of what they looked like (e.g. Einstein). Reshooting scenes of movies without having to get key actors back on location. Immersive storytelling where you get to be placed inside a movie. Actor safety where they can go through transformations for a shoot, such as extreme weight loss or weight gain. I can really only think of it in terms of entertainment. There may be some therapy benefits which are yet to get discovered, but I would imagine that's a whole new level of complexity.
- JoeAltmaier 7y agoPresenting yourself online (think Skype) as more symmetrical, more imposing, more pleasing. For business advantage while negotiating.
- sanlyx 7y agoEvery time I hit HN I can't help but be amazed at how sad the future is getting
- JoeAltmaier 7y agoHa! People who wear power suits, makeup, expensive haircuts and shoes are already doing all this. Technology just means you don't have to spend time in the stylist's chair to accomplish it any more!
- EpicEng 7y agoUntil you have to, you know, actually meet someone in person.
- JoeAltmaier 7y agoI've done entire contracts without ever meeting in person.
- heeen 7y agohttps://youtu.be/2ZKPnuUFwOk?list=PL84DB158CAABE4E07&t=633 https://youtu.be/2ZKPnuUFwOk?list=PL84DB158CAABE4E07&t=633 https://youtu.be/KH1V6CHO1Jk?list=PL84DB158CAABE4E07&t=388 https://youtu.be/KH1V6CHO1Jk?list=PL84DB158CAABE4E07&t=388
- MayeulC 7y agoI see a lot of concern regarding fake pornographic videos. And I get it. But on the other hand, people could release videos they do not want to appear in with a credible alternative to blurring faces (though I guess you could still fingerprint facial expressions). If this gets mainstream enough, "revenge porn" would also lose a lot of its current value, being easily dismissible as "fake". (And so would genuine photographical evidence...) Journalists could use it to hide their sources from the public. If this gets reaaly good, it would become a nice alternative to motion capture for cinematography (pose estimation, including facial expressions). Then swap in the 3D model, or concept art of the person you want to appear on screen. Significantly reduce the costs for makeup, and enable more in post-prod. That was probably already available for big teams with big budgets, but this makes it more mainstream.
- dominotw 7y agoMovies/Media with you, your friends and family as the cast? Personalized entertainment.
- craigsmansion 7y agoReplacing all those hacks and pretenders in every movie ever [1] with the glory that is Nicolas Cage. 1. including "Face off"
- wait-a-minute 7y agoDeep analysis of the body for any visible signs of tumors or diseases. Stroke detection software. Home and car entry without needing to carry any keys. Much faster entry into airports, where we need passports or security checks anyway. Lots of entertainment possibilities.
- jcims 7y agoI got a good laugh out of this one - https://www.youtube.com/watch?v=1kJO_L9gr5Y https://www.youtube.com/watch?v=1kJO_L9gr5Y probably only funny if you watch these folks regularly
- bsenftner 7y agoPersonalized adverting: see yourself / your family using/enjoying products you're yet to purchase. Cars, clothing, trips...
- JoblessWonder 7y agoA different git repository has a "manifesto" of sorts explaining why they think it is ethical to release: https://github.com/deepfakes/faceswap#faceswap-has-ethical-uses https://github.com/deepfakes/faceswap#faceswap-has-ethical-u...
- rohan_shah 7y agoI thought of a non-malicious use. Celebrities could sell the 'rights' to their faces and then ordinary people could 'act' in commercials and later on even movies and then just stick the face of the celebrities?! xD Celebrities will become cheaply available to 'star' in your next Netflix series. (But unfortunately, even if celebrities don't sell their faces, they'll be stolen!)
- TrackerFF 7y agoSay you need to replace an actor in a movie. Great potential there
- costcopizza 7y agoI think the cons are going to heavily outweigh the pros with this tech. Fake news on high octane race gas.
- goolulusaurs 7y agoIt would go a long way towards making many kinds of blackmail difficult or impossible, which seems like a pretty big upside.
- momokoko 7y agoPhotoshop has existed with the ability to modify images kn this way for over a decade. Can you think of many recent examples of video evidence was the sole evidence in a news report?
- MayeulC 7y agoWell, I also like the facts that: * Everyone can access it. Not just sophisticated actors anymore * It might make us rethink the entire "truth" chain, on how we source our information Of course, the two go hand-in-hand, but the latter point is overdue: while video is perhaps more glaring, it's something that's needed in a lot of other areas as well (text -- news articles, messages, mail; sound -- phone calls, etc; image -- photoshop, though we start to get used to it).
- cowpig 7y ago> Everyone can access it. Not just sophisticated actors anymore Why do you like this?
- geerlingguy 7y ago“When everyone’s special, no one is.” Basically, a (good) deepfake was out of reach for all but a few until recently, so if some bad actor with money and resources wanted to fake a video they could do it and few people would even think it could be faked. Nowadays, the bar is way higher for this, as every video can be suspect.
- Oras 7y agoIs it the same technology used by Zao? The example in the gallery does not look as good as the video here https://twitter.com/AllanXia/status/1168049059413643265 https://twitter.com/AllanXia/status/1168049059413643265
- throwawaywego 7y agoI think this tech is too complex to run on mobile devices in 8 seconds for video transfer from 1 selfie. What I think Zao does is preprocess the videos (manually or highly accurate facepoint detection). They pre-calculate the transforms (standard facemorphing algorithms with opacity/alpha tweaks), and shading depending on the scene lighting. Then they just need a good frontal selfie, or do some frontalization, and keypoint detection, and the rest can be rendered/computed without much resources, following a pre-defined script. If more advanced than facemorphing, then perhaps something more like: https://github.com/facebookresearch/supervision-by-registration/raw/master/cache_data/cache/demo.gif https://github.com/facebookresearch/supervision-by-registrat... (pre-fitting a 3D face mask, then texturing it with your selfie)
- JoblessWonder 7y agoYeah, I think the big question is if Zao only allows pre-selected "scenes" that they have already done the processing on or if they allow you to upload any video. From the results, I think you are exactly right in how they are accomplishing those videos.
- puranjay 7y agoVideo evidence is going to be inadmissible very soon. It's ironic that we're essentially being pushed back to a pre-technology, pre-media time. "If you didn't see it with your own two eyes, you can't believe it"
- mikojan 7y agoPeople have been convincingly altering pictures since forever. I could do it no problem when I was 14. Nothing changes.
- ISL 7y agoWere you fourteen in the era of digital image processing, or in the darkroom? It was always possible, but it used to be far more difficult.
- therealx 7y agoI forged many documents in the early digital era because paper still had the trust of the darkroom era. Theres always a transition time like that thats very exploitable.
- CabSauce 7y agoIt'll be the same as anything else. An approach to fake something will be created. Then an approach to identify the fakes... and repeat.
- roywiggins 7y agoVideo evidence has always required testimony to its provenance, hasn't it? Otherwise it can be removed as hearsay. Courts have always been worried about authenticating evidence including video. https://jamespublishing.com/2013/objecting-video-audio-evidence-without-hesitation/ https://jamespublishing.com/2013/objecting-video-audio-evide... https://www.esquiresolutions.com/federal-rules-catch-digital-evidence-avalanche/ https://www.esquiresolutions.com/federal-rules-catch-digital...
- vectorEQ 7y ago
- localhost 7y agoI wonder what this implies for the future of media (audio, photos, video) in general? Do we only consume media that has been signed by the creator and verified by an authority that we trust, e.g., a "blue checkmark" for media? Do we know how effective the SSL certificate verification has been in browsers at influencing consumer behavior?
- ur-whale 7y agoWhy does that matter? Or have you entirely given on critical thinking?
- cityiguana 7y agoCritical thinking only helps you determine what could happen, not what actually did happen.
- ASDF2001 7y agoI feel like this question is just a cheap snide remark. The current political moment provides a lot of evidence for why a technology like this introduces these questions. A technology where we can fake any type of photo or video is dangerous in a world where tweets dominate the news cycle.
- dang 7y agoPersonal attacks will get you banned here. Please review the site guidelines and don't post like this to HN. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- plopz 7y ago> Do we know how effective the SSL certificate verification has been in browsers at influencing consumer behavior? Considering how EV is essentially dead, I would assume it hasn't been very effective.
- brightball 7y agoWe do know that people don't care about trust indicators. They care about warnings for danger, but they don't care about trust indicators. Trust indicators are essentially, something for companies to sell. Because everybody isn't going to pay for the trust indicator and we're going to consume plenty of things that don't have it, the indicator is basically just noise. It's why EV failed and we're about to see it repeat with this BIMI technology in the email world IMO. On the other hand, if you get a big giant danger warning that stands out because everything else doesn't have it...it gets attention.
- luiscosio 7y agoThere is also: https://github.com/deepfakes/faceswap https://github.com/deepfakes/faceswap With a big development community and interesting results.
- InfinityByTen 7y agoDeep Fake with its revenge porn wasn't enough to give this a second thought?
- kevingadd 7y agoResearchers and prototyping engineers generally don't seem concerned with ethics or consequences these days. Hard to be sure whether it's due to a pressure to publish or interest in chasing fame/revenue but there's lots of stuff like this being published lately. Even if some researchers opted not to chase this down there are always more people looking to push the boundaries. The natural conclusion goes past revenge porn to revenge videos of beheadings, murders, etc that can be sent to friends or family members who aren't technically savvy enough to recognize fakes for what they are. Not to mention the existing phenomenon of blaming random civilians for murders and terror attacks - this will be far more convincing when it's paired with convincing fakes. It'll probably claim some lives.
- nefitty 7y agoIs innovation supposed to pause for society to catch up every time something drastic is created or discovered?
- InfinityByTen 7y agoNuclear tech required a rethink. So maybe yes?
- nefitty 7y agoFair!
- jonplackett 7y agoOn a scale of 1 to 10, how much easier does this make Deepfakery?
- jonnismash 7y agoThis technology has been posted on HN before and the comments were all too similar to this one. I understand why a lot of people are falling into the FUD-hole, but we can pretty easily solve the issues mentioned in the comments by inserting a digital key - similar to a SSL certificate, into an image/video upon initial upload. Or even upon creation, maybe using some sort of ledger to verify integrity. I'm not too worried.
- michaelmior 7y agoWhat does that verify? Assuming they key hasn't be compromised, it does verify that the holder of the key uploaded a file at a particular time. It still doesn't show that the video itself is genuine. Although it does seem to allow us to fall back on the trustworthiness of the source which is better than nothing.
- cmroanirgo 7y agoI used to work for a company who provided verifiable audio and video (virtually in the pre digital days) mainly for law enforcement. I remember discussing how it'd be done in digital: a constant stream of verification (eg a hash of the last second) all encoded by a key. It's pretty much how the analogue version worked. I'm sure (if they're still around) that'd be what they're doing.
- Animats 7y agoHollywood has been doing this for a while, for stunt performers. Sometimes well, sometimes badly. We'll be seeing more of that as it works better. As for the political implications, go watch "Wag the Dog" again.
- spaniard_dev 7y agoAm I the only one that cries when seeing non PEP8 compliant Python code? Why do data scientists mistreat Python that way?
- topranks 7y agoYou are the only one.
- gallerdude 7y agoI'm surprised no one's bringing up the precedent of photoshop. For years we've culturally realized that pictures may not tell the whole story, and this is the exact same, down to results that can't quite escape the uncanny valley.
- lorepieri 7y agoOn a (somewhat) related note, I'm working on face recognition with homomorphic encryption, therefore without compromising the user privacy. The bold goal is the first privacy preserving videocamera. If you find this interesting, I would love to chat about it.
- noteness 7y agoI'd love to hear more about it & see the code.