17 ms·

Family fun with deepfakes

427 points by robinhouston 9y ago

fredley 9y ago
By the time the next US election comes around it's very likely this technology, combined with Lyrebird etc. will allow anyone to make [politician] convincingly say or do anything they like. I can't make my mind up if this will make the current fake news / filter bubble situation worse, or if it will make it so bad that people learn to distrust any kind of digital media entirely. I'm still not sure, given that the use of these tools for political gain seems inevitable at this point how we will actually verify a video is real in the near future.
- aluhut 9y ago
  I'm surprised nobody build this Trump watersports video yet. The tool is probably still to complicated to use. There is a business opportunity.
  - evansj 9y ago
    Maybe we're seeing so much coverage of deepfakes just so there is plausible deniability if the watersports video leaks?
    - aluhut 9y ago
      Nice twist.
  - wand3r 9y ago
    What is "Trump watersports"? I haven't heard of it and that is a searchterm I am not super interested in getting google NSFW results from. Is it what I think it is, I haven't heard about this at all?
    - aluhut 9y ago
      There is a conspiracy theory about trump urinating on a prostitute on his visit to Russia some years ago. The theory states that Putin has a tape of it.
    - ddebernardy 9y ago
      Probably a reference to the Trump pee tape: http://www.newsweek.com/fox-news-host-jesse-watters-bizarre-reason-why-trump-pee-tape-isnt-real-if-798083 http://www.newsweek.com/fox-news-host-jesse-watters-bizarre-...
      - wand3r 9y ago
        Yep, that's what I expected. I read the article and while it hasn't been proven, if this exists and was released it would be pretty appalling.
        ddebernardy 9y ago
        Methinks it's only a matter of time when someone creates a phony video using a software like deepfakes. What a bizarre age we live in...
    - jjgreen 9y ago
      The rumour that Russian intelligence have video of Trump getting pissed on by prostitutes, the #goldenshower hashstag is used I believe. No idea of its veracity.
      - lightbyte 9y ago
        The claim is not that he was getting pissed on, it is that he paid prostitutes to pee on a bed that the Obama's previously used.
- giarc 9y ago
  Next election the "grab her by the p*" soundbite would have been "fake sound". It's really two problems. One, making people say things they didn't say. And two, allowing people to deny things they really did say.
  - sharemywin 9y ago
    The second probably being much more damaging.
  - ctrl-j 9y ago
    > Next election the "grab her by the p*" soundbite would have been "fake sound". You mean the last election right? A lot of people believed it was faked.
    - tehwebguy 9y ago
      Well those people cannot be helped, he admitted it plainly.
      - akhilcacharya 9y ago
        No disagreement on that, but even he allegedly started saying it is faked recently.
        j_koreth 9y ago
        source ?
        giarc 9y ago
        http://www.cnn.com/2017/11/27/politics/donald-trump-access-hollywood-conspiracy/index.html http://www.cnn.com/2017/11/27/politics/donald-trump-access-h...
        ctrl-j 9y ago
        https://www.nytimes.com/2017/11/28/us/politics/trump-access-hollywood-tape.html https://www.nytimes.com/2017/11/28/us/politics/trump-access-...
- gk1 9y ago
  > or if it will make it so bad that people learn to distrust any kind of digital media entirely. There are already fake news sites out there that are "so bad" you'd think nobody would believe them, and yet... Still, it's important we inform people this is possible, so that at least the more reasonable among them can view future videos through a skeptical lens. I'm sure by now many HN readers are not surprised by this technology, but many outside of HN would be blown away.
  - losteric 9y ago
    Herd immunity? Perhaps naive people believe those spam sites because smarter people believe the better spam sites that share the same narrative... However, distrusting digital media isn't the solution either - there are plenty of trash magazines publishing conspiracy theories and other absurdities. Plus, digital media has revealed huge segments of America are becoming culturally isolated and undemocratic. A return to print media might shift funds back to deep professional investigative journalism that covers those issues... or just give owners better returns.
- blauditore 9y ago
  I don't think this will be a big issue. The same could already be done with still images years ago, if not decades. And with enough time and resources, the same can be applied reasonably well to videos, manually frame by frame or using CGI. It takes a lot more to make it undetectable as fake though, and I guess the same applies to deepfakes. Once people get used to the fact such face swaps can be done, they'll be more sceptical and demand realness verification in case of published sensitive material.
  - sbarre 9y ago
    Most people barely question the validity or truth of news they see on TV, and that's real footage.. You really think those same people will actually question if a video was real in the first place, particularly if it reinforces their world-view?
  - simias 9y ago
    I'm not so much concerned about people making politicians and other personalities say something they haven't, rather I worry that these personalities will now have plausible deniability for any footage of them doing or saying something embarrassing. It's always harder to prove a negative and in this case you'd have to prove that it can't be a fake. Now imagine figuring out which video is real amid a flurry of fakes.
- sbarre 9y ago
  Putting aside the US or first-world elections/media.. It's worrying to think of how this technology could be used in parts of the world where there is much less widespread understanding of this level of digital manipulation.
- matt_wulfeck 9y ago
  Your fears are greatly overblown. Haven’t we been able to “Photoshop“ people convincingly for a very long time? People are less sceptible to this than you think. Even when scandals do break, it’s not a picture or video that’s proof, it’s a picture or video that produces witnesses.
  - tome 9y ago
    Video has been pretty much immune to "Photoshopping" AFAIK. The new technology also makes it extremely cheap.
    - platz 9y ago
      nein. or there was that time a german variety show 'false flagged' the germans by doctoring a video of yanis varoufakis to have him 'stick the middle finger to germany' during a talk he gave (https://www.youtube.com/watch?v=Vx-1LQu6mAE https://www.youtube.com/watch?v=Vx-1LQu6mAE). There were witnesses; but they didn't stop the confusion and outrage that ensued due to the effects of viral media. Would this pass a sophisticated technical analysis of the video? No - is there time and public understanding for the results of said sophisticated technical analysis to prevent the damage before it is done? no. The notion that witnesses or forensic analysis can combat the speed of information, especially across borders, is a bit quaint.
      - cf498 9y ago
        Its something entirely different to fake a face instead of a finger in a video. Then again, the deep fakes I have seen so far arent really that convincing either. But one can only wonder how much you can improve such a thing. With politicians of whom a wide array of video material exists and were you have the option to create an incident, instead of altering existing ones, the possibilities are really interesting.
        platz 9y ago
        did we really post that link at the same time, or did you edit your other response to match?
- psergeant 9y ago
  I wrote some thoughts on this: https://github.com/pjlsergeant/multimedia-trust-and-certification https://github.com/pjlsergeant/multimedia-trust-and-certific...
- deegles 9y ago
  You can't prevent someone from making a fake, but one could at least use cryptographic signatures to prove they made a statement on video/audio.
- tboyd47 9y ago
  We need to start thinking about what criminal justice will look like when digital recordings are all considered hearsay. That's bound to happen. And perhaps it should... if average people have the technology now, how long has it been in the hands of police departments, newspapers, militaries, and intelligence agencies? We tend to focus on fakes with high-profile people like politicians and celebrities, but it can clearly be done with everyday people as well. The author was able to produce a fake video segment of a well-lit, animated conversation involving his wife. She's directly in front of the camera, speaking and making hand motions for several seconds. How much easier would it be to insert a face into grainy security camera footage, or a shaky clip of a large crowd at a political rally?
  - 794CD01 9y ago
    The presence of one source can be easily dismissed, but if you have 500 independent sources that produce video of the same event from their own angles and timespans in an interconsistent way, it's far more likely that the recorded event happened as opposed to all of these different sources lying.
    - tboyd47 9y ago
      Really? How much harder is it to run the OP's script with 500 input videos as opposed to only one? Especially if they're very similar? How many people have the expertise needed to identify a faked video, and how much training would they need?
      - jakear 9y ago
        The great part is that using a GAN means we cannot automatically distinguish a fake from a real... because that's the whole point of the GAN.
      - 794CD01 9y ago
        At that point there is no such thing as identifying faked videos. Instead you'd rely on the abundance of independent videos and see how consistent they are. Generating a lot of them doesn't do you any good unless you can insert them into the black boxes of the 3 planes and 40 cars in the area, the databases of all the drone operators, the internal storage of all the nearby smartphones, etc.
- 794CD01 9y ago
  People should never have been trusting media in the first place, digital or otherwise. And for the most part, they don't have to. Humans believe what they want to believe or are told to believe. Evidence has little to do with it.
- douglaswlance 9y ago
  Information will need to be verified by consensus on a decentralized blockchain.
- tw1010 9y ago
  Just like there is a social system in place to sift fact from rumors (via watercooler chatter, snopes, networks of trust), at least to a degree where society doesn't collapse into chaos, in the same way will we be able dampen any damage that this new technology might cause. I think this whole "fake videos of politicians" will become a way smaller issue than what doomsayers predict it will be.
- konschubert 9y ago
  It's basically the same situation as we had with print before television. It's all about which sources you trust. In fact, it has always been that way. Deepfakes just makes it more obvious. Which is maybe a good thing.
- zodPod 9y ago
  Something that comes up a lot on the deepfakes sub is the notion that this should make people more skeptical. They use the phrase "If anything can be real, nothing is real" and I think it makes a lot of sense.
  - macawfish 9y ago
    That's not necessarily a good thing. Sometimes it's necessary for people to believe something factual. Blanket skepticism can be very destructive in these cases.
- JohnStrange 9y ago
  It will make things way worse, since politicians from now on can and will plausibly deny any video evidence about them. Candidates will literally rewrite history instead of merely suffering from amnesia and any real evidence will end up in battles between one TV show 'expert' against another. On a bright side, young aspiring actors can make porn movies to get some cash early in their career without necessarily having to fear it will ruin their career many years later - those movies will exist anyway.
- rexpop 9y ago
  Hopefully, this makes celebrity untenable.
- Asdfbla 9y ago
  I find the more concerning thing is not that people will believe that a fake is real, it's more that they won't believe something real is real anymore. Already nowadays the whole thing about fake news and alternative facts has segments of the population flat out not believing the truth. This may very well be greatly exacerbated in the future where, regardless of the amount of evidence you bring, people will just disregard it as fake.
  - zodPod 9y ago
    If it can be faked, is it really evidence? I mean, you're worrying about people not believing assertions at that point. This is already something that can be faked just not by the masses yet and not easily. You should always look for multiple sources of anything and not just believe one source that shows you any evidence of something being true.
    - zokier 9y ago
      There was this essay posted on HN some time back, where the main thesis was that teaching people to be critical about media strengthened the positions of antivaxers, flatearthers and so on, because they did not rely on traditional media so they were then sort of considered more trustworthy. And now deepfakes etc throws just more fuel in the fire that burns trust, people end up more and more believing things that happen to make them feel good.
  - olivermarks 9y ago
    Believing 'the truth' is a huge can of worms here. Verifying the legitimacy and accuracy of 'news' claims and representations has always been very difficult for the lay person. We are now well into an era when 'the truth' is even harder to find. 'Believe nothing of what you hear and only half of what you see' (Edgar Allen Poe) has now become 'believe nothing of what you see' too, at least until you have verified to your level of satisfaction...
- clarkmoody 9y ago
  It's time for cryptographic signatures in cameras (phones and otherwise) so that you can prove something is unadulterated. Apple signing key -> your phone's key -> signature over hash of the media (edit: for video, embed signatures every so often to you can verify a shorter clip out of the whole). Embed Apple's signature and your pubkey in the metadata of the image, and TwitYouFace will verify on upload that it's authentic. Of course, you can't fix stupid unless you talk about a complete overhaul of public education... And even then, people are people.
  - ggg9990 9y ago
    That doesn’t prove anything. Make your fake video, point your phone camera at it, record.
    - bufferoverflow 9y ago
      Try it. Seriously, try and record a realistically looking recording off the screen. Movie pirates have been trying to perfect it for decades, and even their best attempts look obvious.
      - 05 9y ago
        Unless you build DRM right into camera sensor (which is, admittedly, not that hard), you can use an FPGA chip to receive your source video in HDMI format, convert it to CSI interface and send it to the phone SoC over the camera flex cable.
      - ggg9990 9y ago
        If you can make a porn video of someone look like a porn video of someone else, I’m sure the technology to solve this problem is not far off.
    - clarkmoody 9y ago
      Maybe a second layer of AI that cleans up that video-of-video look? Philosophically though, I think it's a good thing that many are learning not to trust information from self-proclaimed authorities :-)
- root_axis 9y ago
  It's likely that we'll soon enter a new era of technological cat-and-mouse as tools are developed to automatically detect artifacts caused by digital processing and counter-techniques are developed to minimize the appearance of those artifacts.
- filoeleven 9y ago
  How will we actually verify a video is real in the near future? I’ve been thinking about this a lot lately, for many of the same reasons. I think one answer can be found in device-level signing of images and A/V feeds. A recorder can integrate its cryptographic signing and keystore on the same chip as its CCD/audio controllers, like Apple’s T2 chip does, so that it has the ability to sign direct feeds from the hardware as being such. This might be more complicated for A/V than still images, since a large enough recording would have to be written to disk. Perhaps some rolling scheme could be devised. For an added measure of verification, the SoC could upload them (optionally encrypted) to a distributed public store like a blockchain or IPFS, proving that they were created at/around a certain time. This seems like it would be useful in concert with other tools for news agencies (who should be leading the way in providing as much documented research and evidence for things as they can, as a matter of course, while protecting their sources), law enforcement (who is losing the public’s trust when it comes to the chain of custody and testimony), etc. We jumped very far ahead both in how much importance we place on digital documents and in how easily we can forge them, so we need to catch up fast on finding robust mechanisms for verification.
- UncleMeat 9y ago
  Kerry was getting photoshopped into propaganda photos in 2004. The new thing isn't fake documents. The new thing is a highly scalable social media infrastructure to share false documents widely.
- amai 9y ago
  There will be no next election in the US.
jacquesm 9y ago
Next up, put yourself and/or family members into any movie. That's really only a matter of time.
- petercooper 9y ago
  When I was a kid in the mid 80s, I got a racing game (Revs) that let you enter names in for the CPU racers. Looking back it seems ridiculous but I remember it being such a thrill to pretend I was racing against "people" I knew. Even that was exciting to me, so what's happening now is absolutely mindboggling(!)
  - 0xdeadbeefbabe 9y ago
    The names example calls for more imagination though. Yeah it's mind boggling, but I wonder if the effect will be about the same or that much more.
    - petercooper 9y ago
      So maybe the "uncanny valley" could even ruin the effect? Some of the samples so far are really good though, so maybe we skip that stage..
- icefox 9y ago
  Before movies, why not insert friends into your video game? Or how about creating a company that sells profiles each of which would be a collection of high quality photos/videos. In turn if you have a video game or movie you can get a random actor with the body you want and swap the face with the face you want? Or even better create a service where individuals can upload their own photos which they can get some of the profit if their profile is used.
  - _up 9y ago
    Another Idea make Bad Actors more convincing with "stock reactions". Track the face of your actor copy Anthony Hopkins reaction face over it and than use deepfake to get the original face back.
digitalsushi 9y ago
Whatever you think about this, it's here to stay - the code is open source and cached on a million hard drives already. If we treat the normal rate of progression, we'll see a year 2023 version for 5 bucks off the App Store, your phone will live-convert the video you're recording at your drinking party with your friends into an all celebrity event. Probably with voice swap. Your drunken roommate's scratchy voice will be ML'd into Frank Sinatra, autotuned, face swapped, and uploaded for a few laughs to facebook. It's here to stay whether you like it or not.
- trs80 9y ago
  Then I will exit my home with a scrambler mask.
  - mental1896 9y ago
    That'll do until anti-scrambler tech comes out
    - baseten 9y ago
      it will be no match for my anti-anti-scrambler tech.
  - toomuchtodo 9y ago
    Until gov or Equifax leaks your ID photo.
- petercooper 9y ago
  Agreed, although I think what you're describing is how it'll be sold, yet most people will surreptitiously use it to indulge in fantasies with "normal" people which will be even more damaging. Got a picture of that co-worker you like? Congrats, now you have him/her in some porn. Interesting times for sure and I think we'll see a lot of flak and fallout in people's personal lives from their creations, far more than we see from "typical" porn use now.
  - 9y ago
ghostcluster 9y ago
Is it just me or are these "deep fakes" just not even very convincing? I felt the same way about all the Nicholas Cage ones that were viral recently. It's so obviously a very shallow 'effect' and can't compensate for very basic differences in bone structure, hair, etc.
- digitalsushi 9y ago
  Regardless of whether this is a good or bad technology to have, I can all but guarantee it will only improve with time.
- lucideer 9y ago
  They'll get more convincing. Then we'll actually get better at recognising subtle differences. Then they'll get even more convincing. Someone will develop algorithms to identify those fakes. Then they'll be further developed and improved to the point that they're impossible to distinguish at every level.
  - yjftsjthsd-h 9y ago
    > Someone will develop algorithms to identify those fakes. One twist: don't adversarial generative networks already work by including a "fake detector" and beating it? This cat-mouse game might last, like, all of a week.
    - lucideer 9y ago
      > This cat-mouse game might last, like, all of a week I'd give it two at least.
- isolli 9y ago
  Like everything in AI, it's probably just a matter of time now.
- ghostbrainalpha 9y ago
  Are you talking about the pornography being unconvincing or the GIF from this article? Fake: http://svencharleer.com/blog/wp-content/uploads/2018/02/anne-1.gif http://svencharleer.com/blog/wp-content/uploads/2018/02/anne... Original: http://svencharleer.com/blog/wp-content/uploads/2018/02/elke-1.gif http://svencharleer.com/blog/wp-content/uploads/2018/02/elke... I could see the fake being easy to detect at high resolution, but personally I have a 0% chance of catching this gif as fake.
deleted 9y ago
[deleted]
deleted 9y ago
[deleted]
taoistextremist 9y ago
In the realm of innocent fun with this stuff, just how much time would it take to insert myself into the entirety of a movie? I'm just imagining how fun it would be to invite friends around to watch something we're all excited about, only to convincingly insert my face into the main character's.
- tomaskafka 9y ago
  Or, once advert industry learns about it, how about replacing all the things on the table in Audrey Hepburn movie with BRAND NEW LEMONMELON FLAVOURED OREOS! :) Ideally on the fly, with each box sold to the highest bidder, adsense-style.
  - logfromblammo 9y ago
    I have also thought about this, and whether I could live with myself after making a huge pile of money to relentlessly annoy every other human on Earth. I conceived it as video bloggers setting up a dynamic replacement volume in their scenes, and compositing a 3-D scene for the sponsored product into it as the stream goes out. One viewer might see an open Coke bottle. Another might see a half-eaten package of Oreos. Another might see framed photographs of adoptable rescue pets. All the vlogger does is avoid entering the ad-volume and the software does the rest. Inserting into old movies would require a bit more sophistication, I think. You have to discern the geometry of the scene, model it, then discover the insertable volumes, match the camera movement in software, and finally use a shader to match the object textures to the film grain. But let's not do this. Please.
    - yjftsjthsd-h 9y ago
      > Inserting into old movies would require a bit more sophistication, I think. You have to discern the geometry of the scene, model it, then discover the insertable volumes, match the camera movement in software, and finally use a shader to match the object textures to the film grain. Only if you're changing volume. We can start with swapping flat surfaces (say, cereal boxes).
ploggingdev 9y ago
Discord banned multiple servers that r/deepfakes created and their reasoning was that it's being used for creating involuntary porn and so is against their ToS. I'm curious to hear your thoughts on the legal status of deepfakes as well as the moral and ethical questions behind the usage of the tech. A thought experiment : I run a chat room site and will be adding support for user created chat rooms (something like discord for cryptocurrency communities with real time price widgets etc). Let's say I went ahead and got them to use my chat room, what would the consequences be? And what do you think about the moral and ethical questions behind hosting such a community? Am I now enabling the creation of deepfakes? By choosing to avoid hosting such a controversial community, am I imposing my moral and ethical reasoning on others (the free speech argument)? How do think about the deepfakes project, which is mostly being used for creating celebrity porn, but the tech itself is interesting and has applications(targeted ads or in stunt doubles etc)? To be clear, I'm not trying to host a chat room for them, this is just a thought experiment. I don't like what it's being used for and don't want to support it in any way.
- ggg9990 9y ago
  Legally, in the US, if you own the copyright to the source images, the copyright to the porn, and you’re not selling the output, it’s 100% legal
  - zawerf 9y ago
    There was a thread on /r/machinelearning titled "Is it legal to use copyright material as training data?" https://www.reddit.com/r/MachineLearning/comments/4qrgh8/is_it_legal_to_use_copyright_material_as_training/ https://www.reddit.com/r/MachineLearning/comments/4qrgh8/is_... But the issue here isn't whether training on copyrighted face images is legal. You can totally take copyright-free photos of anybody as long as they appear in public. The more interesting question I think is whether you own the copyright to the 3D capture of your face. For example disney won't let you get away with using models of their characters. Don't I have the right to not let anyone use the "model" that I spent my literal life growing?
    - 9y ago
chrischen 9y ago
Is there an inplementation of a generalized face swap that doesn’t require hundreds of images of both the target and subject?
- bpicolo 9y ago
  Keep in mind that one video counts as hundreds of images
ChuckMcM 9y ago
It makes the use of stunt doubles easier as well since the actor's face can be mapped to the body double. And it isn't like this point in time wasn't predicted by just about everyone in CS. Given enough computer power and the right algorithm pretty much any data is fungible, video or otherwise. But we've just entered the most dangerous period when most people still believe what they see in a video is what was originally videoed. It is when people will use this technology for deception and get away with it like they did with Photoshop before 'shopped!' came into the more general populations consciousness.
- dllthomas 9y ago
  I don't disagree, but I hope that the transition will be eased by the fact that the photoshop transition is within living memory of so many.
- HorizonXP 9y ago
  A cool, possible solution would be to have every participant in the video to digitally sign the final file to verify that they're actually in the video, consensually. That way, if anyone finds a deep fake video, it would be easy to dispute it since it doesn't have the person's signature. Of course, this would be exceedingly difficult in practice, since you'd need the signing to be opt-out instead of opt-in for it to be truly effective.
adonnjohn 9y ago
Is it possible to utilize deepfake tech to crack iPhone Face Id?
- dawnerd 9y ago
  No more than a picture of the person, which you'd need anyways, so I don't see the point?
- hmate9 9y ago
  The two are unrelated. Face ID looks at a 3D face not a 2d video
dubin 9y ago
To add a data point, a friend and I made one of these over the weekend (with Trump and Michael Scott https://youtu.be/0Rexuh-VY6E https://youtu.be/0Rexuh-VY6E) It was shockingly easy to do. We're technical, but neither of us know a lick about machine learning. It took a couple hours to collect training data (we turned speeches/interviews from each of them into thousands of photos), and 20 hours to train the model. In the future you could automate the data collection for a person even more, to the point where they just need to film a selfie of themselves for a couple of minutes in a couple of different lightnings, and boom, after you train their decoder, you could put them on any celebrity. EDIT: This is what we used https://github.com/deepfakes/faceswap https://github.com/deepfakes/faceswap Would be happy to walk anyone through how to do this
- howToLearnSpark 9y ago
  the Trump clip is gold! Please do more!
  - dubin 9y ago
    Heh, thank you! Any suggestions welcome :)
- michaelbuckbee 9y ago
  A question for the lawyers: If I build a model of a celebrity, do they have copyright on that? Could I sell it?
  - berberous 9y ago
    Depends on the jurisdiction. The term of art is generally 'right of publicity', not copyright. See https://en.wikipedia.org/wiki/Personality_rights https://en.wikipedia.org/wiki/Personality_rights As you might expect, California leads the way in this area in the U.S., given their interest in protecting Hollywood / celebrities.
  - gspetr 9y ago
    https://www.theverge.com/2018/1/30/16945494/deepfakes-porn-face-swap-legal https://www.theverge.com/2018/1/30/16945494/deepfakes-porn-f... Without any precedents YMMV.
algorithmsRcool 9y ago
I am reminded of Ghost in the Shell : SAC where they mention that video and photographic evidence became inadmissible in court due to how easy it was to produce fakes. I think we are perhaps 1-2 years away from this becoming a practical consideration for the judicial system.
- knodi123 9y ago
  I think the reality is that there's a burgeoning opportunity for cryptographically secure chain-of-evidence cameras. Sure, no protection is perfect, but if a manufacturer can reliably state "It will require [X] level of resources and effort to falsify data in our camera", then a lawyer can present footage and ask the jury to draw their own conclusion about whether a criminal's claims of "fake footage!" are reasonable.
  - pixl97 9y ago
    So everybody is going to go out and buy new phones and cameras? Also, using Meltdown and Specter as examples, can these companies actually build cryptographically secure devices. It's easy for a manufacture to say "Our device is secure", and it's easy for the judge to say "You go to jail for life", but it creates a big mess in the legal system when we realize 3 years later that someone extracted the private key from a device, framed somebody, and that person had been put in the electric chair between now and then.
    - knodi123 9y ago
      > So everybody is going to go out and buy new phones and cameras? Probably not. That sounds a little ridiculous. But probably so in the case of security cameras or bodycams or dashcams. > but it creates a big mess in the legal system when we realize 3 years later that someone extracted the private key from a device, framed somebody, Yeah, I covered that. Did you read my second sentence, or only the first one? Evidence can already be faked. Eye-witness testimony has been faked for millenia. Photographic evidence started being faked shortly after the invention of the camera. What you do is, make sure the jury knows how hard it would be to fake, and let them make an informed decision. Does that mean it's technically possible to frame someone? Yes! Just like it is today. Sucks to live in an imperfect world. > and that person had been put in the electric chair between now and then That's a valid argument against the death penalty. But that's a different subject entirely.
vonnik 9y ago
When I see this I think of the future of human robotics interfaces. A robot with a screen for a head, will be able to express anything we do, and tailor its responses to the environment. That, in turn, will probably make them much more acceptable in society.
- pbhjpbhj 9y ago
  Behold the helper robot whose face was anonymously hacked to be goatse and whose owner can't afford to get fixed. Buying groceries in your local supermarket ...
peterchon 9y ago
I see a new business selling face data
- jgautsch 9y ago
  I wonder if there are any companies aggregating some sort of book of faces.
  - zkomp 9y ago
    eeerm... Yeah, like Facebook (it's the actual name) (best to just spell out the obvious)
peterchon 9y ago
https://www.youtube.com/watch?v=ohmajJTcpNk https://www.youtube.com/watch?v=ohmajJTcpNk
acmecorps 9y ago
So, what's the difference between this and photoshoping someone's face to create fakes? Both are the same thing fundamentally, right? I understand the technological significance of it, but not the philosophical/legal etc differences of it.
- Myrmornis 9y ago
  It’s a good question. Is the answer that to a lot of people video is their reality, whereas static images is some crusty technology from the decades before they were born?
mrfusion 9y ago
Does this mean we can make terminator movies with Arnold for hundreds of years?
- pbhjpbhj 9y ago
  I think it means you'll be able to replace all actors in all films with Arnold if you like. Films may come with default actors and the viewer will be able to choose alternate body types or faces. Would be interesting to see how that affects Hollywood celebrity. Will the celebrities do the movies still or just use stand-ins and lens their visage to the final cut? At what point will the star of live action movies be a computer generated skin?
Myrmornis 9y ago
I can’t really see much difference between the original and the faked version. It would be better if it swapped the hair too. (Yes, I do have problems recognizing faces in real life.)
- pixl97 9y ago
  Hair will be harder in the sense it requires a lot more CPU power. Making realistic hair in video games has been a challenge.
randyrand 9y ago
In these discussions, its easy to think about the person harmed by a (known to be) fake video of them having sex. I think it's odd that we purposely ignore the joy created for the other side of (thousands and millions people). In a utilitarianism view we'd weight these sides in a 1000:1 ratio because of this. I don't think a video that I know and everyone else knows is fake is very harmful. And it's clear that many people get joy out of it. I think it should stay (though it's here to stay regardless of the law).
- chug 9y ago
  I'm not quite sure you can just hand wave away the harmfulness. It's very possible that a lot of people would feel highly violated by it. It basically takes away your control over your modesty and to an extent even the autonomy of your body. Yes, the videos are fake, but that doesn't change that the entire premise is for people to imagine it's real. I'm much less decided about the legality side of this, but I think the from the moral side, this is pretty hard to defend. Obviously this is taking things a bit to the extreme, but where exactly do you draw the line from a utilitarian perspective? Is a gang rape morally permissible if the perpetrators enjoy it enough to outweigh the victim's suffering? Why should people ever be (morally) allowed to derive their pleasure from another's suffering?
trampelpfad999 9y ago
As we all (at least in europe) remember, to know the truth is not so easy. https://www.theguardian.com/world/2015/mar/19/i-faked-the-yanis-varoufakis-middle-finger-video-says-german-tv-presenter https://www.theguardian.com/world/2015/mar/19/i-faked-the-ya... This fake video from Varoufakis made pretty huge waves.
arpgy 9y ago
Within a few years there will be apps that take movies of two different people and allow you to just tell it the parts to swap. People will be able to easily make their own videos like this easily. Banning the technology is a temporary and questionable fix at best.
bloopernova 9y ago
For those of us around when the optic nerve implants are available, it's going to be a wild time. You'll be able to experience your own subjective reality, completely differently from anyone else. Want to replace the grocery store with a Minecraft dungeon? Sure! Want to see dragons soaring in the sky above? Sure! Want to pretend your self-driving motorcycle is a speeder bike from Star Wars? Sure!
- odorousrex 9y ago
  Right out of a Vernor Vinge novel.
- blacksite_ 9y ago
  Psychedlics kind of help you get there currently. Depends on where you're looking to go, I suppose.
Pica_soO 9y ago
There will be a hollywood version of do your own story books of course.
eagsalazar2 9y ago
I keep wondering about how this approach can be composed, vs just doing a single replacement. Example: replace person a head in this video with person b (current examples), make composed person more tan, swap hair on composed person with this other person c hair, add muscles, change shirt with this other shirt, etc, etc. It really seems like the same type of operation reapplied repeatedly. Could be powerful character editor type functionality.

427 points by robinhouston 9y ago

By the time the next US election comes around it's very likely this technology, combined with Lyrebird etc. will allow anyone to make [politician] convincingly say or do anything they like. I can't make my mind up if this will make the current fake news / filter bubble situation worse, or if it will make it so bad that people learn to distrust any kind of digital media entirely. I'm still not sure, given that the use of these tools for political gain seems inevitable at this point how we will actually verify a video is real in the near future.

aluhut 9y ago
I'm surprised nobody build this Trump watersports video yet. The tool is probably still to complicated to use. There is a business opportunity.
- evansj 9y ago
  Maybe we're seeing so much coverage of deepfakes just so there is plausible deniability if the watersports video leaks?
  - aluhut 9y ago
    Nice twist.
- wand3r 9y ago
  What is "Trump watersports"? I haven't heard of it and that is a searchterm I am not super interested in getting google NSFW results from. Is it what I think it is, I haven't heard about this at all?
  - aluhut 9y ago
    There is a conspiracy theory about trump urinating on a prostitute on his visit to Russia some years ago. The theory states that Putin has a tape of it.
  - ddebernardy 9y ago
    Probably a reference to the Trump pee tape: http://www.newsweek.com/fox-news-host-jesse-watters-bizarre-reason-why-trump-pee-tape-isnt-real-if-798083 http://www.newsweek.com/fox-news-host-jesse-watters-bizarre-...
    - wand3r 9y ago
      Yep, that's what I expected. I read the article and while it hasn't been proven, if this exists and was released it would be pretty appalling.
      - ddebernardy 9y ago
        Methinks it's only a matter of time when someone creates a phony video using a software like deepfakes. What a bizarre age we live in...
  - jjgreen 9y ago
    The rumour that Russian intelligence have video of Trump getting pissed on by prostitutes, the #goldenshower hashstag is used I believe. No idea of its veracity.
    - lightbyte 9y ago
      The claim is not that he was getting pissed on, it is that he paid prostitutes to pee on a bed that the Obama's previously used.
giarc 9y ago
Next election the "grab her by the p*" soundbite would have been "fake sound". It's really two problems. One, making people say things they didn't say. And two, allowing people to deny things they really did say.
- sharemywin 9y ago
  The second probably being much more damaging.
- ctrl-j 9y ago
  > Next election the "grab her by the p*" soundbite would have been "fake sound". You mean the last election right? A lot of people believed it was faked.
  - tehwebguy 9y ago
    Well those people cannot be helped, he admitted it plainly.
    - akhilcacharya 9y ago
      No disagreement on that, but even he allegedly started saying it is faked recently.
      - j_koreth 9y ago
        source ?
        giarc 9y ago
        http://www.cnn.com/2017/11/27/politics/donald-trump-access-hollywood-conspiracy/index.html http://www.cnn.com/2017/11/27/politics/donald-trump-access-h...
        ctrl-j 9y ago
        https://www.nytimes.com/2017/11/28/us/politics/trump-access-hollywood-tape.html https://www.nytimes.com/2017/11/28/us/politics/trump-access-...
gk1 9y ago
> or if it will make it so bad that people learn to distrust any kind of digital media entirely. There are already fake news sites out there that are "so bad" you'd think nobody would believe them, and yet... Still, it's important we inform people this is possible, so that at least the more reasonable among them can view future videos through a skeptical lens. I'm sure by now many HN readers are not surprised by this technology, but many outside of HN would be blown away.
- losteric 9y ago
  Herd immunity? Perhaps naive people believe those spam sites because smarter people believe the better spam sites that share the same narrative... However, distrusting digital media isn't the solution either - there are plenty of trash magazines publishing conspiracy theories and other absurdities. Plus, digital media has revealed huge segments of America are becoming culturally isolated and undemocratic. A return to print media might shift funds back to deep professional investigative journalism that covers those issues... or just give owners better returns.
blauditore 9y ago
I don't think this will be a big issue. The same could already be done with still images years ago, if not decades. And with enough time and resources, the same can be applied reasonably well to videos, manually frame by frame or using CGI. It takes a lot more to make it undetectable as fake though, and I guess the same applies to deepfakes. Once people get used to the fact such face swaps can be done, they'll be more sceptical and demand realness verification in case of published sensitive material.
- sbarre 9y ago
  Most people barely question the validity or truth of news they see on TV, and that's real footage.. You really think those same people will actually question if a video was real in the first place, particularly if it reinforces their world-view?
- simias 9y ago
  I'm not so much concerned about people making politicians and other personalities say something they haven't, rather I worry that these personalities will now have plausible deniability for any footage of them doing or saying something embarrassing. It's always harder to prove a negative and in this case you'd have to prove that it can't be a fake. Now imagine figuring out which video is real amid a flurry of fakes.
sbarre 9y ago
Putting aside the US or first-world elections/media.. It's worrying to think of how this technology could be used in parts of the world where there is much less widespread understanding of this level of digital manipulation.
matt_wulfeck 9y ago
Your fears are greatly overblown. Haven’t we been able to “Photoshop“ people convincingly for a very long time? People are less sceptible to this than you think. Even when scandals do break, it’s not a picture or video that’s proof, it’s a picture or video that produces witnesses.
- tome 9y ago
  Video has been pretty much immune to "Photoshopping" AFAIK. The new technology also makes it extremely cheap.
  - platz 9y ago
    nein. or there was that time a german variety show 'false flagged' the germans by doctoring a video of yanis varoufakis to have him 'stick the middle finger to germany' during a talk he gave (https://www.youtube.com/watch?v=Vx-1LQu6mAE https://www.youtube.com/watch?v=Vx-1LQu6mAE). There were witnesses; but they didn't stop the confusion and outrage that ensued due to the effects of viral media. Would this pass a sophisticated technical analysis of the video? No - is there time and public understanding for the results of said sophisticated technical analysis to prevent the damage before it is done? no. The notion that witnesses or forensic analysis can combat the speed of information, especially across borders, is a bit quaint.
    - cf498 9y ago
      Its something entirely different to fake a face instead of a finger in a video. Then again, the deep fakes I have seen so far arent really that convincing either. But one can only wonder how much you can improve such a thing. With politicians of whom a wide array of video material exists and were you have the option to create an incident, instead of altering existing ones, the possibilities are really interesting.
      - platz 9y ago
        did we really post that link at the same time, or did you edit your other response to match?
psergeant 9y ago
I wrote some thoughts on this: https://github.com/pjlsergeant/multimedia-trust-and-certification https://github.com/pjlsergeant/multimedia-trust-and-certific...
deegles 9y ago
You can't prevent someone from making a fake, but one could at least use cryptographic signatures to prove they made a statement on video/audio.
tboyd47 9y ago
We need to start thinking about what criminal justice will look like when digital recordings are all considered hearsay. That's bound to happen. And perhaps it should... if average people have the technology now, how long has it been in the hands of police departments, newspapers, militaries, and intelligence agencies? We tend to focus on fakes with high-profile people like politicians and celebrities, but it can clearly be done with everyday people as well. The author was able to produce a fake video segment of a well-lit, animated conversation involving his wife. She's directly in front of the camera, speaking and making hand motions for several seconds. How much easier would it be to insert a face into grainy security camera footage, or a shaky clip of a large crowd at a political rally?
- 794CD01 9y ago
  The presence of one source can be easily dismissed, but if you have 500 independent sources that produce video of the same event from their own angles and timespans in an interconsistent way, it's far more likely that the recorded event happened as opposed to all of these different sources lying.
  - tboyd47 9y ago
    Really? How much harder is it to run the OP's script with 500 input videos as opposed to only one? Especially if they're very similar? How many people have the expertise needed to identify a faked video, and how much training would they need?
    - jakear 9y ago
      The great part is that using a GAN means we cannot automatically distinguish a fake from a real... because that's the whole point of the GAN.
    - 794CD01 9y ago
      At that point there is no such thing as identifying faked videos. Instead you'd rely on the abundance of independent videos and see how consistent they are. Generating a lot of them doesn't do you any good unless you can insert them into the black boxes of the 3 planes and 40 cars in the area, the databases of all the drone operators, the internal storage of all the nearby smartphones, etc.
794CD01 9y ago
People should never have been trusting media in the first place, digital or otherwise. And for the most part, they don't have to. Humans believe what they want to believe or are told to believe. Evidence has little to do with it.
douglaswlance 9y ago
Information will need to be verified by consensus on a decentralized blockchain.
tw1010 9y ago
Just like there is a social system in place to sift fact from rumors (via watercooler chatter, snopes, networks of trust), at least to a degree where society doesn't collapse into chaos, in the same way will we be able dampen any damage that this new technology might cause. I think this whole "fake videos of politicians" will become a way smaller issue than what doomsayers predict it will be.
konschubert 9y ago
It's basically the same situation as we had with print before television. It's all about which sources you trust. In fact, it has always been that way. Deepfakes just makes it more obvious. Which is maybe a good thing.
zodPod 9y ago
Something that comes up a lot on the deepfakes sub is the notion that this should make people more skeptical. They use the phrase "If anything can be real, nothing is real" and I think it makes a lot of sense.
- macawfish 9y ago
  That's not necessarily a good thing. Sometimes it's necessary for people to believe something factual. Blanket skepticism can be very destructive in these cases.
JohnStrange 9y ago
It will make things way worse, since politicians from now on can and will plausibly deny any video evidence about them. Candidates will literally rewrite history instead of merely suffering from amnesia and any real evidence will end up in battles between one TV show 'expert' against another. On a bright side, young aspiring actors can make porn movies to get some cash early in their career without necessarily having to fear it will ruin their career many years later - those movies will exist anyway.
rexpop 9y ago
Hopefully, this makes celebrity untenable.
Asdfbla 9y ago
I find the more concerning thing is not that people will believe that a fake is real, it's more that they won't believe something real is real anymore. Already nowadays the whole thing about fake news and alternative facts has segments of the population flat out not believing the truth. This may very well be greatly exacerbated in the future where, regardless of the amount of evidence you bring, people will just disregard it as fake.
- zodPod 9y ago
  If it can be faked, is it really evidence? I mean, you're worrying about people not believing assertions at that point. This is already something that can be faked just not by the masses yet and not easily. You should always look for multiple sources of anything and not just believe one source that shows you any evidence of something being true.
  - zokier 9y ago
    There was this essay posted on HN some time back, where the main thesis was that teaching people to be critical about media strengthened the positions of antivaxers, flatearthers and so on, because they did not rely on traditional media so they were then sort of considered more trustworthy. And now deepfakes etc throws just more fuel in the fire that burns trust, people end up more and more believing things that happen to make them feel good.
- olivermarks 9y ago
  Believing 'the truth' is a huge can of worms here. Verifying the legitimacy and accuracy of 'news' claims and representations has always been very difficult for the lay person. We are now well into an era when 'the truth' is even harder to find. 'Believe nothing of what you hear and only half of what you see' (Edgar Allen Poe) has now become 'believe nothing of what you see' too, at least until you have verified to your level of satisfaction...
clarkmoody 9y ago
It's time for cryptographic signatures in cameras (phones and otherwise) so that you can prove something is unadulterated. Apple signing key -> your phone's key -> signature over hash of the media (edit: for video, embed signatures every so often to you can verify a shorter clip out of the whole). Embed Apple's signature and your pubkey in the metadata of the image, and TwitYouFace will verify on upload that it's authentic. Of course, you can't fix stupid unless you talk about a complete overhaul of public education... And even then, people are people.
- ggg9990 9y ago
  That doesn’t prove anything. Make your fake video, point your phone camera at it, record.
  - bufferoverflow 9y ago
    Try it. Seriously, try and record a realistically looking recording off the screen. Movie pirates have been trying to perfect it for decades, and even their best attempts look obvious.
    - 05 9y ago
      Unless you build DRM right into camera sensor (which is, admittedly, not that hard), you can use an FPGA chip to receive your source video in HDMI format, convert it to CSI interface and send it to the phone SoC over the camera flex cable.
    - ggg9990 9y ago
      If you can make a porn video of someone look like a porn video of someone else, I’m sure the technology to solve this problem is not far off.
  - clarkmoody 9y ago
    Maybe a second layer of AI that cleans up that video-of-video look? Philosophically though, I think it's a good thing that many are learning not to trust information from self-proclaimed authorities :-)
root_axis 9y ago
It's likely that we'll soon enter a new era of technological cat-and-mouse as tools are developed to automatically detect artifacts caused by digital processing and counter-techniques are developed to minimize the appearance of those artifacts.
filoeleven 9y ago
How will we actually verify a video is real in the near future? I’ve been thinking about this a lot lately, for many of the same reasons. I think one answer can be found in device-level signing of images and A/V feeds. A recorder can integrate its cryptographic signing and keystore on the same chip as its CCD/audio controllers, like Apple’s T2 chip does, so that it has the ability to sign direct feeds from the hardware as being such. This might be more complicated for A/V than still images, since a large enough recording would have to be written to disk. Perhaps some rolling scheme could be devised. For an added measure of verification, the SoC could upload them (optionally encrypted) to a distributed public store like a blockchain or IPFS, proving that they were created at/around a certain time. This seems like it would be useful in concert with other tools for news agencies (who should be leading the way in providing as much documented research and evidence for things as they can, as a matter of course, while protecting their sources), law enforcement (who is losing the public’s trust when it comes to the chain of custody and testimony), etc. We jumped very far ahead both in how much importance we place on digital documents and in how easily we can forge them, so we need to catch up fast on finding robust mechanisms for verification.
UncleMeat 9y ago
Kerry was getting photoshopped into propaganda photos in 2004. The new thing isn't fake documents. The new thing is a highly scalable social media infrastructure to share false documents widely.
amai 9y ago
There will be no next election in the US.

jacquesm 9y ago

Next up, put yourself and/or family members into any movie. That's really only a matter of time.

petercooper 9y ago
When I was a kid in the mid 80s, I got a racing game (Revs) that let you enter names in for the CPU racers. Looking back it seems ridiculous but I remember it being such a thrill to pretend I was racing against "people" I knew. Even that was exciting to me, so what's happening now is absolutely mindboggling(!)
- 0xdeadbeefbabe 9y ago
  The names example calls for more imagination though. Yeah it's mind boggling, but I wonder if the effect will be about the same or that much more.
  - petercooper 9y ago
    So maybe the "uncanny valley" could even ruin the effect? Some of the samples so far are really good though, so maybe we skip that stage..
icefox 9y ago
Before movies, why not insert friends into your video game? Or how about creating a company that sells profiles each of which would be a collection of high quality photos/videos. In turn if you have a video game or movie you can get a random actor with the body you want and swap the face with the face you want? Or even better create a service where individuals can upload their own photos which they can get some of the profit if their profile is used.
- _up 9y ago
  Another Idea make Bad Actors more convincing with "stock reactions". Track the face of your actor copy Anthony Hopkins reaction face over it and than use deepfake to get the original face back.

digitalsushi 9y ago

Whatever you think about this, it's here to stay - the code is open source and cached on a million hard drives already. If we treat the normal rate of progression, we'll see a year 2023 version for 5 bucks off the App Store, your phone will live-convert the video you're recording at your drinking party with your friends into an all celebrity event. Probably with voice swap. Your drunken roommate's scratchy voice will be ML'd into Frank Sinatra, autotuned, face swapped, and uploaded for a few laughs to facebook. It's here to stay whether you like it or not.

trs80 9y ago
Then I will exit my home with a scrambler mask.
- mental1896 9y ago
  That'll do until anti-scrambler tech comes out
  - baseten 9y ago
    it will be no match for my anti-anti-scrambler tech.
- toomuchtodo 9y ago
  Until gov or Equifax leaks your ID photo.
petercooper 9y ago
Agreed, although I think what you're describing is how it'll be sold, yet most people will surreptitiously use it to indulge in fantasies with "normal" people which will be even more damaging. Got a picture of that co-worker you like? Congrats, now you have him/her in some porn. Interesting times for sure and I think we'll see a lot of flak and fallout in people's personal lives from their creations, far more than we see from "typical" porn use now.
- 9y ago

ghostcluster 9y ago

Is it just me or are these "deep fakes" just not even very convincing? I felt the same way about all the Nicholas Cage ones that were viral recently. It's so obviously a very shallow 'effect' and can't compensate for very basic differences in bone structure, hair, etc.

digitalsushi 9y ago
Regardless of whether this is a good or bad technology to have, I can all but guarantee it will only improve with time.
lucideer 9y ago
They'll get more convincing. Then we'll actually get better at recognising subtle differences. Then they'll get even more convincing. Someone will develop algorithms to identify those fakes. Then they'll be further developed and improved to the point that they're impossible to distinguish at every level.
- yjftsjthsd-h 9y ago
  > Someone will develop algorithms to identify those fakes. One twist: don't adversarial generative networks already work by including a "fake detector" and beating it? This cat-mouse game might last, like, all of a week.
  - lucideer 9y ago
    > This cat-mouse game might last, like, all of a week I'd give it two at least.
isolli 9y ago
Like everything in AI, it's probably just a matter of time now.
ghostbrainalpha 9y ago
Are you talking about the pornography being unconvincing or the GIF from this article? Fake: http://svencharleer.com/blog/wp-content/uploads/2018/02/anne-1.gif http://svencharleer.com/blog/wp-content/uploads/2018/02/anne... Original: http://svencharleer.com/blog/wp-content/uploads/2018/02/elke-1.gif http://svencharleer.com/blog/wp-content/uploads/2018/02/elke... I could see the fake being easy to detect at high resolution, but personally I have a 0% chance of catching this gif as fake.

deleted 9y ago

[deleted]

deleted 9y ago

[deleted]

taoistextremist 9y ago

In the realm of innocent fun with this stuff, just how much time would it take to insert myself into the entirety of a movie? I'm just imagining how fun it would be to invite friends around to watch something we're all excited about, only to convincingly insert my face into the main character's.

tomaskafka 9y ago
Or, once advert industry learns about it, how about replacing all the things on the table in Audrey Hepburn movie with BRAND NEW LEMONMELON FLAVOURED OREOS! :) Ideally on the fly, with each box sold to the highest bidder, adsense-style.
- logfromblammo 9y ago
  I have also thought about this, and whether I could live with myself after making a huge pile of money to relentlessly annoy every other human on Earth. I conceived it as video bloggers setting up a dynamic replacement volume in their scenes, and compositing a 3-D scene for the sponsored product into it as the stream goes out. One viewer might see an open Coke bottle. Another might see a half-eaten package of Oreos. Another might see framed photographs of adoptable rescue pets. All the vlogger does is avoid entering the ad-volume and the software does the rest. Inserting into old movies would require a bit more sophistication, I think. You have to discern the geometry of the scene, model it, then discover the insertable volumes, match the camera movement in software, and finally use a shader to match the object textures to the film grain. But let's not do this. Please.
  - yjftsjthsd-h 9y ago
    > Inserting into old movies would require a bit more sophistication, I think. You have to discern the geometry of the scene, model it, then discover the insertable volumes, match the camera movement in software, and finally use a shader to match the object textures to the film grain. Only if you're changing volume. We can start with swapping flat surfaces (say, cereal boxes).

ploggingdev 9y ago

Discord banned multiple servers that r/deepfakes created and their reasoning was that it's being used for creating involuntary porn and so is against their ToS. I'm curious to hear your thoughts on the legal status of deepfakes as well as the moral and ethical questions behind the usage of the tech. A thought experiment : I run a chat room site and will be adding support for user created chat rooms (something like discord for cryptocurrency communities with real time price widgets etc). Let's say I went ahead and got them to use my chat room, what would the consequences be? And what do you think about the moral and ethical questions behind hosting such a community? Am I now enabling the creation of deepfakes? By choosing to avoid hosting such a controversial community, am I imposing my moral and ethical reasoning on others (the free speech argument)? How do think about the deepfakes project, which is mostly being used for creating celebrity porn, but the tech itself is interesting and has applications(targeted ads or in stunt doubles etc)? To be clear, I'm not trying to host a chat room for them, this is just a thought experiment. I don't like what it's being used for and don't want to support it in any way.

ggg9990 9y ago
Legally, in the US, if you own the copyright to the source images, the copyright to the porn, and you’re not selling the output, it’s 100% legal
- zawerf 9y ago
  There was a thread on /r/machinelearning titled "Is it legal to use copyright material as training data?" https://www.reddit.com/r/MachineLearning/comments/4qrgh8/is_it_legal_to_use_copyright_material_as_training/ https://www.reddit.com/r/MachineLearning/comments/4qrgh8/is_... But the issue here isn't whether training on copyrighted face images is legal. You can totally take copyright-free photos of anybody as long as they appear in public. The more interesting question I think is whether you own the copyright to the 3D capture of your face. For example disney won't let you get away with using models of their characters. Don't I have the right to not let anyone use the "model" that I spent my literal life growing?
  - 9y ago

chrischen 9y ago

Is there an inplementation of a generalized face swap that doesn’t require hundreds of images of both the target and subject?

bpicolo 9y ago
Keep in mind that one video counts as hundreds of images

ChuckMcM 9y ago

It makes the use of stunt doubles easier as well since the actor's face can be mapped to the body double. And it isn't like this point in time wasn't predicted by just about everyone in CS. Given enough computer power and the right algorithm pretty much any data is fungible, video or otherwise. But we've just entered the most dangerous period when most people still believe what they see in a video is what was originally videoed. It is when people will use this technology for deception and get away with it like they did with Photoshop before 'shopped!' came into the more general populations consciousness.

dllthomas 9y ago
I don't disagree, but I hope that the transition will be eased by the fact that the photoshop transition is within living memory of so many.
HorizonXP 9y ago
A cool, possible solution would be to have every participant in the video to digitally sign the final file to verify that they're actually in the video, consensually. That way, if anyone finds a deep fake video, it would be easy to dispute it since it doesn't have the person's signature. Of course, this would be exceedingly difficult in practice, since you'd need the signing to be opt-out instead of opt-in for it to be truly effective.

adonnjohn 9y ago

Is it possible to utilize deepfake tech to crack iPhone Face Id?

dawnerd 9y ago
No more than a picture of the person, which you'd need anyways, so I don't see the point?
hmate9 9y ago
The two are unrelated. Face ID looks at a 3D face not a 2d video

dubin 9y ago

To add a data point, a friend and I made one of these over the weekend (with Trump and Michael Scott https://youtu.be/0Rexuh-VY6E https://youtu.be/0Rexuh-VY6E) It was shockingly easy to do. We're technical, but neither of us know a lick about machine learning. It took a couple hours to collect training data (we turned speeches/interviews from each of them into thousands of photos), and 20 hours to train the model. In the future you could automate the data collection for a person even more, to the point where they just need to film a selfie of themselves for a couple of minutes in a couple of different lightnings, and boom, after you train their decoder, you could put them on any celebrity. EDIT: This is what we used https://github.com/deepfakes/faceswap https://github.com/deepfakes/faceswap Would be happy to walk anyone through how to do this

howToLearnSpark 9y ago
the Trump clip is gold! Please do more!
- dubin 9y ago
  Heh, thank you! Any suggestions welcome :)
michaelbuckbee 9y ago
A question for the lawyers: If I build a model of a celebrity, do they have copyright on that? Could I sell it?
- berberous 9y ago
  Depends on the jurisdiction. The term of art is generally 'right of publicity', not copyright. See https://en.wikipedia.org/wiki/Personality_rights https://en.wikipedia.org/wiki/Personality_rights As you might expect, California leads the way in this area in the U.S., given their interest in protecting Hollywood / celebrities.
- gspetr 9y ago
  https://www.theverge.com/2018/1/30/16945494/deepfakes-porn-face-swap-legal https://www.theverge.com/2018/1/30/16945494/deepfakes-porn-f... Without any precedents YMMV.

algorithmsRcool 9y ago

I am reminded of Ghost in the Shell : SAC where they mention that video and photographic evidence became inadmissible in court due to how easy it was to produce fakes. I think we are perhaps 1-2 years away from this becoming a practical consideration for the judicial system.

knodi123 9y ago
I think the reality is that there's a burgeoning opportunity for cryptographically secure chain-of-evidence cameras. Sure, no protection is perfect, but if a manufacturer can reliably state "It will require [X] level of resources and effort to falsify data in our camera", then a lawyer can present footage and ask the jury to draw their own conclusion about whether a criminal's claims of "fake footage!" are reasonable.
- pixl97 9y ago
  So everybody is going to go out and buy new phones and cameras? Also, using Meltdown and Specter as examples, can these companies actually build cryptographically secure devices. It's easy for a manufacture to say "Our device is secure", and it's easy for the judge to say "You go to jail for life", but it creates a big mess in the legal system when we realize 3 years later that someone extracted the private key from a device, framed somebody, and that person had been put in the electric chair between now and then.
  - knodi123 9y ago
    > So everybody is going to go out and buy new phones and cameras? Probably not. That sounds a little ridiculous. But probably so in the case of security cameras or bodycams or dashcams. > but it creates a big mess in the legal system when we realize 3 years later that someone extracted the private key from a device, framed somebody, Yeah, I covered that. Did you read my second sentence, or only the first one? Evidence can already be faked. Eye-witness testimony has been faked for millenia. Photographic evidence started being faked shortly after the invention of the camera. What you do is, make sure the jury knows how hard it would be to fake, and let them make an informed decision. Does that mean it's technically possible to frame someone? Yes! Just like it is today. Sucks to live in an imperfect world. > and that person had been put in the electric chair between now and then That's a valid argument against the death penalty. But that's a different subject entirely.

vonnik 9y ago

When I see this I think of the future of human robotics interfaces. A robot with a screen for a head, will be able to express anything we do, and tailor its responses to the environment. That, in turn, will probably make them much more acceptable in society.

pbhjpbhj 9y ago
Behold the helper robot whose face was anonymously hacked to be goatse and whose owner can't afford to get fixed. Buying groceries in your local supermarket ...

peterchon 9y ago

I see a new business selling face data

jgautsch 9y ago
I wonder if there are any companies aggregating some sort of book of faces.
- zkomp 9y ago
  eeerm... Yeah, like Facebook (it's the actual name) (best to just spell out the obvious)

peterchon 9y ago

https://www.youtube.com/watch?v=ohmajJTcpNk https://www.youtube.com/watch?v=ohmajJTcpNk

acmecorps 9y ago

So, what's the difference between this and photoshoping someone's face to create fakes? Both are the same thing fundamentally, right? I understand the technological significance of it, but not the philosophical/legal etc differences of it.

Myrmornis 9y ago
It’s a good question. Is the answer that to a lot of people video is their reality, whereas static images is some crusty technology from the decades before they were born?

mrfusion 9y ago

Does this mean we can make terminator movies with Arnold for hundreds of years?

pbhjpbhj 9y ago
I think it means you'll be able to replace all actors in all films with Arnold if you like. Films may come with default actors and the viewer will be able to choose alternate body types or faces. Would be interesting to see how that affects Hollywood celebrity. Will the celebrities do the movies still or just use stand-ins and lens their visage to the final cut? At what point will the star of live action movies be a computer generated skin?

Myrmornis 9y ago

I can’t really see much difference between the original and the faked version. It would be better if it swapped the hair too. (Yes, I do have problems recognizing faces in real life.)

pixl97 9y ago
Hair will be harder in the sense it requires a lot more CPU power. Making realistic hair in video games has been a challenge.

randyrand 9y ago

In these discussions, its easy to think about the person harmed by a (known to be) fake video of them having sex. I think it's odd that we purposely ignore the joy created for the other side of (thousands and millions people). In a utilitarianism view we'd weight these sides in a 1000:1 ratio because of this. I don't think a video that I know and everyone else knows is fake is very harmful. And it's clear that many people get joy out of it. I think it should stay (though it's here to stay regardless of the law).

chug 9y ago
I'm not quite sure you can just hand wave away the harmfulness. It's very possible that a lot of people would feel highly violated by it. It basically takes away your control over your modesty and to an extent even the autonomy of your body. Yes, the videos are fake, but that doesn't change that the entire premise is for people to imagine it's real. I'm much less decided about the legality side of this, but I think the from the moral side, this is pretty hard to defend. Obviously this is taking things a bit to the extreme, but where exactly do you draw the line from a utilitarian perspective? Is a gang rape morally permissible if the perpetrators enjoy it enough to outweigh the victim's suffering? Why should people ever be (morally) allowed to derive their pleasure from another's suffering?

trampelpfad999 9y ago

As we all (at least in europe) remember, to know the truth is not so easy. https://www.theguardian.com/world/2015/mar/19/i-faked-the-yanis-varoufakis-middle-finger-video-says-german-tv-presenter https://www.theguardian.com/world/2015/mar/19/i-faked-the-ya... This fake video from Varoufakis made pretty huge waves.

arpgy 9y ago

Within a few years there will be apps that take movies of two different people and allow you to just tell it the parts to swap. People will be able to easily make their own videos like this easily. Banning the technology is a temporary and questionable fix at best.

bloopernova 9y ago

For those of us around when the optic nerve implants are available, it's going to be a wild time. You'll be able to experience your own subjective reality, completely differently from anyone else. Want to replace the grocery store with a Minecraft dungeon? Sure! Want to see dragons soaring in the sky above? Sure! Want to pretend your self-driving motorcycle is a speeder bike from Star Wars? Sure!

odorousrex 9y ago
Right out of a Vernor Vinge novel.
blacksite_ 9y ago
Psychedlics kind of help you get there currently. Depends on where you're looking to go, I suppose.

Pica_soO 9y ago

There will be a hollywood version of do your own story books of course.

eagsalazar2 9y ago

I keep wondering about how this approach can be composed, vs just doing a single replacement. Example: replace person a head in this video with person b (current examples), make composed person more tan, swap hair on composed person with this other person c hair, add muscles, change shirt with this other shirt, etc, etc. It really seems like the same type of operation reapplied repeatedly. Could be powerful character editor type functionality.