12 ms·
Fake celebrity porn is blowing up on Reddit, thanks to artificial intelligence
- r3bl 9y agoThe was a pretty good discussion yesterday in /r/cyberpunk about the possible consequences of this: https://www.reddit.com/r/Cyberpunk/comments/7sexm6/deepfakes_porn_fakery_realistically_pasting/ https://www.reddit.com/r/Cyberpunk/comments/7sexm6/deepfakes... > The subreddit /r/Deepfakes became very active very fast and new deepfakes are submitted every day with varying degrees of realism. The most scary part is that ANYONE can be deepfaked, not just celebrities. Provided you have the right hardware (because neural networks demand beefy video cards for training) you could train a model of your friend and paste her face onto a porn video and boom. All you have to do is download a browser extension that downloads all photos from someone's instagram and work from there. Nobody is safe from this. > I think this here is as cyberpunk as it gets. The technology is 4 months old and has already yielded extremely realistic results. Think of what we will have one year from now. Something like this matches both the high tech and the low life aspect of the cyberpunk genre. EDIT: Pasted the wrong link.
- deleted 9y ago[deleted]
- hackinthebochs 9y agoRevenge porn doesn't even need to be real anymore to terrorize someone. A celeb has plausible deniability when it comes to stuff like this. But an average person whose career is on the line?
- hndamien 9y agoNow they can just say, "oh, I've been deepfaked" and move on. In many ways this will be psychologically liberating for many victims of revenge porn.
- kevingadd 9y agoThat assumes a really high level of technology literacy from the people the fake is being used to deceive. That simply isn't going to be true, probably ever. It's already easy to convince people of transparent falsehoods, plausible faked video evidence will make it worse. It's the "Give me six lines written by the most honest man in the world, and I will find enough in them to hang him" problem except now it's "give me six photos of the most honest man and I will convince everyone he loves to abandon him". There are notable examples of harassment mobs forming and never dissipating off of really scant "evidence", things like accusing school shooting parents of being "crisis actors" or long harassment campaigns based on flimsy claims that a woman slept around. It's disgusting. School bullying leading to suicides is already bad enough, what about when teens are sending forged porn of each other around? Or to classmates' parents to get them in trouble?
- dragonwriter 9y ago> It's already easy to convince people of transparent falsehoods, plausible faked video evidence will make it worse It will for a while, then it'll revert to the point that video “evidence” is—except to experts validating validation tools that most people don't understand—exactly equal to rumor in public estimation, because everyone has been exposed to so much fakery.
- deleted 9y ago[deleted]
- Godel_unicode 9y ago> That assumes a really high level of technology literacy from the people the fake is being used to deceive. Counterpoint: "photoshopped" is a dictionary word.
- hndamien 9y agoIt only took 8 years for people to become Bitcoin experts.
- procedural_love 9y ago> We assume, too, that face swapping is the end game, but it’s clearly just the beginning. Isn't the end game an endless stream of personalized content for everyone? Wherein the entire corpus of human-created media becomes a training set for our fantasies. It is interesting how entertainment is again pushing the boundary of technology. Soon enough this push to make face editing tools for porn more accessible to everyone will allow anyone to: 1) Replace their ex-husband's face in their old family videos with their new husband's face. 2) Create a viral video of Donald Trump murdering someone. 3) Be the star of their favourite movie, porn or otherwise. (What's the effect this would have on people's memories, when they actively see themselves doing everything James Bond does, for instance? Shooting people, being generally powerful, and "getting the girl"?)
- deleted 9y ago[deleted]
- opportune 9y agoThe technology for all of these exists, it's just a matter of motivation. see: https://www.youtube.com/watch?v=ttGUiwfTYvg https://www.youtube.com/watch?v=ttGUiwfTYvg
- IntronExon 9y agoThings are going to get very weird in porn, when you don’t have convince a human to actually do it. I have to assume that early adopters will also be people with predilections which are unserved, or illegal. If people worry about their kids seeing disturbing porn now, imagine when it’s AI generated, photorealistic rape, snuff, child porn. Illegal or not, if it’s purely virtual law enforcement is going to focus on the subset of crimes which involve actual human victims.
- hndamien 9y agoWill they really? There are already several thought crimes.
- jstarfish 9y ago> imagine when it’s AI generated, photorealistic rape, snuff, child porn. Illegal or not, if it’s purely virtual law enforcement is going to focus on the subset of crimes which involve actual human victims. In the US, all of that is already illegal. If you put yourself in a position where what you possess is indistinguishable from the real thing, the courts err on the side of the potential victim. Law enforcement's priorities are not going to change; they don't distinguish between what's virtual or not. If it looks like CP, you can't point to a producer with valid 2257 documentation and it isn't obviously a cartoon, you're cooked.
- herogreen 9y agoMaybe one day everyone will be using cameras with a combination of digital signatures and watermarking technologies. That said not being able to definitely classify videos between "faked" ones and "original" ones could help people suffering from revenge porn (or political manipulations)
- BoiledCabbage 9y agoThis here seems to be the only possible way forward, other than throwing out all concept of video evidence. All video streams (and frames) are signed / watermarked with the source that recorded them. Video editing now becomes impossible without showing a change in ownership. Unfortunately it will be a long number of decades until we can successfully replace the majority of cameras with these new ones that society doesn't yet have reason to understand the need for.
- mirimir 9y agoStrong authentication might protect video in news and court contexts. But I don't see how it'd help for celebrity porn, revenge porn, and other contexts where there is no authenticated original. Unless all the originals were authenticated. But that'd require mandatory authentication on all recording devices, which is rather frightening from a privacy perspective.
- BoiledCabbage 9y ago> But that'd require mandatory authentication on all recording devices, which is rather frightening from a privacy perspective. Yes inherently scary, but is there a solution to still allow a user to maintain control as to when they give up their privacy? Ex. a 3rd party escrow service that will authenticate a video belongs to a user without revealing who the user is? Or some other key sharing scheme that allows users to decide when they want to take ownership of a whistle-blower video or some other govt corruption video in a repressive country? I'm not a public key crypto expert, but I can't imaging this is the first time user controlled authentication has been investigated. The requirement would be that everything is always signed, but (through key chaining, sharing...) a user gets to decide if they reveal their association to it.
- wasx 9y agoThis is... scary. The potential applications of this technology extend far beyond porn. How long until intelligence agencies are using this sort of technology to sabotage political opponents?
- XorNot 9y agoLow resolution security footage is about 1 year from extinction.
- kevin_thibedeau 9y agoWho's putting up the capital to replace millions of low res cameras?
- arca_vorago 9y agoOr to build the infrastructure needed to support anything else?
- UncleMeat 9y agoHow is this new? Kerry was photoshopped into controversial photos in 2004. Now it is video, yes, but fraudulent photos have convinced plenty of people already.
- wasx 9y agoVery true. More famously there were photos of Hitler engaging in debauchery produced by the Brits in WW2, which is why I believe such an act has precedent. But producing videos like this where if done well, you literally struggle to tell the difference between real and fake is much more different than some Polaroids or Photoshop's. It's new due to the relative ease that highly sophisticated fakes can be produced
- JetSpiegel 9y ago> How long until intelligence agencies are using this sort of technology to sabotage political opponents? https://en.wikipedia.org/wiki/Nikolai_Yezhov#Execution https://en.wikipedia.org/wiki/Nikolai_Yezhov#Execution
- GuiA 9y agoThe inevitable outcome is that no recorded media will be taken at face value unless there is immense proof in some way of its veracity. In the short term, this will probably lead to all kinds of terrible things (kids getting bullied through computer generated imagery of them, people being fired for videos of them saying things they never said, jealous spurned lovers attempting to break apart marriages with fake videos, etc.) In the long term, it might actually be a good thing - instilling a strong sense of caution for anything that claims to be recorded from the real world.
- gknoy 9y agoHow do you hold people accountable for bad behavior (e.g. police behaving poorly when making an arrest or traffic stop) when the video evidence is viewed as "easily faked"? I mean, eyewitness testimony has been proven many times to be faulty, but presumably someone with an axe to grind (and some AWS credits?) could make fake videos of all sorts of bad things _from multiple perspectives_. Scary indeed.
- Chaebixi 9y agoWell in the police conduct case, if they're wearing a body camera, you can still have a properly-documented chain-of-custody backing up the photo evidence. We've had good technology for the forgery of printed documents for a long time, and the world hasn't ended.
- Angostura 9y agoCameras that digitally sign each frame?
- PeterisP 9y agoAll that does is link that frame to [a key in] the particular camera. It doesn't verify that the frame was "real" in some way. It defends whoever controls [the key in] that camera from others secretly tampering with that frame afterwards, but it doesn't defend others from tampering by whoever controls [the key in] that camera. Central signature verification or blockchain could verify a timestamp - i.e. prove that the frame was taken (or maliciously created) before a particular time. That's about it.
- bob_theslob646 9y agoThis seems like the beginning of a giant problem. First we had Adobe able to replicate a user's voice after listening to it for 20 to 40 minutes, now this. I guess vein scanning is going to happen sooner or later in order for personal verification.
- ryanmarsh 9y agoSo I clicked the link in the article (for science, so you don’t have to) and I’m blown away. People are doing this on home computer rigs? I thought I was going to find some really crappy paste jobs but instead I found myself having to completely second guess what I was seeing. Some of the videos of course suffer from odd minor defects that give up their authenticity but others were flat out as real as anything else I’ve ever seen. Now I’m concerned about the implications of this. We already know any image can be faked and almost any video but we also laugh at people who say the moon landings were faked. Given this though how could anyone believe video evidence of say, the president with Stormy Daniels, which is a matter of unfortunate import with real consequences? How hard would it be to fake an international incident from multiple vantage points?
- Florin_Andrei 9y agoI've taken some machine learning classes and I've played with TensorFlow a bit. None of this was a big surprise to me. But the overall implications are deeply troubling. I am tempted to say we're not entering a "post-truth" era, but more like a "post-reason" era. It's almost like rational thought has painted itself in a corner. These videos are almost like a mathematical "proof" too complex to be independently verified, or too complex for a human (and those are happening too). If reason is hitting a ceiling, I'm not sure what else we could use to steer a coherent society through whatever murky waters might lay ahead.
- carapace 9y agoYou go from rule-based thinking to feedback-based interaction. Part of the picture is described by the old saw about international politics: "No permanent allies, only permanent interests." I do believe we are entering what I call a "trans-rational" era. There are stable strange attractors beyond the Age of Reason. Our technology is forcing us to confront the questions of who we are and what we want to do with ourselves.
- dictum 9y agoFirst step towards this: https://news.ycombinator.com/item?id=6272626#6272744 https://news.ycombinator.com/item?id=6272626#6272744
- Faaak 9y agoThis technology makes me afraid because it deconstructs many core value I have. You can do plenty of things with it. Create fake porn among others. But is it ethical ? Is it ethical to create fake porn with a celebrity face ? With your ex-girlfriend's ? What about if you keep it for your own personal use ? But then were are the limits ? What about a "teen" ? What about fake child pornography from pictures of childs you found on the internet that permit to alleviate yourself without causing any harm to others ? I would at first think its okay if you keep it for private and own use, but then that becomes scary because we've been "trained" to think that child porn is not okay. Don't know why this is been downvoted ?
- stcredzero 9y agoWhether a tool is ethical is a different question is not the same as the question of whether various activities done with the tool are ethical.
- Faaak 9y agoHonest question: what's the problem if you keep it to yourself ?
- betenoire 9y agoAnother honest question, do you think "keeping things to yourself" really means you affect only yourself?
- stcredzero 9y agoReminds me of what Diana Davison says about the "Male Gaze." There's this old theory from the ancient world, later debunked by muslim academics, called "Emission Theory" where the eyes see by projecting "rays" which reach out and physically touch the world. It's her theory that a good number of feminists who object to the "Male Gaze" are actually using mental models of the world that include Emission Theory. Apparently, there's a large fraction of the populace that believes this erroneous theory, much like many people believe "there's no gravity in space" and think the Earth gets colder in Winter because it's farther away from the Sun. (Though many of those people will then tell you it's Summer in Australia while it's Winter in North America.) The idea that private activities have some kind of outside effect is quite akin to the idea that homosexuals being homosexual cause earthquakes.
- hirundo 9y agoTechnology is degrading the value of photo and video evidence (and probably audio too) asymptotically toward that of famously unreliable testimony from memory. Criminality becomes less risky and/or innocence becomes less protective. Law becomes less effective. A bad result, to the extent that the law isn't an ass. On the plus side artistic tools that help materialize internal life become more effective. We can interact with our dreams and fantasies more readily, to potentially therapeutic benefit. It's hard to say whether this trend holds more danger or promise.
- Chaebixi 9y agoI wouldn't be that pessimistic just yet. It's yet possible that new advances in authentication technology might counteract some of these trends.
- unethical_ban 9y agoWhat do you mean "authentication technology"? Tamper detection? The ability to see that a tool was used? It may slow things down, but this is an arms race.
- BoiledCabbage 9y ago> What do you mean "authentication technology"? Cryptographic signatures. Ie every frame in a video is signed with a 512-bit key that states authoritatively what camera was the source of the video and when it was taken. In order to change any pixels in the video you'd break this key and need to resign it. An attacker would be unable to do this unless they had physical access to the original camera. But it'll be at least 3 decades before this technology is commercialized, people see the demand for it, and the majority of all cameras in the world are replaced by it. Even if this new tech is on the market in a decade (simple tech, no demand / ecosystem yet), but 90% of existing / installed cameras don't have the feature then fake videos still get created with them. Only once ~80% of videos are authenticated, and a significant portion of the remaining 20% are fakes will people be able to dismiss non authenticated video. Up until then it's fakes non-stop. We've got some ugly decades coming up.
- Raphmedia 9y agoI've been thinking about this for a long while. I think this is good. With face recognition, old pictures you might have posted online are very easy to find. Some ex-boyfriend shared a naked picture of you? You are screwed. Now, you can simply say that it is a deepfake. Everyone will have naked pictures of "themselves" online, even if they are fake.
- deleted 9y ago[deleted]
- albertgoeswoof 9y agoI agree - taking this even further, why would you consume these pictures in the first place, if you can just deepfake them.
- peterjlee 9y agoFinding the origin of contents are already a challenge these days. A random idea I have is creating a blockchain where content creators can register their creations to prove their origin. Also, camera manufacturers can get involved and build in a hardware that signs every picture and video captured by that camera which can then be registered to the blockchain.
- rland12 9y agoKodakcoin?
- no1youknowz 9y agoWhen I look at the future, I think back at these videos. Hells Club: Part 1: https://www.youtube.com/watch?v=QajyNRnyPMs https://www.youtube.com/watch?v=QajyNRnyPMs Part 2: https://www.youtube.com/watch?v=wfYlTtA7-ks https://www.youtube.com/watch?v=wfYlTtA7-ks Where I would like this to go. Is either being able to take scenes from different films and create mashups like this. Or perhaps, getting a whole bunch of extras. Narrating lines and acting in-front of basic sets with green screens. Then putting the faces of recognizable actors and using something like Lyrebird for the voices. Where actors have sold the rights of their faces, voices and personality for cheap. Now you have a $100m movie for the cost of $100k. A similar premise of the film: The Congress. ----- I really think in about 5 years, when the software is there and the dedicated IaaS to train the sets is commonly available. We'll start to see some really cool stuff.
- davidw 9y ago10 years out, who are the 'recognizable actors'? Do they just keep recycling the same ones?
- ProxCoques 9y agoIt's interesting that the issue of ensuring authenticity might also swing into censorship too: if politicians only issued certificates for images of themselves doing good things, they can then dismiss those of them doing bad things as fake because the creator couldn't produce the cert.
- matte_black 9y agoYou cannot allow the politician to be the same person that issues certs because then it creates a market for a politician to buy doctored images and certify them. A neutral unbiased third party must do the certs.
- TaylorAlexander 9y agoInteresting. It seems /r/deepfakes (NSFW) is where the content is at (I assume the article doesn’t link to that, but haven’t checked).
- cobbzilla 9y ago"DNA or it didn't happen"
- carapace 9y agoYou can mail-order custom DNA.
- braindongle 9y agoAn interesting angle here is the arms race between manipulation and forensics. In image forensics, clever people are using clever techniques to keep us tethered to some notion of authenticity in digital media. Like this guy: http://www.cs.dartmouth.edu/farid/downloads/publications/wifs17.pdf http://www.cs.dartmouth.edu/farid/downloads/publications/wif... These emerging video manipulation tools open new frontiers for related forensics research. In 10 years, when we see a video of someone doing something horrible, these people are perhaps our only hope of knowing whether or not what we're seeing ever happened.
- gitgud 9y agoA quote that struck me from previous discussion on the topic: @ekimekim 44 days ago "We've already seen this with images and Photoshop. Society and their heuristics of belief will adjust as these new capabilities become widespread. What's more troubling is that as media becomes falsifiable, solid evidence of...well, anything, becomes hard to have. The ultimate loser there is the truth, sadly."
- tomaskafka 9y agoNon-NSFW sample: https://www.reddit.com/r/deepfakes/comments/7sjkw5/ilm_fixed_that_for_you_edition/ https://www.reddit.com/r/deepfakes/comments/7sjkw5/ilm_fixed... > Top is original footage from Rogue One with a strange CGI Carrie Fisher. Movie budget: $200m > Bottom is a 20 minute fake that could have been done in essentially the same way with a visually similar actress. My budget: $0 and some Fleetwood Mac tunes
- hsod 9y agoCan you explain what's going on here? Specifically, I don't understand this sentence: > Bottom is a 20 minute fake that could have been done in essentially the same way with a visually similar actress.
- bryanbuckley 9y agoAccuracy depends on some sort of match between host face and desired face. In this example, the host face was the cgi face which is obviously already very close to the desired face. If filming from scratch and skipping cgi, you need to pick a good host face.
- JetSpiegel 9y agoAfter over 100 years of having a massive professional industry dedicated to creating fake sequences of images for fun and profit, another decrease in the capital needed to create films doesn't seem such a revolutionary thing. Just like people watched that first film of the train moving towards the camera and freaked out, while today that seems quaint, humans as pattern-matchers extraordinaiers will find a way to discern the fakes.
- paul7986 9y agoAnd this is why we need a verified identity system on the Internet. With such in place you will know who created the video as their verified identity is attached to it. If there's no verified identity attached to it the video wont hold any weight. Same goes for everything done on the iNet .. you can use it anonymously where what you say or do doesn't hold much or any weight vs. commenting,posting, etc using your verified ID does. This just one solution that could help with all the fakery on the iNet and the mayhem it brings and will continue to bring but worse.
- Retra 9y agoDo you mean digital signatures?
- paul7986 9y agoMore like a drivers license or govt issued ID card for Internet usage. YOu can use the Internet anonymously as you always have but if you want to get your point across and ensure the veracity of whatever your posting you'll use your Internet verified ID. Im thinking in the case of someone impersonating a public figure where the originator of the video is not the public figure its immediately labeled fake news. Also if it's revenge porn and you upload it without your identity attached to it .. it's immediately labeled fake.
- Retra 9y agoYou could always just digitally sign everything you put out. You can do that today. I don't see how you're proposing anything new or effective. Besides, what you're proposing is stupid for other reasons. If I have a video of a police officer murdering someone, you're just going to say it's fake news unless that officer himself posts it? That's straight bullshit. "Government Sanctioned Truth" bullshit. The truth about a person or agency is not determined solely by what they approve of.
- paul7986 9y agoGovt mandated Internet identity system that you invoke when you want to ensure the veracity of what you filmed and posted. If not invoked then it's labeled fake and just created for fun or shits n giggles. It holds zero weight and all on the Internet know/understand this system like the back of their hand. Again I am thinking of two use cases re: this solution... revenge porn and celebrity & political figure fake videos.
- jondubois 9y agoThis is great for celebrities... From now on, if they get their phones hacked and get their private videos stolen and shared publicly, they can (plausibly) claim that it's not real.
- grinsekatze 9y ago...and the “nothing” is claiming all of Phantastica.
- Koshkin 9y agoJohn C. Wright, in his book The Golden Age, has worked out a pretty complete picture of the ability to mold one's perception of reality to one's liking.
- dontreact 9y agoBetween VR and the possibility to train up person-specific porn generators, porn is only going to become more and more of a super stimulus that is too hard to resist. Best to try and quit fully now.
- deleted 9y ago[deleted]
- sjg007 9y agoWell definitely black mirror but you can now recreate a passed loved one virtually.