4 ms·

It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as so

by lrvick 3y ago

It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. There are potentially a couple of positive social outcomes that may emerge from this trend though. 1. everyone who sees a deepfake of themselves will be forever viscerally skeptical of any claim relying only on photos and videos socially shared online. 2. everyone who sees actual nude photos or videos of people they know will be skeptical they are real, thus taking away the power of revenge porn or ransom porn.

ToucanLoucan 3y ago
> It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. This is just casually dropped as though a photo-realistic AI image == an amateur painting, in production quality, in required effort and time, as though they're even remotely comparable. Which by the by, I also think the amateur painting should be punished as making nude pictures of people you don't get permission from is creepy. > It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. I do not wish to live in a free society then. This is patently absurd. We also cannot stop people murdering each other, stealing from each other, or otherwise causing harm, but we damn well punish it when it happens (when our useless judiciary system can be bothered anyway but that's a different conversation) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying. So horrifying, on so many levels. Is this what we're building? Is this the society we all envisioned as kiddos getting into the Internet and our computers? I think we can do so very, very much better than this and it makes me incredibly depressed seeing what we're permitting to be done with the tools we make. Everyone involved in this should be ashamed. I don't know how I'd live with myself knowing I played even a small role in ushering in entire new vectors of sexual harassment directed at people who already struggle to get a modicum of justice in this world. And for what? "Freedom?" My right to swing my fist ends when it contacts your face, but up until then, you can't do a thing? You just get to have dozens if not hundreds of people swinging at you faking you out every minute of your life and you just have to sit there and take it, because they technically haven't hit you yet? Bollocks. Absolute nonsense. This society might be "free" but it cannot function and the longer we delay in gaining this understanding the more people will be driven out of their minds waiting for us to do so.
- lrvick 3y ago
  Stopping someone from physically harming someone else can come with severe -punishment- as a deterrent and is much more serious than sharing -any- type of data online. Stopping someone from making a digital parody of someone else gets closer to free speech territory and is a lot more complicated. You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings, just like you can outlaw pictures of religious imagery because they hurt feelings, but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy. Giving up the right to online anonymity is a non starter as it would make it impossible for journalists and dissidents to communicate freely which is a requirement for democracy to function.
  - ToucanLoucan 3y ago
    > Stopping someone from physically harming someone else can come with severe -punishment- as a deterrent and is much more serious than sharing -any- type of data online. Stopping someone from making a digital parody of someone else gets closer to free speech territory and is a lot more complicated. This statement is hilarious because we cannot even do the former for schoolchildren, let alone the latter. Between casual bullying resulting in fatalities due to the utter disinterest of the administration at play, to school shootings where police cower outside while someone murders children en masse. Apparently being a child now is just grounds to have people make pornography of you, bully you relentlessly, with the distinct possibility you might die in a hail of gunfire one day for literally no reason. No wonder the birth rate has cratered. > You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings CAN and SHOULD. INCREDIBLY illegal. Like registering the creator as a sex offender illegal, because that's what this is. It's a sex crime. > but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy. So we just give up?
    - lrvick 3y ago
      Unless you have any ideas for a solution, I suggest we focus on education to contextualize what kids will inevitably see online. Kids are going to see over 18 content be it deepfake or hardcore. Censorship approaches do not work.
civilitty 3y ago
[flagged]
- hmcq6 3y ago
  It's a fun thought experiment but in a society where a guy who sleeps with 100 women is "lucky" and a woman who sleeps with 100 guys is a slut it is not enough to just turn the accusations/revenge porn back on the men. Not to mention... you're suggesting that we weaponize homophobia. Funny to think about but in no way is this a realistic solution
- mock-possum 3y ago
  Kinda gross that you immediately made this a guys versus girls thing And then immediately jumped to “lol you’re gay” as something the girls should use against the guys
jazzyjackson 3y ago
> It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. terminally uncreative take. I'm actually curious what you mean by free society tho because clearly we are not totally without rules and taboos, what are we free to do and where do we draw the line? imo deepfakes are not that different from revenge porn, it is an image of you being shared without your consent. doesn't matter that it's 90% synthesized, your image was used in a way designed to hurt you or at least your reputation. we have laws about libel and slander, where's the incompatibility?
- giantg2 3y ago
  There's a difference between between actually stopping it and laws. Sure, the law can be there but it's going to be hard to enforce and we need to see how the case law pans out. What can you actually do to truly stop it? Ban deep fake software? How's that ban going to work out? To what degree is libel and slander present in current schools and how often is it successfully addressed?
- lrvick 3y ago
  The incompatibility lies with it being unenforceable at scale even if illegal. Religious blasphemy is not legal in many countries. Sharing pirated content is not legal in many countries, but we all are on the same internet. Trying to lock up millions of people anonymously sharing data globally that is not even universally illegal is an absurd enforcement problem. At some point enforcement becomes impossible without killing access to a free anonymous internet entirely.
  - grondo4 3y ago
    Right but we're able to relatively effectively scrub child porn from the Internet right? What makes deepfake porn different?
    - lrvick 3y ago
      Child porn is a couple clicks away for anyone with internet access. Any teen boy that wants to see pictures of similarly aged girls, finds them quickly. Remember that such pictures, no matter how gross, are legal in many countries who share the internet with us. I am not saying that is okay, but I am saying it is impossible to stop without shutting down the internet. If anything deepfake tech reduces the demand to abuse real humans to obtain some types of pictures, which might reduce harm, even if in gross way. Also you can find copyright movies too just as easily no matter how many billions have been spent trying to stop it.
ChainOfFools 3y ago
It would not surprise me to find that in 30 years or so a new category of harm is codified into law which amounts to official and social condemnation of behavior that abuses the freedom of a free society. Or to put another way, behavior that pathologically elevates personal or individual freedom ahead of the net freedom of the society that forms an indiv9dual's supporting environment. Particularly such behavior that in order to reduce the immediate harms it causes, would require reducing the freedom of society as a whole. A simple example, rather less sensational or revolting than pseudo-nude photos of schoolchildren but far more pervasive and destructive, and similarly immune to attack under today's laws and circumstances, would be noise pollution ordinances, which are seldom enforced but frequently abused. Such abuses degrade the environment in innumerable, but sub-actionable ways for hundreds of people so that one person can have something they want, ie straight pipes on a harley/whatever being gunned through residential streets at 2:00 a.m. in the morning. I feel like this class of abusive behavior is on the rise (at least, in Spain as well as my country) and will cause a much more catastrophic breakdown of a free society if it remains unchecked, but that we have no means of curbing under our current frameworks without compromising those frameworks themselves. At the same time I'm really tired of this posture of conceding to assholes on the grounds that tjey effectively hold "free society" hostage as a shield for their individual need for validation and/or financial gain.
- __turbobrew__ 3y ago
  For me personally I think it is less that this behaviour is on the rise, but that I’m getting older and it bothers me more. After hearing some stories about what it was like during my grandparents lifetime there were no bylaw officers or noise ordinances or pollution control, so I’m not convinced that we have it worse than a few generations ago.
  - palata 3y ago
    > so I’m not convinced that we have it worse than a few generations ago. If it's about comparing generations, it's quite likely that your grandparents (or at least their parents) won't be here to suffer the consequences of modern society (I mean climate change and biodiversity loss), but you will. So yeah, I'm not convinced that we have it better than a few generations ago.
palata 3y ago
> but the types of measures it would take to stop it are incompatible with a free society. Before going into enforcement, I would have talked about ethics :-). It feels like it is not too much to ask of people who can create new technology that they think about the consequences of their work.
- ChainOfFools 3y ago
  yep. I'm getting really weary of these shallow protests about protecting anonymity and "technology" above all else, that are grounded in some sort of vague idea that technology is beyond moral reproach or has a mind of its own, and isn't in fact just an extension of ordinary, and in the case of this article, mean-spirited and cruelty-motivated human intent. These deflecting arguments amount to no more than a modern irreligious version of "the s/devil/technology made us do it"
  - lrvick 3y ago
    You seem to imply we should have centralized government mandated control over all technology that allows online communication. Countries have tried this experiment like China and North Korea, and it seems to only serve those in power, not the public. Let us say we did ban all open source decentralized technology in the US, and all websites that to not follow government rules, and mandate all digital communication in the US happen via government approved apps and devices. How long do you think it will take before a US president with authoritarian ambitions decides to take advantage of this single point of failure? Free anonymous and private communication is mandatory for a functional democracy. It has some bad side effects, but the side effects of not having it are most assuredly worse.
    - palata 3y ago
      > Free anonymous and private communication is mandatory for a functional democracy. Out of curiosity: can you mention one democracy where people generally have "free anonymous and private communication"?
throw__away7391 3y ago
In a way this isn’t actually a new thing. Throughout most of human history there was nothing like a photograph and it was always possible to fake paintings and other depictions of anything. It’s only this very narrow window of recent history when photos have existed and been broadly considered “proof” of reality while being difficult to forge. This time now appears to be coming to an end.
- darkerside 3y ago
  I'm reminded of the aliens on Galaxy Quest who had no conception of lying, and were all but unable to identify or understand it. Rationally, we may know that pictures and videos are not to be trusted, but we will never stop getting taken in, like an optical illusion.

by lrvick 3y ago

ToucanLoucan 3y ago

> It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. This is just casually dropped as though a photo-realistic AI image == an amateur painting, in production quality, in required effort and time, as though they're even remotely comparable. Which by the by, I also think the amateur painting should be punished as making nude pictures of people you don't get permission from is creepy. > It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. I do not wish to live in a free society then. This is patently absurd. We also cannot stop people murdering each other, stealing from each other, or otherwise causing harm, but we damn well punish it when it happens (when our useless judiciary system can be bothered anyway but that's a different conversation) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying. So horrifying, on so many levels. Is this what we're building? Is this the society we all envisioned as kiddos getting into the Internet and our computers? I think we can do so very, very much better than this and it makes me incredibly depressed seeing what we're permitting to be done with the tools we make. Everyone involved in this should be ashamed. I don't know how I'd live with myself knowing I played even a small role in ushering in entire new vectors of sexual harassment directed at people who already struggle to get a modicum of justice in this world. And for what? "Freedom?" My right to swing my fist ends when it contacts your face, but up until then, you can't do a thing? You just get to have dozens if not hundreds of people swinging at you faking you out every minute of your life and you just have to sit there and take it, because they technically haven't hit you yet? Bollocks. Absolute nonsense. This society might be "free" but it cannot function and the longer we delay in gaining this understanding the more people will be driven out of their minds waiting for us to do so.

lrvick 3y ago
Stopping someone from physically harming someone else can come with severe -punishment- as a deterrent and is much more serious than sharing -any- type of data online. Stopping someone from making a digital parody of someone else gets closer to free speech territory and is a lot more complicated. You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings, just like you can outlaw pictures of religious imagery because they hurt feelings, but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy. Giving up the right to online anonymity is a non starter as it would make it impossible for journalists and dissidents to communicate freely which is a requirement for democracy to function.
- ToucanLoucan 3y ago
  > Stopping someone from physically harming someone else can come with severe -punishment- as a deterrent and is much more serious than sharing -any- type of data online. Stopping someone from making a digital parody of someone else gets closer to free speech territory and is a lot more complicated. This statement is hilarious because we cannot even do the former for schoolchildren, let alone the latter. Between casual bullying resulting in fatalities due to the utter disinterest of the administration at play, to school shootings where police cower outside while someone murders children en masse. Apparently being a child now is just grounds to have people make pornography of you, bully you relentlessly, with the distinct possibility you might die in a hail of gunfire one day for literally no reason. No wonder the birth rate has cratered. > You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings CAN and SHOULD. INCREDIBLY illegal. Like registering the creator as a sex offender illegal, because that's what this is. It's a sex crime. > but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy. So we just give up?
  - lrvick 3y ago
    Unless you have any ideas for a solution, I suggest we focus on education to contextualize what kids will inevitably see online. Kids are going to see over 18 content be it deepfake or hardcore. Censorship approaches do not work.

civilitty 3y ago

[flagged]

hmcq6 3y ago
It's a fun thought experiment but in a society where a guy who sleeps with 100 women is "lucky" and a woman who sleeps with 100 guys is a slut it is not enough to just turn the accusations/revenge porn back on the men. Not to mention... you're suggesting that we weaponize homophobia. Funny to think about but in no way is this a realistic solution
mock-possum 3y ago
Kinda gross that you immediately made this a guys versus girls thing And then immediately jumped to “lol you’re gay” as something the girls should use against the guys

jazzyjackson 3y ago

> It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. terminally uncreative take. I'm actually curious what you mean by free society tho because clearly we are not totally without rules and taboos, what are we free to do and where do we draw the line? imo deepfakes are not that different from revenge porn, it is an image of you being shared without your consent. doesn't matter that it's 90% synthesized, your image was used in a way designed to hurt you or at least your reputation. we have laws about libel and slander, where's the incompatibility?

giantg2 3y ago
There's a difference between between actually stopping it and laws. Sure, the law can be there but it's going to be hard to enforce and we need to see how the case law pans out. What can you actually do to truly stop it? Ban deep fake software? How's that ban going to work out? To what degree is libel and slander present in current schools and how often is it successfully addressed?
lrvick 3y ago
The incompatibility lies with it being unenforceable at scale even if illegal. Religious blasphemy is not legal in many countries. Sharing pirated content is not legal in many countries, but we all are on the same internet. Trying to lock up millions of people anonymously sharing data globally that is not even universally illegal is an absurd enforcement problem. At some point enforcement becomes impossible without killing access to a free anonymous internet entirely.
- grondo4 3y ago
  Right but we're able to relatively effectively scrub child porn from the Internet right? What makes deepfake porn different?
  - lrvick 3y ago
    Child porn is a couple clicks away for anyone with internet access. Any teen boy that wants to see pictures of similarly aged girls, finds them quickly. Remember that such pictures, no matter how gross, are legal in many countries who share the internet with us. I am not saying that is okay, but I am saying it is impossible to stop without shutting down the internet. If anything deepfake tech reduces the demand to abuse real humans to obtain some types of pictures, which might reduce harm, even if in gross way. Also you can find copyright movies too just as easily no matter how many billions have been spent trying to stop it.

ChainOfFools 3y ago

It would not surprise me to find that in 30 years or so a new category of harm is codified into law which amounts to official and social condemnation of behavior that abuses the freedom of a free society. Or to put another way, behavior that pathologically elevates personal or individual freedom ahead of the net freedom of the society that forms an indiv9dual's supporting environment. Particularly such behavior that in order to reduce the immediate harms it causes, would require reducing the freedom of society as a whole. A simple example, rather less sensational or revolting than pseudo-nude photos of schoolchildren but far more pervasive and destructive, and similarly immune to attack under today's laws and circumstances, would be noise pollution ordinances, which are seldom enforced but frequently abused. Such abuses degrade the environment in innumerable, but sub-actionable ways for hundreds of people so that one person can have something they want, ie straight pipes on a harley/whatever being gunned through residential streets at 2:00 a.m. in the morning. I feel like this class of abusive behavior is on the rise (at least, in Spain as well as my country) and will cause a much more catastrophic breakdown of a free society if it remains unchecked, but that we have no means of curbing under our current frameworks without compromising those frameworks themselves. At the same time I'm really tired of this posture of conceding to assholes on the grounds that tjey effectively hold "free society" hostage as a shield for their individual need for validation and/or financial gain.

__turbobrew__ 3y ago
For me personally I think it is less that this behaviour is on the rise, but that I’m getting older and it bothers me more. After hearing some stories about what it was like during my grandparents lifetime there were no bylaw officers or noise ordinances or pollution control, so I’m not convinced that we have it worse than a few generations ago.
- palata 3y ago
  > so I’m not convinced that we have it worse than a few generations ago. If it's about comparing generations, it's quite likely that your grandparents (or at least their parents) won't be here to suffer the consequences of modern society (I mean climate change and biodiversity loss), but you will. So yeah, I'm not convinced that we have it better than a few generations ago.

palata 3y ago

> but the types of measures it would take to stop it are incompatible with a free society. Before going into enforcement, I would have talked about ethics :-). It feels like it is not too much to ask of people who can create new technology that they think about the consequences of their work.

ChainOfFools 3y ago
yep. I'm getting really weary of these shallow protests about protecting anonymity and "technology" above all else, that are grounded in some sort of vague idea that technology is beyond moral reproach or has a mind of its own, and isn't in fact just an extension of ordinary, and in the case of this article, mean-spirited and cruelty-motivated human intent. These deflecting arguments amount to no more than a modern irreligious version of "the s/devil/technology made us do it"
- lrvick 3y ago
  You seem to imply we should have centralized government mandated control over all technology that allows online communication. Countries have tried this experiment like China and North Korea, and it seems to only serve those in power, not the public. Let us say we did ban all open source decentralized technology in the US, and all websites that to not follow government rules, and mandate all digital communication in the US happen via government approved apps and devices. How long do you think it will take before a US president with authoritarian ambitions decides to take advantage of this single point of failure? Free anonymous and private communication is mandatory for a functional democracy. It has some bad side effects, but the side effects of not having it are most assuredly worse.
  - palata 3y ago
    > Free anonymous and private communication is mandatory for a functional democracy. Out of curiosity: can you mention one democracy where people generally have "free anonymous and private communication"?

throw__away7391 3y ago

In a way this isn’t actually a new thing. Throughout most of human history there was nothing like a photograph and it was always possible to fake paintings and other depictions of anything. It’s only this very narrow window of recent history when photos have existed and been broadly considered “proof” of reality while being difficult to forge. This time now appears to be coming to an end.

darkerside 3y ago
I'm reminded of the aliens on Galaxy Quest who had no conception of lying, and were all but unable to identify or understand it. Rationally, we may know that pictures and videos are not to be trusted, but we will never stop getting taken in, like an optical illusion.