4 ms·

I struggle to see the harm in deep fake pornogra9hy as long as people realize its fake. Am I missing something?

drsopp 3y ago
I guess the problem is when people don't realize it's fake. There is a lag between people who are familiar with this technology and those who are not. Edit: I worked in school over a decade, and I can definitely see the huge potential for harm here with digital bullying.
- lynx23 3y ago
  The problem with bullying is the bullying itself, and not the specific way in which it is performed. It does not matter if you are being bullied old-fashioned analog, or hypermodern digital. The issue is that we basically have never thought of a reliable way to protect children from other children, in the age category where it is necessary. It is one of those semi-taboo topics where we actually prefer to look away and hope a solution will solidify on its own. Because we dont want to accept that children can be cruel assholes. At least not our children! Maybe the spoiled brat from the neighborhood, yeah, that child is... But not mine! My child is sweet honey, the best human ever conceived...
- Xenoamorphous 3y ago
  I also see the opposite happening: real footage of someone doing something nasty, and claiming it's a deepfake. If we get to the point where deepfakes are indistinguishable from the real thing, does it mean footage can no longer be used in court?
  - Cthulhu_ 3y ago
    That'll be an interesting one to see play out. Thing is though, forgeries / fake evidence in the form of image and video editing isn't anything new - how is that challenged in court these days? I can imagine they have an expert witness testify that it's fake or not, but they can and have been wrong as well.
  - matheusmoreira 3y ago
    > does it mean footage can no longer be used in court? Perhaps we'll see the rise of cryptographically verified videos: the camera signs the output of their sensors, proving it was captured from the real world.
    - didntcheck 3y ago
      That's been suggested periodically for many years, but I'm not sure I'd ever trust it. I think people would always find a way to spoof the inputs, even if it's just by the analogue hole of focusing on a screen. This would be non-trivial, but both private and state espionage groups would be willing to pour a lot of money into the prospect of being able to have a "undeniably real" doctored image
Madmallard 3y ago
It's already good enough to not be really obviously fake.
- idle_zealot 3y ago
  I don't think that's what concord meant. They meant that as convincing fake imagery becomes easier to produce and more common, people will learn that any image could easily be faked, regardless of how photorealistic it appears. i.e. people will lose all trust in "photographs", so fake images will lose their efficacy as blackmail.
  - Cthulhu_ 3y ago
    I mean this isn't new, photographs can be doctored and have been for... as long as photos have been around pretty much. It doesn't matter if they're fake or not for the purposes of blackmail though.
    - idle_zealot 3y ago
      Currently, with skill and effort, a photo can be faked. People know that a given photo may be fake, but generally expect photos to be real unless they are motivated to think otherwise. The idea is that once the ease of producing a fake reaches a certain threshold (a layman can go to a website and type "$PERSON engaging in $QUESTIONABLE_ACTIVITY") then fake photos will greatly outnumber real ones and trust in photographs will fully erode.
  - exodust 3y ago
    Exactly. The "victim" of a deepfake simply needs to say "it's fake". The burden of integrity is then on whoever is indulging the fake images, spreading them or raising questions about them. Added benefit, as you say, is that any image can now be faked. So even a real photo of someone caught in the act, could be dismissed by the victim as fake if they wanted to make the problem go away.
rawrawrawrr 3y ago
I think the opposite - once deep fake porn becomes good enough, blackmailing/revenge porn will no longer be a thing. Anyone can just say that those images are fake, and porn of celebrities, exes, politicians will all be worthless.
- vmfunction 3y ago
  Looking forward to that, when digital images/video no longer means anything to people. Hopefully then people will lean to ONLY trust and value real in person interactions... Until of course we invents holodeck ;-)
testtestabcdef 3y ago
Just a few more months then people will stop talking about it, as usual.
7speter 3y ago
It has the power to deceive and leave a lasting impression on those who may see it in passing and not realize its fake. Also, it can be demeaning considering the faked subjects would have no say consent-wise in the distribution of the faked content. Think of what happened with Hulk Hogan, who had a sex tape released without his permission. Though it was real, it left a lasting impression on a given audience, and it tarnished his reputation (further), and he successfully sued because, iirc, the video was “leaked” without his consent.
OliveMate 3y ago
It's pretty abhorrent to practically digitally unclothe someone (unless they fully consent to it), and more times than not the content produced by this will be shared online to countless other people. The thing is, you cannot guarantee that everyone will know it's fake, and when you weaponise it against people who can't distinguish it there'll be consequences. Currently we see people found to be doing Adult Content online fired from their jobs, but what if a technically-blind boss gets sent a ton of deepfakes of an employee and is falsely told they do content like that on the side? What if a doubting partner suddenly receives deepfakes of their partner with someone else? What if your political rival starts spreading deepfakes with wildly rabid claims about yourself which the media picks up and reports on – bullshit has spread around the world by the time the truth has tied its shoelaces. Think about the absolute inane content people believe on Facebook. Think about the misinformation that gets spread over Twitter just because the fake information would be funny. Think about the stuff you've seen online which turned out not to be so real. Even if the images generated aren't real, the subject of them can't convince the world they're not, and there's no way for them to stop the violation of their self-image.
- concordDance 3y ago
  > It's pretty abhorrent to practically digitally unclothe someone (unless they fully consent to it), I'm having a hard time wrapping my head around this. Is the same true of doing so mentally? My post specifically posits that the images are known to be fake and asks if THAT has any harm. Obviously framing people for things is a harm.
ehnto 3y ago
Defamation, identity theft, abuse of individuals directly by circulating porn of them in their communities. Fake or not, it's very impactful. It's not an ideological issue but a very practical one, it's already causing harm. You might be thinking of celebrities where the idea that they have done porn is so far fetched that it's obvious it would be fake. But what about minor personalities like youtubers? It could tank a career if it's unclear something is real or not. Just like a false assault allegation, it will still cause a lot if harm even if the truth prevails. Lastly, on an individual level, people are going to get hurt by sociopaths circling imagery of coworkers or whatever, it's messy human social business and the pragmatic person might imagine the truth is all that matters, but it definitely isn't. You can ruin someone's social life and mental health with mere rumours, let alone what will look like proof of ireputable activity.
Version467 3y ago
Don't you think it feels incredibly degrading and violating to have your face stitched into porn in a way that's supposed to be as realistic as possible? Without your consent? Doesn't really matter if the video is real, the people who this is done to have not signed up for this. And you know people are jerking off to it. I don't doubt for a second that it's absolutely repulsive to see yourself in a deepfake porn. A couple of popular streamers who have had that happened to them have reported feeling similar feelings of shame and violation to the way they felt when they experienced actual sexual assault. That's pretty alarming and should count for something, no? Also, this gives a completely new dimension to parasocial relationships. There are lots of people struggling with real realtionships that find solace in live streams. As they continue to engage in the community, they can begin to feel like they have an actual relationship with those streamers, which is obviously dangerous since it's such a one-sided thing. Deepfake porn elevates this even further and makes it that much more likely for people to develop an unhealthy obsession with people that they will likely never meet, instead of working on their real interpersonal relationships.
- chii 3y ago
  On the other hand, if said deep fake was never publicly revealed, and is only utilized by the person who made it (and not distributed), then is there harm done? The deep fake subject is completely ignorant of it in this scenario. In other words, is the harm not in the production, but the wide distribution of the material, and the harm is in the reputation of the subject (even if explicitly labeled fake), and not in the actual contents? And so this begs the question - if an AI could deep fake something on demand, without ever distributing it except to the requester, then is there any harm done?
  - Cthulhu_ 3y ago
    Well, consider revenge porn. A couple shares images or makes videos of themselves, never publicly revealed, only utilized by the person / people who made it, no harm done right? But then the relationship breaks up and one of the partners publishes the footage. Or their account or device get compromised and the material goes public. There's a series of new laws with regards to revenge porn, and deepfake porn will likely fall under that category. It's not about whether it stays private, it's the risk of it going public. I mean this issue isn't anything new either, it's pretty straightforward and common to edit a celebrity's face onto a nude model.
Cthulhu_ 3y ago
People won't realize it's fake... and people won't care either. You struggle to see the harm because you haven't been a victim, and I presume you're not a woman either. So yeah, sorry to get personal but you're missing something.
- exodust 3y ago
  Why does it matter if people don't immediately realize it's fake? Why are those people looking at pornographic images? For any harm to happen, the status of the image as real needs to be confirmed and sustained over time. If the victim says "it's fake", why would people doubt that or not care? That would just mean those who give oxygen to the fake images, are themselves in danger of spreading false information and losing all credibility. Only trolls and trashy people bypass concern for their own credibility. As for "consent", who cares? It's a dirty fake image made by losers, and is the modern equivalent of words written on a public bathroom wall: "For a good time, call Jo". Jo never consented to that, but who cares, it's already content from the gutter.
DrJokepu 3y ago
Quite apart from any moral considerations that other commenters have addressed, in cold business terms such deep fakes directly compete with the works of these models and therefore it’s not in their interest to permit their circulation.
- Der_Einzige 3y ago
  Sorry, your only options are to not have pictures of yourself on the internet, or be seen in public. This of course is pretty much impossible for all but most introverted or hermits.
  - matheusmoreira 3y ago
    Who knows? Maybe society will change for the better once this technology becomes widespread. Maybe people will finally stop publishing their lives for the world to see.
- __loam 3y ago
  You've basically explained why none of these ai image models are fair use.

I guess the problem is when people don't realize it's fake. There is a lag between people who are familiar with this technology and those who are not. Edit: I worked in school over a decade, and I can definitely see the huge potential for harm here with digital bullying.

lynx23 3y ago
The problem with bullying is the bullying itself, and not the specific way in which it is performed. It does not matter if you are being bullied old-fashioned analog, or hypermodern digital. The issue is that we basically have never thought of a reliable way to protect children from other children, in the age category where it is necessary. It is one of those semi-taboo topics where we actually prefer to look away and hope a solution will solidify on its own. Because we dont want to accept that children can be cruel assholes. At least not our children! Maybe the spoiled brat from the neighborhood, yeah, that child is... But not mine! My child is sweet honey, the best human ever conceived...
Xenoamorphous 3y ago
I also see the opposite happening: real footage of someone doing something nasty, and claiming it's a deepfake. If we get to the point where deepfakes are indistinguishable from the real thing, does it mean footage can no longer be used in court?
- Cthulhu_ 3y ago
  That'll be an interesting one to see play out. Thing is though, forgeries / fake evidence in the form of image and video editing isn't anything new - how is that challenged in court these days? I can imagine they have an expert witness testify that it's fake or not, but they can and have been wrong as well.
- matheusmoreira 3y ago
  > does it mean footage can no longer be used in court? Perhaps we'll see the rise of cryptographically verified videos: the camera signs the output of their sensors, proving it was captured from the real world.
  - didntcheck 3y ago
    That's been suggested periodically for many years, but I'm not sure I'd ever trust it. I think people would always find a way to spoof the inputs, even if it's just by the analogue hole of focusing on a screen. This would be non-trivial, but both private and state espionage groups would be willing to pour a lot of money into the prospect of being able to have a "undeniably real" doctored image

Madmallard 3y ago

It's already good enough to not be really obviously fake.

idle_zealot 3y ago
I don't think that's what concord meant. They meant that as convincing fake imagery becomes easier to produce and more common, people will learn that any image could easily be faked, regardless of how photorealistic it appears. i.e. people will lose all trust in "photographs", so fake images will lose their efficacy as blackmail.
- Cthulhu_ 3y ago
  I mean this isn't new, photographs can be doctored and have been for... as long as photos have been around pretty much. It doesn't matter if they're fake or not for the purposes of blackmail though.
  - idle_zealot 3y ago
    Currently, with skill and effort, a photo can be faked. People know that a given photo may be fake, but generally expect photos to be real unless they are motivated to think otherwise. The idea is that once the ease of producing a fake reaches a certain threshold (a layman can go to a website and type "$PERSON engaging in $QUESTIONABLE_ACTIVITY") then fake photos will greatly outnumber real ones and trust in photographs will fully erode.
- exodust 3y ago
  Exactly. The "victim" of a deepfake simply needs to say "it's fake". The burden of integrity is then on whoever is indulging the fake images, spreading them or raising questions about them. Added benefit, as you say, is that any image can now be faked. So even a real photo of someone caught in the act, could be dismissed by the victim as fake if they wanted to make the problem go away.

rawrawrawrr 3y ago

I think the opposite - once deep fake porn becomes good enough, blackmailing/revenge porn will no longer be a thing. Anyone can just say that those images are fake, and porn of celebrities, exes, politicians will all be worthless.

vmfunction 3y ago
Looking forward to that, when digital images/video no longer means anything to people. Hopefully then people will lean to ONLY trust and value real in person interactions... Until of course we invents holodeck ;-)

testtestabcdef 3y ago

Just a few more months then people will stop talking about it, as usual.

7speter 3y ago

It has the power to deceive and leave a lasting impression on those who may see it in passing and not realize its fake. Also, it can be demeaning considering the faked subjects would have no say consent-wise in the distribution of the faked content. Think of what happened with Hulk Hogan, who had a sex tape released without his permission. Though it was real, it left a lasting impression on a given audience, and it tarnished his reputation (further), and he successfully sued because, iirc, the video was “leaked” without his consent.

OliveMate 3y ago

It's pretty abhorrent to practically digitally unclothe someone (unless they fully consent to it), and more times than not the content produced by this will be shared online to countless other people. The thing is, you cannot guarantee that everyone will know it's fake, and when you weaponise it against people who can't distinguish it there'll be consequences. Currently we see people found to be doing Adult Content online fired from their jobs, but what if a technically-blind boss gets sent a ton of deepfakes of an employee and is falsely told they do content like that on the side? What if a doubting partner suddenly receives deepfakes of their partner with someone else? What if your political rival starts spreading deepfakes with wildly rabid claims about yourself which the media picks up and reports on – bullshit has spread around the world by the time the truth has tied its shoelaces. Think about the absolute inane content people believe on Facebook. Think about the misinformation that gets spread over Twitter just because the fake information would be funny. Think about the stuff you've seen online which turned out not to be so real. Even if the images generated aren't real, the subject of them can't convince the world they're not, and there's no way for them to stop the violation of their self-image.

concordDance 3y ago
> It's pretty abhorrent to practically digitally unclothe someone (unless they fully consent to it), I'm having a hard time wrapping my head around this. Is the same true of doing so mentally? My post specifically posits that the images are known to be fake and asks if THAT has any harm. Obviously framing people for things is a harm.

ehnto 3y ago

Defamation, identity theft, abuse of individuals directly by circulating porn of them in their communities. Fake or not, it's very impactful. It's not an ideological issue but a very practical one, it's already causing harm. You might be thinking of celebrities where the idea that they have done porn is so far fetched that it's obvious it would be fake. But what about minor personalities like youtubers? It could tank a career if it's unclear something is real or not. Just like a false assault allegation, it will still cause a lot if harm even if the truth prevails. Lastly, on an individual level, people are going to get hurt by sociopaths circling imagery of coworkers or whatever, it's messy human social business and the pragmatic person might imagine the truth is all that matters, but it definitely isn't. You can ruin someone's social life and mental health with mere rumours, let alone what will look like proof of ireputable activity.

Version467 3y ago

Don't you think it feels incredibly degrading and violating to have your face stitched into porn in a way that's supposed to be as realistic as possible? Without your consent? Doesn't really matter if the video is real, the people who this is done to have not signed up for this. And you know people are jerking off to it. I don't doubt for a second that it's absolutely repulsive to see yourself in a deepfake porn. A couple of popular streamers who have had that happened to them have reported feeling similar feelings of shame and violation to the way they felt when they experienced actual sexual assault. That's pretty alarming and should count for something, no? Also, this gives a completely new dimension to parasocial relationships. There are lots of people struggling with real realtionships that find solace in live streams. As they continue to engage in the community, they can begin to feel like they have an actual relationship with those streamers, which is obviously dangerous since it's such a one-sided thing. Deepfake porn elevates this even further and makes it that much more likely for people to develop an unhealthy obsession with people that they will likely never meet, instead of working on their real interpersonal relationships.

chii 3y ago
On the other hand, if said deep fake was never publicly revealed, and is only utilized by the person who made it (and not distributed), then is there harm done? The deep fake subject is completely ignorant of it in this scenario. In other words, is the harm not in the production, but the wide distribution of the material, and the harm is in the reputation of the subject (even if explicitly labeled fake), and not in the actual contents? And so this begs the question - if an AI could deep fake something on demand, without ever distributing it except to the requester, then is there any harm done?
- Cthulhu_ 3y ago
  Well, consider revenge porn. A couple shares images or makes videos of themselves, never publicly revealed, only utilized by the person / people who made it, no harm done right? But then the relationship breaks up and one of the partners publishes the footage. Or their account or device get compromised and the material goes public. There's a series of new laws with regards to revenge porn, and deepfake porn will likely fall under that category. It's not about whether it stays private, it's the risk of it going public. I mean this issue isn't anything new either, it's pretty straightforward and common to edit a celebrity's face onto a nude model.

Cthulhu_ 3y ago

People won't realize it's fake... and people won't care either. You struggle to see the harm because you haven't been a victim, and I presume you're not a woman either. So yeah, sorry to get personal but you're missing something.

exodust 3y ago
Why does it matter if people don't immediately realize it's fake? Why are those people looking at pornographic images? For any harm to happen, the status of the image as real needs to be confirmed and sustained over time. If the victim says "it's fake", why would people doubt that or not care? That would just mean those who give oxygen to the fake images, are themselves in danger of spreading false information and losing all credibility. Only trolls and trashy people bypass concern for their own credibility. As for "consent", who cares? It's a dirty fake image made by losers, and is the modern equivalent of words written on a public bathroom wall: "For a good time, call Jo". Jo never consented to that, but who cares, it's already content from the gutter.

DrJokepu 3y ago

Quite apart from any moral considerations that other commenters have addressed, in cold business terms such deep fakes directly compete with the works of these models and therefore it’s not in their interest to permit their circulation.

Der_Einzige 3y ago
Sorry, your only options are to not have pictures of yourself on the internet, or be seen in public. This of course is pretty much impossible for all but most introverted or hermits.
- matheusmoreira 3y ago
  Who knows? Maybe society will change for the better once this technology becomes widespread. Maybe people will finally stop publishing their lives for the world to see.
__loam 3y ago
You've basically explained why none of these ai image models are fair use.