6 ms·

I could very well be wrong about this, and usually I fall more on the e/acc side, but I think deepfakes are perhaps the gravest immediate concern of AI. Yes, th

by ssalka 2y ago

I could very well be wrong about this, and usually I fall more on the e/acc side, but I think deepfakes are perhaps the gravest immediate concern of AI. Yes, they've been around for well over a decade, but never before has their creation (& thus volume) been so accessible to the average layperson. A simple 10 minutes on CivitAI will tell you all you need to know about how prolific this problem is. IMO it is only a matter of time before this same issue comes to the forefront in the US (I'm kinda surprised it hasn't already become a bigger topic). I could easily see custom celeb LORAs/emmbeddings and realistic SD/SDXL checkpoints being banned outright for this reason. Yes, it would be an infringement on free speech, but weighed against the interests & safety of the public, the courts might see a move like that as reasonable.

transformi 2y ago
Banning Loras/Embedding won't help that..cause you can generate realistic images with only one reference photo these days...
- ssalka 2y ago
  Fair point, technologists will always innovate faster than politicians are able to regulate. Cat's outta the bag now, but that only means the problem will continue to escalate until more drastic measures are deemed necessary. It's that or we will come to accept some level of societal degradation due to the abuse/misuse of AI tools becoming normalized.
deleted 2y ago
[deleted]
slibhb 2y ago
I don't think of myself as on the e/acc side, but I think that deepfakes are here to stay and we'll just have to get used to them. I don't see any way to put the genie back in the bottle but I also don't think it'll take that long to adapt.
- whiplash451 2y ago
  This is giving up a little quickly on watermarking image generation. I do think it’s a hard problem (especially if we want robustness to compression etc) but amenable to research.
m3kw9 2y ago
It won’t be a problem if it gets saturated, people bored of things easily
- ssalka 2y ago
  As AI image generation (and soon video generation) gets to be a more saturated space, we will end up in the state that you cannot trust anything you see online, even if you are educated, well-informed, and meticulously research the topic. It has already been inching that way for a long time, but generative AI definitely is adding new fuel to the fire.
  - ryandrake 2y ago
    I mean, you already kind of cannot trust anything you see online, it's just that so many people still haven't got the message. Maybe everyone needs a deepfake porno of themselves sent to them so they finally grok that everything online should be by default considered fake.
- mingus88 2y ago
  Ah yes, “waiting for the sexual abuser to get bored” sounds like a good tactic and very comforting for young girls
  - m3kw9 2y ago
    A head attached to a fake body as prevalent as a click, nobody cares. 2 years back it would be, but like I said nobody will gaf once every one and their mom can have their heads attached to any body
bobthepanda 2y ago
South Korea has a very bad gender relations problem partially stemming from an extremely lopsided gender ratio. In the 90s it reached as high as 115 male children to 100, and now all those extra young men are angry adults. https://theconversation.com/south-koreas-gender-imbalance-is-bad-news-for-men-outnumbering-women-many-face-bleak-marriage-prospects-217661 https://theconversation.com/south-koreas-gender-imbalance-is...
comboy 2y ago
It's ban photoshop all over again. It's just the new reality. Porn is not even the worst thing. If anything it can make real photos/videos harder to get viral. But augmentation will be (if not already is) used to make very tiny adjustments, like you make one candidate/celebrity face slightly more symmetrical and the other one less, change their gait just a little bit. These changes are non-perceptible, are not interesting enough to make a scandal, and they can have huge impact on how people perceive other people. I mean, I am not suggesting that it isn't a puppet show without those, but the game is evolving.
- gjsman-1000 2y ago
  The above sounds like the opinion of someone who doesn’t have a daughter.
  - giraffe_lady 2y ago
    Even trying to be helpful here you're still just centering men's feelings about "their" women.
    - gjsman-1000 2y ago
      Do you seriously think most women are on board with this? Many do not know it’s happening. Those that do and aren’t chronically online find it horrifying and even traumatizing.
      - giraffe_lady 2y ago
        No believe me I know, I have heard women talking about this coming for years. It was already a very common harassment technique before AI made it free, easy, and accessible. You can tell how few women are involved at the key stages of building new tech because of how rarely obvious harassment vectors are accounted for during development. The problem I have is with trying to get men to care about it because it affects their daughter or whatever.
2OEH8eoCRo0 2y ago
> Yes, it would be an infringement on free speech Freedom of speech is mainly so that we can criticize the government. How does deepfake porn or impersonation have anything to do with speech?
- JumpCrisscross 2y ago
  > Freedom of speech is mainly so that we can criticize the government Freedom of speech is a bigger concept than the First Amendment. Technically curtailing one's ability to make AI porn of your colleagues is a restriction on speech. That said, so is every spam filter, fraud statute and retaliation against a declaration of war. A right doesn't have to be absolute to be good (or vice versa).
  - 2OEH8eoCRo0 2y ago
    All speech is information but not all information is speech
    - JumpCrisscross 2y ago
      > information != speech Information intentionally communicated by a person to another is speech by the strictest classical definition. (I guess technically the creation of the AI porn isn't speech in the way a falling tree in a deserted forest doesn't make a perceived sound. But it's a schoolyard riddle for a reason.)
  - gpm 2y ago
    > spam filter Your other examples, yes. This one, not at all. Freedom of speech is against government intervention, not against private agreements. Rather in fact an absolute form of freedom of speech would protect against the government from stopping you from forwarding all your correspondence to someone else, and would protect against the government stopping that someone else from removing the spam and sending the rest back to you. Non-government run spam filters aren't just not offending freedom of speech, they are protected by it.
pessimizer 2y ago
> I could very well be wrong about this, and usually I fall more on the e/acc side, but I think deepfakes are perhaps the gravest immediate concern of AI. I don't think they're a concern at all, other than that they will ruin the evidentiary value of photographs. They'll do that whether or not the general public has access to them. The concern about fake pictures of people naked is stupid. The same process mentioned above happens to naked pictures of people: they lose evidentiary value. General access to the ability to make deepfakes not only makes it so people won't believe what they see now, but also won't believe actual revenge porn. It simply ceases to be a problem. You can claim it's a fake, and no one can disprove it. This is a reason for celebration. The standard response for that has been for people to simply claim that it won't matter, and that people will destroy the life of some young innocent girl if a mean boy spreads fake pictures of her naked. The answer is that people who harass a young girl are guilty of harassment, and should be jailed. You shouldn't be harassing people even if naked pictures of them are real, which they often are. And if you'd attack a person over pictures that you could fake up yourself in five minutes, you're either a) a moron or b) somebody using the photos as an excuse, and it would be a benefit for society for you to be locked away from it. I suspect a lot of people really worried about deepfake porn deep down are worried about their ability to properly identify and harass the harlots if people won't believe the pictures that you send to their family and friends anymore. From my perspective, they're literally fighting to preserve revenge porn.
- ssalka 2y ago
  > ruin the evidentiary value of photographs This is going to be a monumental issue when the fact that we can't trust any photographs fully takes hold. Maybe watermarking will help with that. More likely people will develop tools to circumvent watermarking technology (either on the imprinting side, or on the verification side). > people won't believe what they see now, but also won't believe actual revenge porn. It simply ceases to be a problem. This may be a comforting way to think about it, but it won't work in all cases. Consider instead of revenge porn, the case of CSAM – that will never be a solved problem until 100% of it is wiped from the face of the earth and it is never produced again. Making it less believable because there is so much AI-generated equivalents out there is a not a solution to the problem, it exacerbates the problem.
deleted 2y ago
[deleted]

by ssalka 2y ago

transformi 2y ago

Banning Loras/Embedding won't help that..cause you can generate realistic images with only one reference photo these days...

ssalka 2y ago
Fair point, technologists will always innovate faster than politicians are able to regulate. Cat's outta the bag now, but that only means the problem will continue to escalate until more drastic measures are deemed necessary. It's that or we will come to accept some level of societal degradation due to the abuse/misuse of AI tools becoming normalized.

deleted 2y ago

[deleted]

slibhb 2y ago

I don't think of myself as on the e/acc side, but I think that deepfakes are here to stay and we'll just have to get used to them. I don't see any way to put the genie back in the bottle but I also don't think it'll take that long to adapt.

whiplash451 2y ago
This is giving up a little quickly on watermarking image generation. I do think it’s a hard problem (especially if we want robustness to compression etc) but amenable to research.

m3kw9 2y ago

It won’t be a problem if it gets saturated, people bored of things easily

ssalka 2y ago
As AI image generation (and soon video generation) gets to be a more saturated space, we will end up in the state that you cannot trust anything you see online, even if you are educated, well-informed, and meticulously research the topic. It has already been inching that way for a long time, but generative AI definitely is adding new fuel to the fire.
- ryandrake 2y ago
  I mean, you already kind of cannot trust anything you see online, it's just that so many people still haven't got the message. Maybe everyone needs a deepfake porno of themselves sent to them so they finally grok that everything online should be by default considered fake.
mingus88 2y ago
Ah yes, “waiting for the sexual abuser to get bored” sounds like a good tactic and very comforting for young girls
- m3kw9 2y ago
  A head attached to a fake body as prevalent as a click, nobody cares. 2 years back it would be, but like I said nobody will gaf once every one and their mom can have their heads attached to any body

bobthepanda 2y ago

South Korea has a very bad gender relations problem partially stemming from an extremely lopsided gender ratio. In the 90s it reached as high as 115 male children to 100, and now all those extra young men are angry adults. https://theconversation.com/south-koreas-gender-imbalance-is-bad-news-for-men-outnumbering-women-many-face-bleak-marriage-prospects-217661 https://theconversation.com/south-koreas-gender-imbalance-is...

comboy 2y ago

It's ban photoshop all over again. It's just the new reality. Porn is not even the worst thing. If anything it can make real photos/videos harder to get viral. But augmentation will be (if not already is) used to make very tiny adjustments, like you make one candidate/celebrity face slightly more symmetrical and the other one less, change their gait just a little bit. These changes are non-perceptible, are not interesting enough to make a scandal, and they can have huge impact on how people perceive other people. I mean, I am not suggesting that it isn't a puppet show without those, but the game is evolving.

gjsman-1000 2y ago
The above sounds like the opinion of someone who doesn’t have a daughter.
- giraffe_lady 2y ago
  Even trying to be helpful here you're still just centering men's feelings about "their" women.
  - gjsman-1000 2y ago
    Do you seriously think most women are on board with this? Many do not know it’s happening. Those that do and aren’t chronically online find it horrifying and even traumatizing.
    - giraffe_lady 2y ago
      No believe me I know, I have heard women talking about this coming for years. It was already a very common harassment technique before AI made it free, easy, and accessible. You can tell how few women are involved at the key stages of building new tech because of how rarely obvious harassment vectors are accounted for during development. The problem I have is with trying to get men to care about it because it affects their daughter or whatever.

2OEH8eoCRo0 2y ago

> Yes, it would be an infringement on free speech Freedom of speech is mainly so that we can criticize the government. How does deepfake porn or impersonation have anything to do with speech?

JumpCrisscross 2y ago
> Freedom of speech is mainly so that we can criticize the government Freedom of speech is a bigger concept than the First Amendment. Technically curtailing one's ability to make AI porn of your colleagues is a restriction on speech. That said, so is every spam filter, fraud statute and retaliation against a declaration of war. A right doesn't have to be absolute to be good (or vice versa).
- 2OEH8eoCRo0 2y ago
  All speech is information but not all information is speech
  - JumpCrisscross 2y ago
    > information != speech Information intentionally communicated by a person to another is speech by the strictest classical definition. (I guess technically the creation of the AI porn isn't speech in the way a falling tree in a deserted forest doesn't make a perceived sound. But it's a schoolyard riddle for a reason.)
- gpm 2y ago
  > spam filter Your other examples, yes. This one, not at all. Freedom of speech is against government intervention, not against private agreements. Rather in fact an absolute form of freedom of speech would protect against the government from stopping you from forwarding all your correspondence to someone else, and would protect against the government stopping that someone else from removing the spam and sending the rest back to you. Non-government run spam filters aren't just not offending freedom of speech, they are protected by it.

pessimizer 2y ago

> I could very well be wrong about this, and usually I fall more on the e/acc side, but I think deepfakes are perhaps the gravest immediate concern of AI. I don't think they're a concern at all, other than that they will ruin the evidentiary value of photographs. They'll do that whether or not the general public has access to them. The concern about fake pictures of people naked is stupid. The same process mentioned above happens to naked pictures of people: they lose evidentiary value. General access to the ability to make deepfakes not only makes it so people won't believe what they see now, but also won't believe actual revenge porn. It simply ceases to be a problem. You can claim it's a fake, and no one can disprove it. This is a reason for celebration. The standard response for that has been for people to simply claim that it won't matter, and that people will destroy the life of some young innocent girl if a mean boy spreads fake pictures of her naked. The answer is that people who harass a young girl are guilty of harassment, and should be jailed. You shouldn't be harassing people even if naked pictures of them are real, which they often are. And if you'd attack a person over pictures that you could fake up yourself in five minutes, you're either a) a moron or b) somebody using the photos as an excuse, and it would be a benefit for society for you to be locked away from it. I suspect a lot of people really worried about deepfake porn deep down are worried about their ability to properly identify and harass the harlots if people won't believe the pictures that you send to their family and friends anymore. From my perspective, they're literally fighting to preserve revenge porn.

ssalka 2y ago
> ruin the evidentiary value of photographs This is going to be a monumental issue when the fact that we can't trust any photographs fully takes hold. Maybe watermarking will help with that. More likely people will develop tools to circumvent watermarking technology (either on the imprinting side, or on the verification side). > people won't believe what they see now, but also won't believe actual revenge porn. It simply ceases to be a problem. This may be a comforting way to think about it, but it won't work in all cases. Consider instead of revenge porn, the case of CSAM – that will never be a solved problem until 100% of it is wiped from the face of the earth and it is never produced again. Making it less believable because there is so much AI-generated equivalents out there is a not a solution to the problem, it exacerbates the problem.

deleted 2y ago

[deleted]