9 ms·

Thank you, I think this is the root of everything: https://www.reddit.com/r/StableDiffusion/comments/y9ga5s/comment/it5kkic/ https://www.reddit.com/r/StableDiff

by mzs 4y ago

Thank you, I think this is the root of everything: https://www.reddit.com/r/StableDiffusion/comments/y9ga5s/comment/it5kkic/ https://www.reddit.com/r/StableDiffusion/comments/y9ga5s/com... >We want to crush any chance of CP. If folks use it for that entire generative AI space will go radioactive and yes there are some things that can be done to make it much much harder for folks to abuse and we are working with THORN and others right now to make it a reality.

camdenlock 4y ago
This is utterly ridiculous. Even if the generated content is repulsive, it’s still generated; no one is being harmed. Stop this babysitting nonsense.
- trap_goes_hot 4y ago
  Its not really black/white. Deepfakes can be traumatizing to the victims.
jimbob45 4y ago
Isn’t the dream to use this stuff to generate virtual CSAM? Could you not kill the CSAM market overnight by flooding it with AI-generated material?
- krainboltgreene 4y ago
  - jimbob45 4y ago
    I'm advocating for a system in which less CSAM is made IRL...I don't understand how that could possibly be controversial.
    - m4x 4y ago
      I agree that if all CSAM was virtual and no IRL abuse occurred anymore, that would be a vast improvement, despite the continued existence of CSAM. But I suspect many abusers aren't just in it for the images. They want to abuse real people. And in that case, even if all images are virtual, they still feed into real abuses in real life.
    - yanderekko 4y ago
      People don't want to seriously grapple with these sorts of harm reduction arguments. They see sick people getting off on horrific things and want that stopped and the MSM will be more than eager to parade out a string of "why is [company X] allowing this to happen?" articles until the company becomes radioactive to investors. It's a new form of deplatforming - just as people have made careers out of trying to get speech/expression that they dislike removed from the internet, now we're going to see AI companies cripple their own models to ensure that they can't be used to produce speech/expression that are disfavored, out of fear of the reputational consequences of not doing so.
      - krainboltgreene 4y ago
        > People don't want to seriously grapple with these sorts of harm reduction arguments Because there's no evidence it works and the idea makes no fucking sense. It approaches the problem in a way that all experts agree is wrong.
        Huh1337 4y ago
        To ban something you need evidence that it's causing some harm, not vice versa.
        miracle2k 4y ago
        > Because there's no evidence it works and the idea makes no fucking sense. It approaches the problem in a way that all experts agree is wrong. Experts in what exactly? There are two ways to defend a law that penalizes virtual child pornography: - On evidence that there is harm. - On general moral terms, aka "we just don't like that this is happening". Worth noting that a ban on generated CSAM images was struck down as unconstitutional in Ashcroft_v._Free_Speech_Coalition. https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalition https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit...
- fear-anger-hate 4y ago
  It is an interesting ethical question that needs more research done in to it. Though given how people's brains tend to shut down upon hearing the topic, I feel like the general public would be vastly opposed to it even if it was proven to lead to better societal outcomes (decreased child abuse).
- seydor 4y ago
  there is a history to that. In some US states it became illegal to have cgi images of CP , Secondlife had that problem and they still ban it. I think it got turned down on free speech grounds but there are still some kind of restrictions
  - jimbob45 4y ago
    Those restrictions made sense in a world without Stable Diffusion because CGI images were thought to stimulate interest in photorealistic CSAM and photorealistic CSAM couldn't be acquired without outright acquiring actual CSAM. Now that we can readily generate photorealistic CSAM, there's little to no risk of inadvertently creating an customer base for actual CSAM.
    - seydor 4y ago
      They are never going to accept that argument. I mean, some SD applications like the interior designer market this as a great tool for potential buyers to try out ideas before they buy.
- Genghis_9000 4y ago
zarzavat 4y ago
The only way to do that is to not release the model, not let people run the model locally, and basically turn it into DALL-E with a web UI and rented compute. Which defeats the entire point of Stable Diffusion.
- RamblingCTO 4y ago
  Which centralizes control of new tech and prevents innovation imho. Would be a shitty development.
- visarga 4y ago
  You can use AI in two steps, one for the general image and the second one to change the age, with access only to the crop of the head. There was a novel by Asimov where someone did a crime by composing the actions of two robots, who were doing perfectly legal things each one separately.
hda2 4y ago
>> We want to crush any chance of CP. If folks use it for that entire generative AI space will go radioactive and yes there are some things that can be done to make it much much harder for folks to abuse and we are working with THORN and others right now to make it a reality. I'm absolutely certain Linux has been used to kill children. Detergent, pesticide, and even pillows too! Tools shouldn't be limited based on what the worst way they can be used. Stable diffusion is an absolute positive for society. Even when it's used to generate CP, every image that model creates is not one that involves a real kid. The cat is out of the bag when OpenAI announced DALLE. Stable Diffusion only accelerated it a bit. Even if Sablity or lawmakers manage to prevent or outlaw open models, criminals will continue to build their own and ignore laws. The only thing their reluctance does is harm Stability and give their competition a chance to catch up. Perhaps that's a good thing. Maybe it's time for another organization to take the lead.
- geysersam 4y ago
  This is what happens when people/legislators choose to not see further than their own noses. Many products benefit pedophiles in one way or another. Mobile phones, computers, video editing software, cars (vans?). "But that's ridiculous, of course we can't prohibit cars just because they can be used by criminals." Exactly.
- d110af5ccf 4y ago
  > criminals will continue to build their own and ignore laws More specifically, researchers and hobbyists would be made criminals in this hypothetical. Why would I stop playing with ML just because someone passed a useless and poorly thought out law?
faeriechangling 4y ago
You can make child pornography with a can of paint and your hand. What a totally laughable farce and excuse. People are making (notice the tense) REAMS of pornography with AI just as they have with literally every technology that has ever been invented to produce creative works.
RamblingCTO 4y ago
I just had the (very unreflected) thought that generated CP is better than sharing actual victim's pictures? Might be too short-sighted, didn't think about it that hard. But (preventing CP by using) this tech will not prevent CP nor sexual assault of minors. I get the dilemma as a creator though. I wouldn't want my products to be used that way too.
- boppo1 4y ago
  If the results don't have enough artifacts to identidy it as ai generated, that could give cover to collections of images where actual minors were involved. Not great.
  - RamblingCTO 4y ago
    That's a good point! To be fair, there's just no good solution here imho.
plutonorm 4y ago
I'm very interested in this tech and have been working with it. I have heard tell of a CSAM model. I wont say anymore because I don't want to feed that hideous fire. I would add though - if Moore's law continues this will be almost unstoppable in a decade or too.

by mzs 4y ago

camdenlock 4y ago

This is utterly ridiculous. Even if the generated content is repulsive, it’s still generated; no one is being harmed. Stop this babysitting nonsense.

trap_goes_hot 4y ago
Its not really black/white. Deepfakes can be traumatizing to the victims.

jimbob45 4y ago

Isn’t the dream to use this stuff to generate virtual CSAM? Could you not kill the CSAM market overnight by flooding it with AI-generated material?

krainboltgreene 4y ago
- jimbob45 4y ago
  I'm advocating for a system in which less CSAM is made IRL...I don't understand how that could possibly be controversial.
  - m4x 4y ago
    I agree that if all CSAM was virtual and no IRL abuse occurred anymore, that would be a vast improvement, despite the continued existence of CSAM. But I suspect many abusers aren't just in it for the images. They want to abuse real people. And in that case, even if all images are virtual, they still feed into real abuses in real life.
  - yanderekko 4y ago
    People don't want to seriously grapple with these sorts of harm reduction arguments. They see sick people getting off on horrific things and want that stopped and the MSM will be more than eager to parade out a string of "why is [company X] allowing this to happen?" articles until the company becomes radioactive to investors. It's a new form of deplatforming - just as people have made careers out of trying to get speech/expression that they dislike removed from the internet, now we're going to see AI companies cripple their own models to ensure that they can't be used to produce speech/expression that are disfavored, out of fear of the reputational consequences of not doing so.
    - krainboltgreene 4y ago
      > People don't want to seriously grapple with these sorts of harm reduction arguments Because there's no evidence it works and the idea makes no fucking sense. It approaches the problem in a way that all experts agree is wrong.
      - Huh1337 4y ago
        To ban something you need evidence that it's causing some harm, not vice versa.
      - miracle2k 4y ago
        > Because there's no evidence it works and the idea makes no fucking sense. It approaches the problem in a way that all experts agree is wrong. Experts in what exactly? There are two ways to defend a law that penalizes virtual child pornography: - On evidence that there is harm. - On general moral terms, aka "we just don't like that this is happening". Worth noting that a ban on generated CSAM images was struck down as unconstitutional in Ashcroft_v._Free_Speech_Coalition. https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalition https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit...
fear-anger-hate 4y ago
It is an interesting ethical question that needs more research done in to it. Though given how people's brains tend to shut down upon hearing the topic, I feel like the general public would be vastly opposed to it even if it was proven to lead to better societal outcomes (decreased child abuse).
seydor 4y ago
there is a history to that. In some US states it became illegal to have cgi images of CP , Secondlife had that problem and they still ban it. I think it got turned down on free speech grounds but there are still some kind of restrictions
- jimbob45 4y ago
  Those restrictions made sense in a world without Stable Diffusion because CGI images were thought to stimulate interest in photorealistic CSAM and photorealistic CSAM couldn't be acquired without outright acquiring actual CSAM. Now that we can readily generate photorealistic CSAM, there's little to no risk of inadvertently creating an customer base for actual CSAM.
  - seydor 4y ago
    They are never going to accept that argument. I mean, some SD applications like the interior designer market this as a great tool for potential buyers to try out ideas before they buy.
Genghis_9000 4y ago

zarzavat 4y ago

The only way to do that is to not release the model, not let people run the model locally, and basically turn it into DALL-E with a web UI and rented compute. Which defeats the entire point of Stable Diffusion.

RamblingCTO 4y ago
Which centralizes control of new tech and prevents innovation imho. Would be a shitty development.
visarga 4y ago
You can use AI in two steps, one for the general image and the second one to change the age, with access only to the crop of the head. There was a novel by Asimov where someone did a crime by composing the actions of two robots, who were doing perfectly legal things each one separately.

hda2 4y ago

>> We want to crush any chance of CP. If folks use it for that entire generative AI space will go radioactive and yes there are some things that can be done to make it much much harder for folks to abuse and we are working with THORN and others right now to make it a reality. I'm absolutely certain Linux has been used to kill children. Detergent, pesticide, and even pillows too! Tools shouldn't be limited based on what the worst way they can be used. Stable diffusion is an absolute positive for society. Even when it's used to generate CP, every image that model creates is not one that involves a real kid. The cat is out of the bag when OpenAI announced DALLE. Stable Diffusion only accelerated it a bit. Even if Sablity or lawmakers manage to prevent or outlaw open models, criminals will continue to build their own and ignore laws. The only thing their reluctance does is harm Stability and give their competition a chance to catch up. Perhaps that's a good thing. Maybe it's time for another organization to take the lead.

geysersam 4y ago
This is what happens when people/legislators choose to not see further than their own noses. Many products benefit pedophiles in one way or another. Mobile phones, computers, video editing software, cars (vans?). "But that's ridiculous, of course we can't prohibit cars just because they can be used by criminals." Exactly.
d110af5ccf 4y ago
> criminals will continue to build their own and ignore laws More specifically, researchers and hobbyists would be made criminals in this hypothetical. Why would I stop playing with ML just because someone passed a useless and poorly thought out law?

faeriechangling 4y ago

You can make child pornography with a can of paint and your hand. What a totally laughable farce and excuse. People are making (notice the tense) REAMS of pornography with AI just as they have with literally every technology that has ever been invented to produce creative works.

RamblingCTO 4y ago

I just had the (very unreflected) thought that generated CP is better than sharing actual victim's pictures? Might be too short-sighted, didn't think about it that hard. But (preventing CP by using) this tech will not prevent CP nor sexual assault of minors. I get the dilemma as a creator though. I wouldn't want my products to be used that way too.

boppo1 4y ago
If the results don't have enough artifacts to identidy it as ai generated, that could give cover to collections of images where actual minors were involved. Not great.
- RamblingCTO 4y ago
  That's a good point! To be fair, there's just no good solution here imho.

plutonorm 4y ago

I'm very interested in this tech and have been working with it. I have heard tell of a CSAM model. I wont say anymore because I don't want to feed that hideous fire. I would add though - if Moore's law continues this will be almost unstoppable in a decade or too.