4 ms·

We can likely make the very strong assumption that the training data didn't contain any CSAM so it would be more difficult for the model to produce CSAM. Also,

We can likely make the very strong assumption that the training data didn't contain any CSAM so it would be more difficult for the model to produce CSAM. Also, I would imagine they trained the model without porn too, so inferring CSAM based on legal adult porn would also be quite difficult. Am I missing something?

zarzavat 4y ago
These models are intended to converge to the capabilities of human artists and beyond. A human artist is obviously capable of generating CSAM, even if they have never seen that before. Filtering of training data is countered by increasing capabilities to generalize: Two years ago, that was a viable strategy: models could barely produce what was in the training data again. Today models can generalize much better and compose concepts they have been trained on into new concepts that they haven’t. Two years from now, filtering will be irrelevant.
- d3nj4l 4y ago
  Not only that, but with techniques like in-painting you could start with something that wasn't CP and then progressively make the model generate parts of it which then make up such an image right now. Stability saying they want to release an open version of SD that can't make CP is like a pen maker selling a pen that can't make CP: horseshit.
rngname22 4y ago
Actually I saw Stable Diffusion-generated semi-stylized / semi-photorealistic (kind of like photorealistic-ish anime) CSAM on 4chan literally a day or so ago when I randomly decided to go to 4chan and saw that AI art threads are super popular there right now. Keep in mind, there already are a lot of illustrated/anime style pictures of CSAM on that site of years though (something that is legal in many countries), so it's sort of becoming a blurred area as these AI art generators are still somewhat like that but now are getting to be more photorealistic. As far as the models not being trained on NSFW content, there was already leaked models that were, and there are unofficial models trained by outsiders using SD that are specifically trained on for example adult image websites.

zarzavat 4y ago

These models are intended to converge to the capabilities of human artists and beyond. A human artist is obviously capable of generating CSAM, even if they have never seen that before. Filtering of training data is countered by increasing capabilities to generalize: Two years ago, that was a viable strategy: models could barely produce what was in the training data again. Today models can generalize much better and compose concepts they have been trained on into new concepts that they haven’t. Two years from now, filtering will be irrelevant.

d3nj4l 4y ago
Not only that, but with techniques like in-painting you could start with something that wasn't CP and then progressively make the model generate parts of it which then make up such an image right now. Stability saying they want to release an open version of SD that can't make CP is like a pen maker selling a pen that can't make CP: horseshit.

rngname22 4y ago

Actually I saw Stable Diffusion-generated semi-stylized / semi-photorealistic (kind of like photorealistic-ish anime) CSAM on 4chan literally a day or so ago when I randomly decided to go to 4chan and saw that AI art threads are super popular there right now. Keep in mind, there already are a lot of illustrated/anime style pictures of CSAM on that site of years though (something that is legal in many countries), so it's sort of becoming a blurred area as these AI art generators are still somewhat like that but now are getting to be more photorealistic. As far as the models not being trained on NSFW content, there was already leaked models that were, and there are unofficial models trained by outsiders using SD that are specifically trained on for example adult image websites.