4 ms·

New Stable Diffusion 3 unable to generate human bodies due to nudity filter

50 points by isaacfrond 2y ago

gnabgib 2y ago
Discussion [0] (20 points, 12 hours ago, 9 comments) [0]: https://news.ycombinator.com/item?id=40661990 https://news.ycombinator.com/item?id=40661990
jrm4 2y ago
This whole situation is pretty fascinating; SD makes the best image generator ecosystem ever (in terms of potential/capability) and now it essentially has to spend most of its time deliberately making it worse because "business/society."
- __loam 2y ago
  They raised 100m, spent it all, and gave away their product for free with no monetization plan. They're also hated by most "creative" workers, are being sued by a myriad of parties, and are under government scrutiny in a number of jurisdictions for enabling political deep fakes and non-consentual pornography. There's a reason why they're trying to save face lol.
  - jrm4 2y ago
    Lol, I didn't know they raised that much! Honestly, I thank them for their service, probably the best 100m ever spent on something like this.
    - __loam 2y ago
      I hope their capital incineration machine is discouraging to others.
      - jrm4 2y ago
        uh oh can't say such sinful things 'round these parts :)
        __loam 2y ago
        Right, if we say too much then maybe titans of industry will write another manifesto about how we're all too mean to them when we point out these businesses have negative impact on society or are completely unsound from a business perspective.
wokwokwok 2y ago
Eh, /r/stablediffusion has a bunch of top level posts showing pretty good output from the model. Maybe if you don’t care about making deepfake porn, it’s actually fine? (…and if for whatever reason you are you can still use all the other models they’ve released that support that and it’s really not a big deal… or wait for a fine tuned version /shrug)
- tuwtuwtuwtuw 2y ago
  Did you try it yourself? For me it falls apart if you try to generate anything containing humans. It also struggles with other animals. If you want to generate an image of yourself, a city, someone sitting in a car, a store or a million other things, humans are often included. Try generating an image of a busy street with no people - it gets a bit dull. Sure, I don't have to use SD3, but SAI explicitly said that this new model would be better att for example generating hands, so of course people are disappointed.
- popalchemist 2y ago
  It's all images of humans. Not just nude ones.
  - Viliam1234 2y ago
    All humans are nude under the clothes. The only safe way is to kill them all. Thus begins the AI apocalypse...
mrandish 2y ago
To the extent SD's "harm reduction" filters are making it less useful, I blame the concern trolls and pearl-clutching, click-pandering media pundits for creating and feeding the current atmosphere. Concern and outrage have become a cottage industry, yet little has really changed here. Making naughty images good enough to deceive has been possible for 20 years. AI just makes it much easier. Of course trolls, spammers and random delinquents are using it to do more of what they've always done. That this would happen was so obviously inevitable to anyone paying attention, I find the current levels of "concern" from supposedly savvy, experienced pundits dubious at best. Personally, I find all the faux drama around this rather uninteresting. I acknowledge my viewpoint is likely unpopular but (IMHO) the level of perceptual heat being generated in mainstream and social media now exceeds the probable real-world net harm from LLMs generating fake images, reinforcing offensive stereotypes or (gasp) confidently spouting wrong answers. It's nothing that Photoshop, TikTok and search engines haven't already been been doing for years. Non-tech people's perceptions are already in the process of adapting to this new normal, while most of us tech-savvy people have already been appropriately skeptical of uncorroborated images and videos for many years. The downside to the perceptual tempest is that fear a few users will transgress some boundary and become tomorrow's click-bait headline du jour is causing researchers and devs to waste a lot of cycles chasing ill-defined (and probably unattainable) harm reduction goalposts that keep moving just out of reach. It's nerfing a potentially useful emerging technology before we've even seen how capable it might actually be. Instead, I'm doomed to interact with some straight-jacketed LLM that's been tortured into maintaining a faux chipper, inoffensive persona that's a cringey parody of a real, authentic and useful assistant.
- dehugger 2y ago
  "AI just makes it much easier." is glossing over a whole lot there. With the right model some shitty teen can generate an album of hundreds of images depicting your daughter in mere minutes, images that are functionally identical to reality. The pearl clutching is warranted.
smusamashah 2y ago
Ideogram is mostly great at generating humans. It mostly generates perfect bodies, hands, feet and other limbs. They have NSFW filters which step in after the image has been generated.

50 points by isaacfrond 2y ago

Discussion [0] (20 points, 12 hours ago, 9 comments) [0]: https://news.ycombinator.com/item?id=40661990 https://news.ycombinator.com/item?id=40661990

jrm4 2y ago

This whole situation is pretty fascinating; SD makes the best image generator ecosystem ever (in terms of potential/capability) and now it essentially has to spend most of its time deliberately making it worse because "business/society."

__loam 2y ago
They raised 100m, spent it all, and gave away their product for free with no monetization plan. They're also hated by most "creative" workers, are being sued by a myriad of parties, and are under government scrutiny in a number of jurisdictions for enabling political deep fakes and non-consentual pornography. There's a reason why they're trying to save face lol.
- jrm4 2y ago
  Lol, I didn't know they raised that much! Honestly, I thank them for their service, probably the best 100m ever spent on something like this.
  - __loam 2y ago
    I hope their capital incineration machine is discouraging to others.
    - jrm4 2y ago
      uh oh can't say such sinful things 'round these parts :)
      - __loam 2y ago
        Right, if we say too much then maybe titans of industry will write another manifesto about how we're all too mean to them when we point out these businesses have negative impact on society or are completely unsound from a business perspective.

wokwokwok 2y ago

Eh, /r/stablediffusion has a bunch of top level posts showing pretty good output from the model. Maybe if you don’t care about making deepfake porn, it’s actually fine? (…and if for whatever reason you are you can still use all the other models they’ve released that support that and it’s really not a big deal… or wait for a fine tuned version /shrug)

tuwtuwtuwtuw 2y ago
Did you try it yourself? For me it falls apart if you try to generate anything containing humans. It also struggles with other animals. If you want to generate an image of yourself, a city, someone sitting in a car, a store or a million other things, humans are often included. Try generating an image of a busy street with no people - it gets a bit dull. Sure, I don't have to use SD3, but SAI explicitly said that this new model would be better att for example generating hands, so of course people are disappointed.
popalchemist 2y ago
It's all images of humans. Not just nude ones.
- Viliam1234 2y ago
  All humans are nude under the clothes. The only safe way is to kill them all. Thus begins the AI apocalypse...

mrandish 2y ago

To the extent SD's "harm reduction" filters are making it less useful, I blame the concern trolls and pearl-clutching, click-pandering media pundits for creating and feeding the current atmosphere. Concern and outrage have become a cottage industry, yet little has really changed here. Making naughty images good enough to deceive has been possible for 20 years. AI just makes it much easier. Of course trolls, spammers and random delinquents are using it to do more of what they've always done. That this would happen was so obviously inevitable to anyone paying attention, I find the current levels of "concern" from supposedly savvy, experienced pundits dubious at best. Personally, I find all the faux drama around this rather uninteresting. I acknowledge my viewpoint is likely unpopular but (IMHO) the level of perceptual heat being generated in mainstream and social media now exceeds the probable real-world net harm from LLMs generating fake images, reinforcing offensive stereotypes or (gasp) confidently spouting wrong answers. It's nothing that Photoshop, TikTok and search engines haven't already been been doing for years. Non-tech people's perceptions are already in the process of adapting to this new normal, while most of us tech-savvy people have already been appropriately skeptical of uncorroborated images and videos for many years. The downside to the perceptual tempest is that fear a few users will transgress some boundary and become tomorrow's click-bait headline du jour is causing researchers and devs to waste a lot of cycles chasing ill-defined (and probably unattainable) harm reduction goalposts that keep moving just out of reach. It's nerfing a potentially useful emerging technology before we've even seen how capable it might actually be. Instead, I'm doomed to interact with some straight-jacketed LLM that's been tortured into maintaining a faux chipper, inoffensive persona that's a cringey parody of a real, authentic and useful assistant.

dehugger 2y ago
"AI just makes it much easier." is glossing over a whole lot there. With the right model some shitty teen can generate an album of hundreds of images depicting your daughter in mere minutes, images that are functionally identical to reality. The pearl clutching is warranted.

smusamashah 2y ago

Ideogram is mostly great at generating humans. It mostly generates perfect bodies, hands, feet and other limbs. They have NSFW filters which step in after the image has been generated.