3 ms·

Should be banned on performance. It was obvious with how many times they said “SAFETY” (lobotomized) during their press releases and comments, but I’m still pr

Should be banned on performance. It was obvious with how many times they said “SAFETY” (lobotomized) during their press releases and comments, but I’m still pretty surprised just how bad it is. If you haven’t seen it, the first image is the now-iconic SD3 unofficial picture. https://www.reddit.com/r/AIfreakout/comments/1dekkbz/why_is_sd3_so_bad_at_generating_girls_lying_on/ https://www.reddit.com/r/AIfreakout/comments/1dekkbz/why_is_... Luckily for open source and local running, Lumina and PixArt are catching up.

artninja1988 2y ago
To be fair, SDXL was really bad at launch too. Now people are comparing sd3 base model to fine tuned SDXL outputs... But yeah, I don't have much hope for stability now that the original diffusion team and even comfyanonymous all left. Long live Chinese OS models, I guess
- Auracle 2y ago
  SDXL had bad details but I could absolutely do people in poses other than standing up. While those grass laying images are iconic, Lykon (one of their employees) said on Discord “I don’t know what people could possibly be promoting to get human blobs.” I sent him the first image I did of “photo of a woman getting a massage on a massage table” or something like that and it’s even worse. I went back and checked and while SDXL isn’t great at it it’s not a complete body horror unlike SD3. People have noticed that SD3 is way better at posing Barbies than people, so it’s pretty clear they screwed it up in the name of safety.
spywaregorilla 2y ago
I'm fairly confident anyone thinking this is about safety is just paranoid and misinformed. SD3 medium is a much smaller model than SDXL. I don't really get why they would have published it in the current state, but I'm kind of certain that the full sized versions will be absolutely fine. You don't need pornography in the training set to understand basic human, non-genital anatomy. Small model LLMs are hot garbage too.
- ilaksh 2y ago
  Comfy seemed to be saying they deliberately manipulated the weights to basically sabotage anything remotely erotic.
  - spywaregorilla 2y ago
    Sure. You have to twist SDXL's arm hard to get it to draw a penis too. That does not explain it being really bad. What does explain it being really bad, is the fact that it's just small and bad.
- DrSiemer 2y ago
  The model can effortlessly create a highly realistic and anatomically correct photograph of a koala riding a motorcycle, but begins to produce crazy body horror as soon as a human shape is involved. It's neutered. Artist styles are aggressively scrubbed out as well. It has no idea what a Picasso or Dali style is, even though there must be plenty of derivative work in the trainingdata, that should have taught it what those styles are even with all the original works removed. It can do the Mona Lisa, but the face itself will become comically chunky, almost as if it is censoring itself by consciously turning "forbidden" content into something that is definitely not that.
  - spywaregorilla 2y ago
    I fully believe that artist styles and specific people have been removed from the training data. I fully believe it doesn't know about porn or especially porny positioning. I don't really care. That's reasonably something for community fine tunes. I don't believe for a second that the model can "effortlessly" produce a highly realistic koala. It's just not a good model. It makes a ton of mistakes. I don't really believe that it is meaningfully worse at drawing non-erotic humans than any other similarly. People are feeling personally attacked that the model isn't meeting their standards.
    - DrSiemer 2y ago
      Here's the first five results of a koala riding a motorcycle and the first five of a man kicking a football: https://imgur.com/gallery/sd3-tests-u8iXAcx https://imgur.com/gallery/sd3-tests-u8iXAcx I added the koala kicking a football as well, I have to admit that they come out just as bad. Maybe it's irregular positions in general it just can't do? But from the tests I've done so far mainly humanoid shapes almost always have some issues. The problem is, if certain training data is blanket removed, it creates holes in the understanding a model has. We see this in language models as well: censored models can get very obstinate in their refusal to discuss certain completely normal topics, because it links them to something that has been scrubbed.

artninja1988 2y ago

To be fair, SDXL was really bad at launch too. Now people are comparing sd3 base model to fine tuned SDXL outputs... But yeah, I don't have much hope for stability now that the original diffusion team and even comfyanonymous all left. Long live Chinese OS models, I guess

Auracle 2y ago
SDXL had bad details but I could absolutely do people in poses other than standing up. While those grass laying images are iconic, Lykon (one of their employees) said on Discord “I don’t know what people could possibly be promoting to get human blobs.” I sent him the first image I did of “photo of a woman getting a massage on a massage table” or something like that and it’s even worse. I went back and checked and while SDXL isn’t great at it it’s not a complete body horror unlike SD3. People have noticed that SD3 is way better at posing Barbies than people, so it’s pretty clear they screwed it up in the name of safety.

spywaregorilla 2y ago

I'm fairly confident anyone thinking this is about safety is just paranoid and misinformed. SD3 medium is a much smaller model than SDXL. I don't really get why they would have published it in the current state, but I'm kind of certain that the full sized versions will be absolutely fine. You don't need pornography in the training set to understand basic human, non-genital anatomy. Small model LLMs are hot garbage too.

ilaksh 2y ago
Comfy seemed to be saying they deliberately manipulated the weights to basically sabotage anything remotely erotic.
- spywaregorilla 2y ago
  Sure. You have to twist SDXL's arm hard to get it to draw a penis too. That does not explain it being really bad. What does explain it being really bad, is the fact that it's just small and bad.
DrSiemer 2y ago
The model can effortlessly create a highly realistic and anatomically correct photograph of a koala riding a motorcycle, but begins to produce crazy body horror as soon as a human shape is involved. It's neutered. Artist styles are aggressively scrubbed out as well. It has no idea what a Picasso or Dali style is, even though there must be plenty of derivative work in the trainingdata, that should have taught it what those styles are even with all the original works removed. It can do the Mona Lisa, but the face itself will become comically chunky, almost as if it is censoring itself by consciously turning "forbidden" content into something that is definitely not that.
- spywaregorilla 2y ago
  I fully believe that artist styles and specific people have been removed from the training data. I fully believe it doesn't know about porn or especially porny positioning. I don't really care. That's reasonably something for community fine tunes. I don't believe for a second that the model can "effortlessly" produce a highly realistic koala. It's just not a good model. It makes a ton of mistakes. I don't really believe that it is meaningfully worse at drawing non-erotic humans than any other similarly. People are feeling personally attacked that the model isn't meeting their standards.
  - DrSiemer 2y ago
    Here's the first five results of a koala riding a motorcycle and the first five of a man kicking a football: https://imgur.com/gallery/sd3-tests-u8iXAcx https://imgur.com/gallery/sd3-tests-u8iXAcx I added the koala kicking a football as well, I have to admit that they come out just as bad. Maybe it's irregular positions in general it just can't do? But from the tests I've done so far mainly humanoid shapes almost always have some issues. The problem is, if certain training data is blanket removed, it creates holes in the understanding a model has. We see this in language models as well: censored models can get very obstinate in their refusal to discuss certain completely normal topics, because it links them to something that has been scrubbed.