7 ms·

The reason I saw mentioned is that NSFW content was so prevalent in their training set that, if they did not explicitly exclude it, it was actually difficult fo

by saynay 4y ago

The reason I saw mentioned is that NSFW content was so prevalent in their training set that, if they did not explicitly exclude it, it was actually difficult for it to NOT produce NSFW results even from normal inputs. For example, if you ask for anything with a woman in it, it would probably give you a naked woman in the results.

parhamn 4y ago
This exact example made me wonder how the 'bias' situation will ever be worked around (besides training the model to an individuals preexisting biases). What should the picture of a 'woman' on wikipedia be? Should they be clothed? In western clothing? What skin color? Age? Surely everyone imagines something slightly different when they think 'woman'.
- LudwigNagasena 4y ago
  Yeah, for starters one should at least try to provide a coherent operational definition of “bias”.
- ben_w 4y ago
  At the time of writing, it's this: https://commons.m.wikimedia.org/wiki/File:Woman_at_Lover%27s_Bridge_Tanjung_Sepat_(cropped).jpg https://commons.m.wikimedia.org/wiki/File:Woman_at_Lover%27s... Which is close to what I would've guessed by picking a high population density area, and a likely location and candidate for a decent portait photo on a wildly popular wiki. Still not sure why the top was edited from red to grey.
- mgdlbp 4y ago
  Perfect example: English Wikipedia has bikeshedded lead images of this kind ad nauseum over these two decades, with many times more text written in discussions of this than in the articles themselves. The current image for [[woman]] was chosen last year in an RfC that ended with a rare counted vote.¹ This prompted a re-debate of [[human]]² that trailed off in dissastisfaction toward the only established guideline in the area, which forbids montages. That, in turn was last affirmed by a series of debates in 2015/2016.³ ¹ https://en.wikipedia.org/wiki/Talk:Woman/Archive_19 https://en.wikipedia.org/wiki/Talk:Woman/Archive_19 ² https://en.wikipedia.org/wiki/Talk:Human/Archive_35#Argument_made_in_the_FAQ https://en.wikipedia.org/wiki/Talk:Human/Archive_35#Argument... ³ https://en.wikipedia.org/wiki/MOS:NOETHNICGALLERIES https://en.wikipedia.org/wiki/MOS:NOETHNICGALLERIES
andai 4y ago
Well if you ask for just a woman you're going to get just the woman!
yreg 4y ago
No, if you ask SD without a filter for a woman, it won't probably give you a naked woman. It is actually rather rare, but I don't see what's so horrible when it does happen. When you go to a gallery you will see a lot of naked people. It's not a big deal.
- lbotos 4y ago
  Sure, and there is a spectrum of - naked because that's the default human - naked and posed in a way that is artistically beautiful - naked and posed in a way that is sexually provocative SD is convinced past a certain threshold of flesh-colored space that it could fall into the sexually proactive group so they err on the side of caution. I'm not mad at them for defaulting to that. If you want to push past the default filter it's easy enough to remove. Seems sensible to me when the training set does include a lot of literal pornography.
  - yreg 4y ago
    I have yet to see an innocent prompt that results in a naked person posed in a sexually provocative way, but if someone discovered one, please post it along with a seed. I don't fault SD for having a simply removable filter. I was ranting about the applications built on top of stable diffusion that force the filter to be engaged. (But of course everyone is free to create the software they wish, I'm not entitled to complain it should work differently.)
- TigeriusKirk 4y ago
  I use SD with no filter and I wouldn't say it's rare. I do think some words trigger it more often than others. For example, I've seen a prompt with "beautiful" turn up half the results with nudity, whereas removing the word I got no nudity.
  - yreg 4y ago
    `Beautiful` is already pushing the model toward sexualizing her, I mean the subject is going to be beautiful by default since the model was trained on LAION Aesthetics. But yes, `a beautiful woman` is a fairly innocent prompt. My experience (also using SD without the filter all the time) is that the model very rarely produces sexual results unless intentionally pushed that way. Perhaps our mileages vary.
napier 4y ago
The LAION-5B dataset was curated by researchers at Heidelberg University, and while it was filtered for nsfw content, the human form is less of an issue in much of Europe compared to views and attitudes in the US.
numpad0 4y ago
"Person" is enough and SD knows that. No need to put gender as foundation of identity.

by saynay 4y ago

parhamn 4y ago

This exact example made me wonder how the 'bias' situation will ever be worked around (besides training the model to an individuals preexisting biases). What should the picture of a 'woman' on wikipedia be? Should they be clothed? In western clothing? What skin color? Age? Surely everyone imagines something slightly different when they think 'woman'.

LudwigNagasena 4y ago
Yeah, for starters one should at least try to provide a coherent operational definition of “bias”.
ben_w 4y ago
At the time of writing, it's this: https://commons.m.wikimedia.org/wiki/File:Woman_at_Lover%27s_Bridge_Tanjung_Sepat_(cropped).jpg https://commons.m.wikimedia.org/wiki/File:Woman_at_Lover%27s... Which is close to what I would've guessed by picking a high population density area, and a likely location and candidate for a decent portait photo on a wildly popular wiki. Still not sure why the top was edited from red to grey.
mgdlbp 4y ago
Perfect example: English Wikipedia has bikeshedded lead images of this kind ad nauseum over these two decades, with many times more text written in discussions of this than in the articles themselves. The current image for [[woman]] was chosen last year in an RfC that ended with a rare counted vote.¹ This prompted a re-debate of [[human]]² that trailed off in dissastisfaction toward the only established guideline in the area, which forbids montages. That, in turn was last affirmed by a series of debates in 2015/2016.³ ¹ https://en.wikipedia.org/wiki/Talk:Woman/Archive_19 https://en.wikipedia.org/wiki/Talk:Woman/Archive_19 ² https://en.wikipedia.org/wiki/Talk:Human/Archive_35#Argument_made_in_the_FAQ https://en.wikipedia.org/wiki/Talk:Human/Archive_35#Argument... ³ https://en.wikipedia.org/wiki/MOS:NOETHNICGALLERIES https://en.wikipedia.org/wiki/MOS:NOETHNICGALLERIES

andai 4y ago

Well if you ask for just a woman you're going to get just the woman!

yreg 4y ago

No, if you ask SD without a filter for a woman, it won't probably give you a naked woman. It is actually rather rare, but I don't see what's so horrible when it does happen. When you go to a gallery you will see a lot of naked people. It's not a big deal.

lbotos 4y ago
Sure, and there is a spectrum of - naked because that's the default human - naked and posed in a way that is artistically beautiful - naked and posed in a way that is sexually provocative SD is convinced past a certain threshold of flesh-colored space that it could fall into the sexually proactive group so they err on the side of caution. I'm not mad at them for defaulting to that. If you want to push past the default filter it's easy enough to remove. Seems sensible to me when the training set does include a lot of literal pornography.
- yreg 4y ago
  I have yet to see an innocent prompt that results in a naked person posed in a sexually provocative way, but if someone discovered one, please post it along with a seed. I don't fault SD for having a simply removable filter. I was ranting about the applications built on top of stable diffusion that force the filter to be engaged. (But of course everyone is free to create the software they wish, I'm not entitled to complain it should work differently.)
TigeriusKirk 4y ago
I use SD with no filter and I wouldn't say it's rare. I do think some words trigger it more often than others. For example, I've seen a prompt with "beautiful" turn up half the results with nudity, whereas removing the word I got no nudity.
- yreg 4y ago
  `Beautiful` is already pushing the model toward sexualizing her, I mean the subject is going to be beautiful by default since the model was trained on LAION Aesthetics. But yes, `a beautiful woman` is a fairly innocent prompt. My experience (also using SD without the filter all the time) is that the model very rarely produces sexual results unless intentionally pushed that way. Perhaps our mileages vary.

napier 4y ago

The LAION-5B dataset was curated by researchers at Heidelberg University, and while it was filtered for nsfw content, the human form is less of an issue in much of Europe compared to views and attitudes in the US.

numpad0 4y ago

"Person" is enough and SD knows that. No need to put gender as foundation of identity.