5 ms·

> Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to: ... >+ Sexual content witho

by tifik 4y ago

> Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to: ... >+ Sexual content without consent of the people who might see it I understand that it's their TOS and they can put pretty much anything in there, but this item seems... odd. I don't really know why exactly this stands out to me. Maybe it's because it's practically un-enforceable? Are they just covering all their bases legally? Trying to think of a good metaphore; let's try this: If you are an artist and someone commissions you to create an art piece that might be sexual, can you say "ok, but you have to ask for consent before you show it to people", and you enshrine it in the contract. Obviously gross violations like trolling by spamming porn are pretty clear cut, but what about the more nuanced cases when you say, display it on your personal website? Are you supposed to have an NSFW overlay? Isn't opening a website sort of implying that you consent to seeing whatever is on there, unless you have a strong preconception of what content the page is expected to display? I might be hugely overthinking this.

netruk44 4y ago
I think the issue they're mainly worried about might be exemplified with a prompt of 'my little pony'. A children's show with quite a lot of adult imagery associated with it on the internet. A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. Did the researchers manage to filter out these images before training? Or is the model aware of both 'kinds' of 'my little pony' images? If the researchers aren't sure they got rid of all of the adult content, then there's really no way to guarantee the model isn't about to ruin some oblivious person's day. So then, do you require people generating images to be intricately familiar with the training dataset? Or do you attempt to prevent any kind of surprise like this by just blocking 'unexpected' interactions like this?
- jimbob45 4y ago
  A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. So everyone has to have gimpy AI just because parents can't be expected to take responsibility for what their child does and does not see? Why the fuck is a child being allowed to play with something that can very easily spit out salacious images accidentally? Wouldn't it be significantly easier to add censorship to the prompt input instead? It seems like these tech companies see yet another opportunity to add censorship to their products and can hardly hide their giddy excitement.
  - RC_ITR 4y ago
    Because in aggregate, children seeing those things has impacts on society. Like sure would it be better if parents monitored their children’s 4chan use? Ofc. Is that at all a practical approach to eliminating Elliot Roger idolization? No.
  - netruk44 4y ago
    Just to be clear, the child was just an example of someone who could theoretically experience 'cruel' treatment from the current version of stable diffusion. I'm absolutely not recommending people let their children use the model unsupervised. It doesn't have to be a parenting problem, though. The same could be said (for example) of a random mother trying to get inspiration for a 'my little pony' birthday cake for their child, and being presented with the 'other' kind of image unintentionally, without their consent. I think they would be justifiably upset in that situation. If we were to imagine someone attempting to put stable diffusion into some future consumer product, I think they would have to be concerned about these kinds of scenarios. Therefore, the scientists are trying to figure out how to accomplish the filtering. FWIW, I don't think a model could be made that actively prevented people from using their own NSFW training data. The only difference in the future will be that the public models won't be able to do it 'for free' with no modifications needed. You'll have to train your own model, or wait for someone else to train one.
    - MayeulC 4y ago
      Interestingly, better results might be achieved by exposing the model to a large corpus of apropriately tagged NSFW data, so that the prompt may exclude it. I imagine img2img could also make an image SFW, or vice-versa. I'd be curious to know what kind of alterations it would make.
  - calebkaiser 4y ago
    I would recommend looking more closely at the article. Stability.ai, the company who developed and released the model being discussed, have not added a safety filter to the model. As the article points out, the filter is specifically implemented by HuggingFace's Diffusers library, which is a popular library for working with diffusion models (but again, to be clear, not the only option for using Stable Diffusion). The library is also open source, and turning off the safety filter would be trivial if you felt compelled to do so. So, "these tech companies" aren't overcome by glee over censoring you. One company implemented one filter in one open source and easily editable library.
formerly_proven 4y ago
Stable Diffusion is developed at LMU Munich and this particular line basically paraphrases § 184 of the German criminal code, which makes it a misdemeanor crime to put porn in places reachable by minors or to show porn to someone without being asked to do so, among other things. I dunno why they felt compelled to include it though. Regarding your examples, most of these are technically criminal in Germany, because the only legally safe way to have a place not-reachable-by-minors means adhering to German youth protection laws, which you're not going to, just like every porn site, Twitter, Reddit etc.
properparity 4y ago
We need to let it completely loose and get everyone exposed to it everywhere so that maybe we can finally get rid of this insane taboo and uptightness about sex and nudity we have in society.
- octagons 4y ago
  Nudity? Yes. Pornography? No.
  - archontes 4y ago
    No rules.
  - rcoveson 4y ago
    Football? Yes. Violence? No. Try getting that rule passed on any form of media.
    - octagons 4y ago
      Sorry, I wasn’t clear. I’m not suggesting any regulation. I’m saying that I agree that “society” (in my case, American culture) has taken the idea of shielding children from viewing pornography to an extreme, where nudity in media, even in a non-sexual context, is often censored. I think this ultimately causes more harm to a society instead of benefitting it. I don’t think this is a very unique viewpoint, but my choice of words in that other comment didn’t communicate this point very well.
- pbhjpbhj 4y ago
  >insane taboo and uptightness about sex and nudity we have in society // In the UK we're on aggregate definitely too uptight about nudity, but sex ... inhibition towards things like infidelity, promiscuity, fecundity, seems like a relatively good thing. Sex being the preserve of committed relationships is not a problem to fix to my view. It sounds like you think we should basically be bonobos? Preoccupied with carnal interactions to the exclusion of all else?
bawolff 4y ago
To me, i think it seems weird because its disconnected from stable diffusion. I think the comparison would be if google maps had a terms of service forbidding using it to plan getaway routes during bank robberies. Like yes bank robberies are wrong, but if someone did that the sin would not be with google maps.
- BoorishBears 4y ago
  That's because maps are boring now. Use Autopilot in a getaway and it's not on Tesla, but rest assured images of a robber hanging out a Tesla handling lane centering would be repeated ad-nauseum as "Tesla aids robbers in getaway!!!!" Giving themselves an easy PR out like "the user broke our ToS in doing <insert bad thing>" is just being forward thinking
krisoft 4y ago
> If you are an artist and someone commissions you to create an art piece that might be sexual, can you say "ok, but you have to ask for consent before you show it to people", and you enshrine it in the contract. Yes. Obviously. How is that a question? > Are you supposed to have an NSFW overlay? Sounds like a reasonable way to comply with the condition. > I might be hugely overthinking this. I agree.
digitallyfree 4y ago
The SD terms also mention that the model and its generated outputs cannot be used for disinformation, medical advice, and several other things. It looks like the only way to legally protect yourself would be to require a contract from everyone buying your SD artwork asserting that they will also comply with the full SD license terms. While this may work if you're selling the art electronically and provide the buyer with a set of terms to accept, this would be difficult if you're selling the work physically. For instance if I sell a postcard with SD art on it in a convenience store, the buyer won't be signing any contracts. However the buyer could display that postcard in a manner that is technically disinformation (e.g. going around telling people the picture on the postcard is a genuine photograph) and suddenly that becomes a license violation.
nonethewiser 4y ago
They dont want it being used for porn. I wouldnt either. But they should just say that.

by tifik 4y ago

netruk44 4y ago

I think the issue they're mainly worried about might be exemplified with a prompt of 'my little pony'. A children's show with quite a lot of adult imagery associated with it on the internet. A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. Did the researchers manage to filter out these images before training? Or is the model aware of both 'kinds' of 'my little pony' images? If the researchers aren't sure they got rid of all of the adult content, then there's really no way to guarantee the model isn't about to ruin some oblivious person's day. So then, do you require people generating images to be intricately familiar with the training dataset? Or do you attempt to prevent any kind of surprise like this by just blocking 'unexpected' interactions like this?

jimbob45 4y ago
A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. So everyone has to have gimpy AI just because parents can't be expected to take responsibility for what their child does and does not see? Why the fuck is a child being allowed to play with something that can very easily spit out salacious images accidentally? Wouldn't it be significantly easier to add censorship to the prompt input instead? It seems like these tech companies see yet another opportunity to add censorship to their products and can hardly hide their giddy excitement.
- RC_ITR 4y ago
  Because in aggregate, children seeing those things has impacts on society. Like sure would it be better if parents monitored their children’s 4chan use? Ofc. Is that at all a practical approach to eliminating Elliot Roger idolization? No.
- netruk44 4y ago
  Just to be clear, the child was just an example of someone who could theoretically experience 'cruel' treatment from the current version of stable diffusion. I'm absolutely not recommending people let their children use the model unsupervised. It doesn't have to be a parenting problem, though. The same could be said (for example) of a random mother trying to get inspiration for a 'my little pony' birthday cake for their child, and being presented with the 'other' kind of image unintentionally, without their consent. I think they would be justifiably upset in that situation. If we were to imagine someone attempting to put stable diffusion into some future consumer product, I think they would have to be concerned about these kinds of scenarios. Therefore, the scientists are trying to figure out how to accomplish the filtering. FWIW, I don't think a model could be made that actively prevented people from using their own NSFW training data. The only difference in the future will be that the public models won't be able to do it 'for free' with no modifications needed. You'll have to train your own model, or wait for someone else to train one.
  - MayeulC 4y ago
    Interestingly, better results might be achieved by exposing the model to a large corpus of apropriately tagged NSFW data, so that the prompt may exclude it. I imagine img2img could also make an image SFW, or vice-versa. I'd be curious to know what kind of alterations it would make.
- calebkaiser 4y ago
  I would recommend looking more closely at the article. Stability.ai, the company who developed and released the model being discussed, have not added a safety filter to the model. As the article points out, the filter is specifically implemented by HuggingFace's Diffusers library, which is a popular library for working with diffusion models (but again, to be clear, not the only option for using Stable Diffusion). The library is also open source, and turning off the safety filter would be trivial if you felt compelled to do so. So, "these tech companies" aren't overcome by glee over censoring you. One company implemented one filter in one open source and easily editable library.

formerly_proven 4y ago

Stable Diffusion is developed at LMU Munich and this particular line basically paraphrases § 184 of the German criminal code, which makes it a misdemeanor crime to put porn in places reachable by minors or to show porn to someone without being asked to do so, among other things. I dunno why they felt compelled to include it though. Regarding your examples, most of these are technically criminal in Germany, because the only legally safe way to have a place not-reachable-by-minors means adhering to German youth protection laws, which you're not going to, just like every porn site, Twitter, Reddit etc.

properparity 4y ago

We need to let it completely loose and get everyone exposed to it everywhere so that maybe we can finally get rid of this insane taboo and uptightness about sex and nudity we have in society.

octagons 4y ago
Nudity? Yes. Pornography? No.
- archontes 4y ago
  No rules.
- rcoveson 4y ago
  Football? Yes. Violence? No. Try getting that rule passed on any form of media.
  - octagons 4y ago
    Sorry, I wasn’t clear. I’m not suggesting any regulation. I’m saying that I agree that “society” (in my case, American culture) has taken the idea of shielding children from viewing pornography to an extreme, where nudity in media, even in a non-sexual context, is often censored. I think this ultimately causes more harm to a society instead of benefitting it. I don’t think this is a very unique viewpoint, but my choice of words in that other comment didn’t communicate this point very well.
pbhjpbhj 4y ago
>insane taboo and uptightness about sex and nudity we have in society // In the UK we're on aggregate definitely too uptight about nudity, but sex ... inhibition towards things like infidelity, promiscuity, fecundity, seems like a relatively good thing. Sex being the preserve of committed relationships is not a problem to fix to my view. It sounds like you think we should basically be bonobos? Preoccupied with carnal interactions to the exclusion of all else?

bawolff 4y ago

To me, i think it seems weird because its disconnected from stable diffusion. I think the comparison would be if google maps had a terms of service forbidding using it to plan getaway routes during bank robberies. Like yes bank robberies are wrong, but if someone did that the sin would not be with google maps.

BoorishBears 4y ago
That's because maps are boring now. Use Autopilot in a getaway and it's not on Tesla, but rest assured images of a robber hanging out a Tesla handling lane centering would be repeated ad-nauseum as "Tesla aids robbers in getaway!!!!" Giving themselves an easy PR out like "the user broke our ToS in doing <insert bad thing>" is just being forward thinking

krisoft 4y ago

> If you are an artist and someone commissions you to create an art piece that might be sexual, can you say "ok, but you have to ask for consent before you show it to people", and you enshrine it in the contract. Yes. Obviously. How is that a question? > Are you supposed to have an NSFW overlay? Sounds like a reasonable way to comply with the condition. > I might be hugely overthinking this. I agree.

digitallyfree 4y ago

The SD terms also mention that the model and its generated outputs cannot be used for disinformation, medical advice, and several other things. It looks like the only way to legally protect yourself would be to require a contract from everyone buying your SD artwork asserting that they will also comply with the full SD license terms. While this may work if you're selling the art electronically and provide the buyer with a set of terms to accept, this would be difficult if you're selling the work physically. For instance if I sell a postcard with SD art on it in a convenience store, the buyer won't be signing any contracts. However the buyer could display that postcard in a manner that is technically disinformation (e.g. going around telling people the picture on the postcard is a genuine photograph) and suddenly that becomes a license violation.

nonethewiser 4y ago

They dont want it being used for porn. I wouldnt either. But they should just say that.