4 ms·

Wait until someone trains a model like this, for porn. There seems to be a post-DALLE obscenity detector on openAI's tool, as so far I've found it to be entire

Wait until someone trains a model like this, for porn. There seems to be a post-DALLE obscenity detector on openAI's tool, as so far I've found it to be entirely robust against deliberate typos designed to avoid simple 'bad word lists'. Ask it for a "pruple violon" and you get purple violins... you get the deal. "Metastable" prompts that may or may not generate obscene (content with nudity, guns, violence as I've found) results sometimes shown non-obscene generations, and sometimes trigger a warning.

jowday 4y ago
If I had to guess, I'd bet they have a supervised classifier trained to recognize bad content (violence, porn, etc) that they use to filter the generated images before passing them to the user, on top of the bad word lists.
- isoprophlex 4y ago
  Exactly!
- cmarschner 4y ago
  Most likely they just take the one from bing. Or, if they trained a better one, it goes vice versa sooner or later
- spread_love 4y ago
  This is mentioned, "content filters" are "blocking images that violate our content policy — which does not allow users to generate violent, adult, or political content, among other categories" and they "limited DALL·E’s exposure to these concepts by removing the most explicit content from its training data."
jug 4y ago
I’ve thought about this and in fact porn generation sounds like a good thing?? It ensures that it’s victimless. Of course, there is a problem with generation of illegal (underage) porn but other than this, I think it could be helpful for this world.
- dx034 4y ago
  If all of the child porn industry switched to generated images they'd still be horrible people but many kids would be saved from having these pictures taken. So a commercial model should certainly ban it, but I don't think it's the biggest thing we have to worry about.
- ffggvv 4y ago
  i wonder if you could say “person who looks 15 but is 18”
zionic 4y ago
Honestly that part pisses me off. Who cares if their AI "makes porn" or something "offensive".
- fishtoaster 4y ago
  I suspect it's more a business restriction than a moral one. If OpenAI allows people to make porn with these tools, people will make a ton of it. OpenAI will become known as "the company that makes the porn-generating AIs," not "the company that keeps pushing the boundaries of AI." Being known as the porn-ai company is bad for business, so they restrict it.
- nsxwolf 4y ago
  The part I really don't understand the purpose of the threat "Further policy violations may lead to an automatic suspension of your account". Why do that? Just refusing to run my query is sufficient. Who is harmed if I continue to bang my head against that wall?
  - frabcus 4y ago
    Because it is a solid line (the policy rule) drawn across a fractal boundary (actual porn), and given lots of attempts you can find somewhere inside the line but across the boundary. Stopping more than so many attempts makes this much harder / much less likely.
alana314 4y ago
I tried the term "cockeyed" and got a TOS violation notice
seydor 4y ago
Porn is cheaper to make, and probably pleasant. But, fantasy porn is not. I can see it sparkling a revival of fantastic erotica

jowday 4y ago

If I had to guess, I'd bet they have a supervised classifier trained to recognize bad content (violence, porn, etc) that they use to filter the generated images before passing them to the user, on top of the bad word lists.

isoprophlex 4y ago
Exactly!
cmarschner 4y ago
Most likely they just take the one from bing. Or, if they trained a better one, it goes vice versa sooner or later
spread_love 4y ago
This is mentioned, "content filters" are "blocking images that violate our content policy — which does not allow users to generate violent, adult, or political content, among other categories" and they "limited DALL·E’s exposure to these concepts by removing the most explicit content from its training data."

jug 4y ago

I’ve thought about this and in fact porn generation sounds like a good thing?? It ensures that it’s victimless. Of course, there is a problem with generation of illegal (underage) porn but other than this, I think it could be helpful for this world.

dx034 4y ago
If all of the child porn industry switched to generated images they'd still be horrible people but many kids would be saved from having these pictures taken. So a commercial model should certainly ban it, but I don't think it's the biggest thing we have to worry about.
ffggvv 4y ago
i wonder if you could say “person who looks 15 but is 18”

zionic 4y ago

Honestly that part pisses me off. Who cares if their AI "makes porn" or something "offensive".

fishtoaster 4y ago
I suspect it's more a business restriction than a moral one. If OpenAI allows people to make porn with these tools, people will make a ton of it. OpenAI will become known as "the company that makes the porn-generating AIs," not "the company that keeps pushing the boundaries of AI." Being known as the porn-ai company is bad for business, so they restrict it.
nsxwolf 4y ago
The part I really don't understand the purpose of the threat "Further policy violations may lead to an automatic suspension of your account". Why do that? Just refusing to run my query is sufficient. Who is harmed if I continue to bang my head against that wall?
- frabcus 4y ago
  Because it is a solid line (the policy rule) drawn across a fractal boundary (actual porn), and given lots of attempts you can find somewhere inside the line but across the boundary. Stopping more than so many attempts makes this much harder / much less likely.

alana314 4y ago

I tried the term "cockeyed" and got a TOS violation notice

seydor 4y ago

Porn is cheaper to make, and probably pleasant. But, fantasy porn is not. I can see it sparkling a revival of fantastic erotica