2 ms·

I always assumed that image generation services had prompt and output filtering. The reason I believe this is that I've crafted prompts that seemed to pass the

I always assumed that image generation services had prompt and output filtering. The reason I believe this is that I've crafted prompts that seemed to pass the prompt filter, generated, and then were unable to be displayed[1]. Presumably there is an output filter that determines if the generation violates ToS against things like nudity. Though in my case, I wasn't able to generate images that too closely resembled Ansel Adams photographs. Shame on me, I guess. 1. This reads a little like being able to deduce what's going on by studying the timing, now that I reflect on it.

Footnote7341 3y ago
Yes bing image creator uses two different levels of filters, one for the prompt and one for the output. gpt image creator does the same thing, as does the dalle-3 API
simion314 3y ago
I had similar experience with OpenAI GPT API, following their rules we check the customer prompts with their endpoint for "safety" , then if it is safe we continue and generate the content. But sometimes even if the prompt was "safe" the result would be blocked as unsafe, because the AI either generated something "unsafe" from a safe prompt or the output filter had a false positive. Of course we had to pay for something that was not our fault. So it is clear that are many filters involved, some input filter, then the AI is "aligned" and threats you as a child , then there is an output filter.
- Mountain_Skies 3y ago
  Beige color clothes are to be avoided if any of the people in the image are young adults or younger. The input prompt filter correctly understands that the user isn't trying to generate anything "unsafe", but the output filter sees clothes that are too close to Caucasian flesh tone on someone younger than middle age and flags the output. Leave everything as-is but change the clothing color to something darker or leave the color alone and exclude children and young adults, then things come out fine. Though I can't see the flagged images, it seems clear that they're not generated nudes but false positives.
GaggiX 3y ago
They do but the technique shown in the paper is meant to bypass both. Paper: https://arxiv.org/pdf/2305.12082.pdf https://arxiv.org/pdf/2305.12082.pdf
herbst 3y ago
Most smaller API providers don't enforce any limits or filters like that.

Footnote7341 3y ago

Yes bing image creator uses two different levels of filters, one for the prompt and one for the output. gpt image creator does the same thing, as does the dalle-3 API

simion314 3y ago

I had similar experience with OpenAI GPT API, following their rules we check the customer prompts with their endpoint for "safety" , then if it is safe we continue and generate the content. But sometimes even if the prompt was "safe" the result would be blocked as unsafe, because the AI either generated something "unsafe" from a safe prompt or the output filter had a false positive. Of course we had to pay for something that was not our fault. So it is clear that are many filters involved, some input filter, then the AI is "aligned" and threats you as a child , then there is an output filter.

Mountain_Skies 3y ago
Beige color clothes are to be avoided if any of the people in the image are young adults or younger. The input prompt filter correctly understands that the user isn't trying to generate anything "unsafe", but the output filter sees clothes that are too close to Caucasian flesh tone on someone younger than middle age and flags the output. Leave everything as-is but change the clothing color to something darker or leave the color alone and exclude children and young adults, then things come out fine. Though I can't see the flagged images, it seems clear that they're not generated nudes but false positives.

GaggiX 3y ago

They do but the technique shown in the paper is meant to bypass both. Paper: https://arxiv.org/pdf/2305.12082.pdf https://arxiv.org/pdf/2305.12082.pdf

herbst 3y ago

Most smaller API providers don't enforce any limits or filters like that.