3 ms·
there are built in moderation tools you should turn on if you have external customers generating images, or inputing data that might be sketch
by strangescript 11mo ago
there are built in moderation tools you should turn on if you have external customers generating images, or inputing data that might be sketch
- ceejayoz 11mo agoAnd these tools are perfect?
- bhouston 11mo agoIt can be super hard to moderate before an image is generated though. People can write in cryptic language and then say decode this message and generate an image the result, etc. The downside of LLMs is that they are super hard to moderate because they will gladly arbitrarily encode input and output. You need to use an LLM as advanced as the one you are running in production to actually check if they are obscene.
- fukka42 11mo ago[dead]
- samtheprogram 11mo agoThe example in this blog post, they did something recommended by Google and still got banned. Based on that, I'm not sure their built in moderation tools are enough insurance.