3 ms·
There's a reason there's been so much research into AI moderation, although so far it always seems to either be useless or throwing babies out with the bathwate
by coldacid 5y ago
There's a reason there's been so much research into AI moderation, although so far it always seems to either be useless or throwing babies out with the bathwater. Not to mention that without human systems to review what the AI decides (or even with) it tends to appear to users as automatic censorship (and they're usually not wrong about that).
- yeldarb 5y agoWe had pretty good success on https://paint.wtf https://paint.wtf (AI-judged Pictionary game) using CLIP for content moderation. Feel free to probe it a bit by drawing a dick or something on one of the prompts; it seems to do a pretty good job (if it doesn’t work please hit the “Report” button and let us know). Wrote about it here: https://blog.roboflow.com/zero-shot-content-moderation-openai-new-clip-model/ https://blog.roboflow.com/zero-shot-content-moderation-opena...
- PaulHoule 5y agoThe difficulty of it is the imbalance. Once I had 10,000 NSFW images in a collection of 1,000,000. You might say that 99% of the images are good and for most classification problems 99% accuracy would be wonderful. But it's not good enough. You might find some simple tricks that eliminate 90% of those NSFW images and now you have 99.9% accuracy. That makes you look like a genius in the machine learning world. But it's not good enough. Even 99.99% accuracy (100 bad images) is not good enough Even if somebody found one NSFW image you could get kicked out of Adsense. One child porn image uploaded to your site won't put you in jail but it is still completely unacceptable.