4 ms·

I made a local anonymous chat app with AI for content moderation

• brianabedi 3y ago
  I was inspired by Yik Yak’s early success, and subsequent failure due to an inability to manage bad content from anonymous users. I realized automated content moderation was the answer to this problem, and in 3 weeks spun up a full stack app integrated with a ML model. Test it if you have time and please don’t hold back with the feedback!
• brucethemoose2 3y ago
  What do you use for moderation? OpenAI API? This seems like a decent candidate for local inference on iPhones. You could make the app almost server free, theoretically, and still have each client moderate themselves, and I would say Mistral 7B is now "smart enough" for moderation, or maybe some smaller safety filter.
  • brianabedi 3y ago
    I’m using Google’s perspective api, it’s quite nuanced in its assessment of different types of abusive content. I set a threshold on 2 parameters that I feel does a great job of filtering out incendiary comments but allowing jokes that might be slightly risky but still acceptable. That’s a great point about the iOS local inference option, I’m currently using Expo so it’s cross platform and my current set up is almost instant with Perspective api. I will def explore using Swift and apples native ML models for future apps though so thanks for suggesting
    • Mobil1 3y ago
      [dead]