5 ms·

You can either work smarter or harder with content moderation. Unfortunately, most companies choose harder and just outsource the eyeballs to TPCs and cheap ov

You can either work smarter or harder with content moderation. Unfortunately, most companies choose harder and just outsource the eyeballs to TPCs and cheap overseas labor with predictable results.

qingcharles 2y ago
I get it though. If you have money, then the easier solution is the one you can just throw money at to get instant results; mental health of your employees be damned.
SpicyLemonZest 2y ago
When your platform gets large enough you have to choose harder. Facebook tried to choose smarter in Myanmar, and look at how well it worked out. There’s simply no algorithmic solution to the problem where your users genuinely want to see the content you’re trying to moderate. Even if you build flawless tools that ensure the algorithm won’t promote anyone talking about killing, your users will invent a new word “unaliving” and use that instead.
- onlypassingthru 2y ago
  ContentMod is a war and not a battle. Tooling should be flexible and adaptable as the adversaries' tactics change and evolve. One Algorithm To Moderate It All™ is a fool's errand. Today's nonsensical word/emoji/ASCII art is tomorrow's euphemism for drugs or hate or ???. The smart, and only lasting solution is to build a team of Navy Seals and give them advanced tools to quickly identify and neutralize the adversary before any significant damage is done.
wpietri 2y ago
I think you underestimate the extent to which companies try to do both. I know Twitter and Facebook both spent big for years and years on increasingly sophisticated ways to automatically recognize problematic content. Even they couldn't afford to just throw labor at the problem; a few years back when I ran the numbers Twitter's revenue was about $1/user/month, or about ¢1 per tweet. No matter where you outsource too, that doesn't buy much labor. If you look at the manuals for the outsourced moderators [1], it's clear they were being asked to work at a pretty sophisticated level. That level certainly wasn't available from ML systems before a couple of years ago, and I'm skeptical that it's there now, given that many of the hard moderation decisions come down to contextual judgements, and given how good awful people are at innovating their way around automated detection systems. [1] which you can do here among other places: https://www.theguardian.com/news/series/facebook-files https://www.theguardian.com/news/series/facebook-files
- onlypassingthru 2y ago
  The Integrity Ops moderation tools I am familiar with are often designed and built as an afterthought to the platform. Surfacing problematic content for review is only one piece of the puzzle and insufficient for effective moderation. The moderation described in the Guardian article is far from sophisticated. It is, in fact, the most basic way to moderate a platform. Create a queue of flagged content and have people manually review it.

You can either work smarter or harder with content moderation. Unfortunately, most companies choose harder and just outsource the eyeballs to TPCs and cheap overseas labor with predictable results.

qingcharles 2y ago

I get it though. If you have money, then the easier solution is the one you can just throw money at to get instant results; mental health of your employees be damned.

SpicyLemonZest 2y ago

When your platform gets large enough you have to choose harder. Facebook tried to choose smarter in Myanmar, and look at how well it worked out. There’s simply no algorithmic solution to the problem where your users genuinely want to see the content you’re trying to moderate. Even if you build flawless tools that ensure the algorithm won’t promote anyone talking about killing, your users will invent a new word “unaliving” and use that instead.

onlypassingthru 2y ago
ContentMod is a war and not a battle. Tooling should be flexible and adaptable as the adversaries' tactics change and evolve. One Algorithm To Moderate It All™ is a fool's errand. Today's nonsensical word/emoji/ASCII art is tomorrow's euphemism for drugs or hate or ???. The smart, and only lasting solution is to build a team of Navy Seals and give them advanced tools to quickly identify and neutralize the adversary before any significant damage is done.

wpietri 2y ago

I think you underestimate the extent to which companies try to do both. I know Twitter and Facebook both spent big for years and years on increasingly sophisticated ways to automatically recognize problematic content. Even they couldn't afford to just throw labor at the problem; a few years back when I ran the numbers Twitter's revenue was about $1/user/month, or about ¢1 per tweet. No matter where you outsource too, that doesn't buy much labor. If you look at the manuals for the outsourced moderators [1], it's clear they were being asked to work at a pretty sophisticated level. That level certainly wasn't available from ML systems before a couple of years ago, and I'm skeptical that it's there now, given that many of the hard moderation decisions come down to contextual judgements, and given how good awful people are at innovating their way around automated detection systems. [1] which you can do here among other places: https://www.theguardian.com/news/series/facebook-files https://www.theguardian.com/news/series/facebook-files

onlypassingthru 2y ago
The Integrity Ops moderation tools I am familiar with are often designed and built as an afterthought to the platform. Surfacing problematic content for review is only one piece of the puzzle and insufficient for effective moderation. The moderation described in the Guardian article is far from sophisticated. It is, in fact, the most basic way to moderate a platform. Create a queue of flagged content and have people manually review it.