4 ms·

> Can we build better AI Nope. That's what the people in charge of our social media companies keep trying, over and over, and it doesn't work at all. AI-modera

> Can we build better AI Nope. That's what the people in charge of our social media companies keep trying, over and over, and it doesn't work at all. AI-moderation is like the Ring is to Golem for them. They can't accept that "cheap and fair" moderation isn't currently possible. > Can you crowdsource the moderation task This is the clear answer, imo, and it isn't obvious to me why these C-level execs avoid it. If I had to make a guess, maybe they think engaging with millions of their users about moderation would open a can of worms.

shadowgovt 6y ago
When the goal is to prevent users from seeing this content, crowd-sourcing moderation defeats the purpose. Users don't want the back-stop to be "a critical mass of community members object;" for content like this, they want to see zero of it, ever, and will choose to use another site that satisfies that hard constraint if FB cannot.
- uniqueid 6y ago
  > they want to see zero of it, ever Many want that, but very few (probably nobody at all, by this point) expect it. > When the goal is to prevent users from > seeing this content, crowd-sourcing moderation > defeats the purpose. It's not actually all-or-nothing, and if it were, it would defeat it no less frequently than paid moderation already does. People tolerated toxic internet content before social media not because they never encountered any, but because they encountered much less. No system can ever stop 100% of users from seeing 100% of the material they find objectionable. Every user has their own idea of what material is beyond the pale, and even if we invent AI that is competent at blocking content, and capable of tailoring its filters per-user, it will still fail some of the time, because users are people, and people's tastes change over time. Aside from that, the way Youtube, Reddit, Twitter et al, currently operate, a user who reports objectionable content typically waits days, weeks, or years (or forever) for the company to take action. If you give a user a little real agency, it goes a long way to mitigate their displeasure over occasional exposure to unwanted content.
  - shadowgovt 6y ago
    What real agency does the system you propose offer? Because one down-moderation from a volunteer moderator is a drop in the bucket. Meanwhile, if your system requires users to view beheading videos regularly, people will just migrate to a site that doesn't require that.
    - uniqueid 6y ago
      If I had to come up with a system, it would be some sort of reputation hierarchy, with a small number of paid employees at the top. Each level is responsible for auditing the levels under them. > What real agency does the system you propose offer? The ability for millions of users to moderate themselves. So... fast response time, by opinionated, emotionally-invested moderators, as opposed to the status quo: slow response time by paid burn-outs following a flow-chart. > if your system requires users to view beheading videos regularly I'm pretty sure that content would be reported faster than anything else. To be fair, another part of the issue is ban-evasion using multiple accounts, and that indeed requires additional measures to handle. Sadly, there's a disincentive to dealing with fake accounts, because trolls count as "sign-ups" too. > people will just migrate to a site that doesn't require that. Like Youtube and Facebook still occassionally do. Well, most users haven't abandoned them yet, so there goes that theory :)
      - shadowgovt 6y ago
        > occasionally Precisely. How often? Close to never.
        uniqueid 6y ago
        Give or take a live-streamed Christchurch massacre?
        shadowgovt 6y ago
        Yes. One disastrous livestream in the history of the feature, with further controls added almost immediately as a result.
selimthegrim 6y ago
I don’t think Gollum had a Hebrew letter on his back...
- uniqueid 6y ago
  Right you are. It's been a while since I last read The King of the Rings.

shadowgovt 6y ago

When the goal is to prevent users from seeing this content, crowd-sourcing moderation defeats the purpose. Users don't want the back-stop to be "a critical mass of community members object;" for content like this, they want to see zero of it, ever, and will choose to use another site that satisfies that hard constraint if FB cannot.

uniqueid 6y ago
> they want to see zero of it, ever Many want that, but very few (probably nobody at all, by this point) expect it. > When the goal is to prevent users from > seeing this content, crowd-sourcing moderation > defeats the purpose. It's not actually all-or-nothing, and if it were, it would defeat it no less frequently than paid moderation already does. People tolerated toxic internet content before social media not because they never encountered any, but because they encountered much less. No system can ever stop 100% of users from seeing 100% of the material they find objectionable. Every user has their own idea of what material is beyond the pale, and even if we invent AI that is competent at blocking content, and capable of tailoring its filters per-user, it will still fail some of the time, because users are people, and people's tastes change over time. Aside from that, the way Youtube, Reddit, Twitter et al, currently operate, a user who reports objectionable content typically waits days, weeks, or years (or forever) for the company to take action. If you give a user a little real agency, it goes a long way to mitigate their displeasure over occasional exposure to unwanted content.
- shadowgovt 6y ago
  What real agency does the system you propose offer? Because one down-moderation from a volunteer moderator is a drop in the bucket. Meanwhile, if your system requires users to view beheading videos regularly, people will just migrate to a site that doesn't require that.
  - uniqueid 6y ago
    If I had to come up with a system, it would be some sort of reputation hierarchy, with a small number of paid employees at the top. Each level is responsible for auditing the levels under them. > What real agency does the system you propose offer? The ability for millions of users to moderate themselves. So... fast response time, by opinionated, emotionally-invested moderators, as opposed to the status quo: slow response time by paid burn-outs following a flow-chart. > if your system requires users to view beheading videos regularly I'm pretty sure that content would be reported faster than anything else. To be fair, another part of the issue is ban-evasion using multiple accounts, and that indeed requires additional measures to handle. Sadly, there's a disincentive to dealing with fake accounts, because trolls count as "sign-ups" too. > people will just migrate to a site that doesn't require that. Like Youtube and Facebook still occassionally do. Well, most users haven't abandoned them yet, so there goes that theory :)
    - shadowgovt 6y ago
      > occasionally Precisely. How often? Close to never.
      - uniqueid 6y ago
        Give or take a live-streamed Christchurch massacre?
        shadowgovt 6y ago
        Yes. One disastrous livestream in the history of the feature, with further controls added almost immediately as a result.

selimthegrim 6y ago

I don’t think Gollum had a Hebrew letter on his back...

uniqueid 6y ago
Right you are. It's been a while since I last read The King of the Rings.