15 ms·

Between bad actors taking advantage of DMCA takedowns and brigading mobs, it seems that automated (or semi-automated) moderating is absolutely a no-go at scale.

by dvt 5y ago

Between bad actors taking advantage of DMCA takedowns and brigading mobs, it seems that automated (or semi-automated) moderating is absolutely a no-go at scale. Incidentally, it's also why these networks (YouTube/Facebook/Twitter/etc.) can achieve such a massive scale with basically zero human moderation (i.e. cheaply). Remember when we had forums and there were a few unsung heroes that kept our communities clean and tidy(-ish)? Even here, I'm pretty sure @dang is basically reading HN and cleaning it up all day. Doing this at scale with semi-automated systems is quite literally impossible but until the courts start sanctioning social networks for their ad-hoc censorship (if this is even enforceable), we are sadly unlikely to see a change.

onion2k 5y ago
Obviously, automated (or semi-automated) moderating is absolutely a no-go at scale. I don't think that's obvious at all. You're saying that it's more important not to remove videos that aren't in violation of the platform's terms, eg false positives in the automated removal process, than it is to remove videos that are, eg actual content that should be removed. Why is it more important that someone's video remains on Facebook than Facebook removes illegal or toxic content? If acceptable videos are removed by mistake is that actually a problem that means the automated process isn't fit for purpose? ... social networks for their ad-hoc censorship... Facebook removing a video is not censorship. They're not saying "no one can see this video". They're saying "we don't want to host this video". Those are absolutely not the same thing.
- josephcsible 5y ago
  > Why is it more important that someone's video remains on Facebook than Facebook removes illegal or toxic content? For the same reason that "It is better that ten guilty persons escape than that one innocent suffer". > Facebook removing a video is not censorship. They're not saying "no one can see this video". They're saying "we don't want to host this video". Those are absolutely not the same thing. Facebook has so much market share that they're uncomfortably close to the same thing.
  - vkou 5y ago
    > For the same reason that "It is better that ten guilty persons escape than that one innocent suffer". Unfortunately, that is not a mantra that will keep your platform DMCA compliant. Also, how exactly does it apply to an internet mob calling for someone's head? Do we apply this mantra to protect the mob's behaviour, or to protect the person the mob is targeting?
    - josephcsible 5y ago
      There are plenty of contentious moderation decisions that are unrelated to the DMCA.
      - vkou 5y ago
        Which is why my post had two parts.
        josephcsible 5y ago
        For your second part: by not listening to the mob but not banning them either.
        croon 5y ago
        Great personal mantra, terrible preventive policy.
    - kevin_thibedeau 5y ago
      DMCA compliance is easy. You just have to respond to takedown notices expeditiously. Active moderation for copyright infringement is not required. You don't have to care about infringing content you don't know about if no notice is sent. You can force notices to be mailed to a PO box if that is the desired path for contacting your complaint agent.
      - vkou 5y ago
- dvt 5y ago
  > Why is it more important that someone's video remains on Facebook than Facebook removes illegal or toxic content? This is a straw-man; that's not what I'm saying at all. In fact, I'm arguing that you can probably have both -- by using human moderators. > Facebook removing a video is not censorship. It is censorship. They're saying "no one can see this video on our platform." Do you really want to get in some silly semantic debate when you know full well what I meant? It seems you're being purposefully uncharitable.
  - afarrell 5y ago
    > when you know full well what I meant Reminder: humans lack telepathy and so they depend on body language, tone, and word semantics to communicate.
    - unishark 5y ago
      It isn't really a semantic issue. Removing someone's communication from your site is literally censorship. It's simply not govt censorship. But censorship by quasi-monopolies makes it arguably similar and still an ethics debate.
      - onion2k 5y ago
        Are you suggesting that video platforms should not be allowed to remove content? That's a very extreme point of view.
        unishark 5y ago
        There are two sides in the debate I referred to and my post did not advocate either side.
        MikeUt 5y ago
        I want to highlight the very common, very dishonest debate trick you just performed. You skipped right past calling the thing for what it is (censorship), past admitting it's a problem, and you're now arguing against a hypothetical extreme solution no-one has yet proposed. Can we assume that you now agree it's censorship, and that it's a problem, and it's only the solution you proposed that bothers you?
Wowfunhappy 5y ago
The trouble is, manual moderating systems are also a no-go at scale. So what do you do?
- chmod775 5y ago
  You revert back to becoming a 'dumb pipe' and make moderating someone else's problem. Your ISP doesn't moderate for you either.
  - xvector 5y ago
    Okay, now we are back where we started. How does that 'someone else' moderate content at scale? Eventually someone has to take the burden.
    - curryst 5y ago
      I think it basically looks like Reddit. You carve up the site into fiefdoms and then have users self-moderate. Moderation that happens at the site layer largely targets fiefdoms rather than users. New accounts are easy to create, but new communities are harder because you have to wait for members. Plus most of your moderators are free. I wouldn't be surprised if Reddit only pays for people to moderate the default subs since advertisers and new users are more likely to see them.
    - toast0 5y ago
      If we're all bunched into small communities, there is no 'at scale'. I regularly hang out in several different communities of thousands of people, including this one. All of them, except the really small ones, have one or a few people who moderate the forums, and ways to report things that are posted. If someone posts a link to disturbing video to HN, it's going to disappear right quick; even if the source lives on. If I try out a community where there's a lot of disturbing videos posted, I'm going to stop going there; but other people might go there specifically because of those video links. If the videos are legal, that's their business and none of mine; I don't want to see it, and by avoiding their community, I won't.
  - 5y ago
creato 5y ago
I think there's less difference than you think. Big social networks employ moderators too. People complain about the moderators on small forums too. I think the only difference is that politicians don't even pretend to care about the complaints of moderators on small forums.
- bccdee 5y ago
  The thing is, small forums can wind up being completely good or completely bad, because the net quality of a small mod team has a high variance. So if you look properly, you can find the good ones. On a big platform, you wind up with a very uniform mediocrity.
ergot_vacation 5y ago
Back when Net Neutrality was hot topic, the nightmare scenario that was always trotted out was that the Internet would become cable television, devoid of the wide diversity of content and voices it has today. Every time I see a story like this (and there are a LOT of them) I half-jokingly wonder if we'll reach that future not because of the collapse of NN, but simply because everyone other than corporations will have been banned from all major platforms. With that in mind, consider that this "failure" of scale may be less of a bug and more of a feature as far as the platforms are concerned. All the big players HATE the Internet as it originally existed. All those independent voices talking all over the place were difficult or impossible to monetize, and the legal hazard of trying to do business near anything resembling a truly free discussion forum is a nightmare. Far better to do away with all that pesky multi-directional communication and go back to the good old days: big company make product, big company market product, little people buy product. No need for discussion or community. That just gets in the way. Maybe, if your lords are feeling generous, they'll give you a comment section where you can talk about how great product is. Any other posts will of course be scrubbed.
- AussieWog93 5y ago
  I think if all independent voices were banned from large platforms, people would just move elsewhere. I'm not saying that the large platform would collapse, but that a more independent platform would operate in parallel and take care of peoples' desire to both create and consume independent content. It's a cycle that's been going on since at least the 19th century (via pamphlets), and I don't see it stopping any time soon.
  - trhway 5y ago
    the key difference of this time is network effect. People aren't going to move elsewhere, and instead they (more precisely - their brains) will be slow boiled like that frog.
  - taneq 5y ago
    Of course, the draw card is that people can add their comments and feel like they're involved. You don't want anyone else actually reading the great unwashed's comments, though!
darig 5y ago
> it seems that automated (or semi-automated) moderating is absolutely a no-go at scale. It follows that society of any kind is a no-go at scale.
nradov 5y ago
Does is actually matter for Facebook as a business? Outside the HN bubble, very few users will actually quit just because some random page was suspended. Most users won't even notice.
- kevin_thibedeau 5y ago
  It matters when someone with a high enough profile can trigger an investigation into their business practices.
lettergram 5y ago
IMO it’s because he’s the owner of theDonald.win (community migrated to patriots.win). My bet is his voice triggered the system.
- belatw 5y ago
  His voice, tone and mannerisms are pretty triggering for sure. They trigger my gag reflexes.
- fersc 5y ago
  where does it say he's the owner of thedonald.win?
halflings 5y ago
From another one of your comments: > I'm arguing that you can probably have both -- by using human moderators. I don't think human moderators can review 720K hours of video per day. (on YouTube) You would need to hire >100,000 moderators to continuously watch videos all day and have some redundancy in reviews, then 1,000s of supervisors to make sure moderators are doing their job correctly. (almost impossible when you review videos all day with no break) Let's not even get into comment moderation...
- hutzlibu 5y ago
  It would be enough, if human moderators would decide over "flagged" content - or in general - review the KI decisions.
  - andreareina 5y ago
    And flags from accounts that routinely flag ok content should count less.
    - hutzlibu 5y ago
      Or not at all obviously. And hundreds of other fine tuning measures, sure.
phonebucket 5y ago
> Even here, I'm pretty sure @dang is basically reading HN and cleaning it up all day. I'm not sure about this. I believe there are automated systems and shadow bans on HN (not entirely sure how they work). There's certainly community moderation via flagging and downvoting, which is essentially how the FB video was removed as well.
- fsflover 5y ago
  https://news.ycombinator.com/item?id=25048415 https://news.ycombinator.com/item?id=25048415

by dvt 5y ago

onion2k 5y ago

Obviously, automated (or semi-automated) moderating is absolutely a no-go at scale. I don't think that's obvious at all. You're saying that it's more important not to remove videos that aren't in violation of the platform's terms, eg false positives in the automated removal process, than it is to remove videos that are, eg actual content that should be removed. Why is it more important that someone's video remains on Facebook than Facebook removes illegal or toxic content? If acceptable videos are removed by mistake is that actually a problem that means the automated process isn't fit for purpose? ... social networks for their ad-hoc censorship... Facebook removing a video is not censorship. They're not saying "no one can see this video". They're saying "we don't want to host this video". Those are absolutely not the same thing.

josephcsible 5y ago
> Why is it more important that someone's video remains on Facebook than Facebook removes illegal or toxic content? For the same reason that "It is better that ten guilty persons escape than that one innocent suffer". > Facebook removing a video is not censorship. They're not saying "no one can see this video". They're saying "we don't want to host this video". Those are absolutely not the same thing. Facebook has so much market share that they're uncomfortably close to the same thing.
- vkou 5y ago
  > For the same reason that "It is better that ten guilty persons escape than that one innocent suffer". Unfortunately, that is not a mantra that will keep your platform DMCA compliant. Also, how exactly does it apply to an internet mob calling for someone's head? Do we apply this mantra to protect the mob's behaviour, or to protect the person the mob is targeting?
  - josephcsible 5y ago
    There are plenty of contentious moderation decisions that are unrelated to the DMCA.
    - vkou 5y ago
      Which is why my post had two parts.
      - josephcsible 5y ago
        For your second part: by not listening to the mob but not banning them either.
        croon 5y ago
        Great personal mantra, terrible preventive policy.
  - kevin_thibedeau 5y ago
    DMCA compliance is easy. You just have to respond to takedown notices expeditiously. Active moderation for copyright infringement is not required. You don't have to care about infringing content you don't know about if no notice is sent. You can force notices to be mailed to a PO box if that is the desired path for contacting your complaint agent.
    - vkou 5y ago
dvt 5y ago
> Why is it more important that someone's video remains on Facebook than Facebook removes illegal or toxic content? This is a straw-man; that's not what I'm saying at all. In fact, I'm arguing that you can probably have both -- by using human moderators. > Facebook removing a video is not censorship. It is censorship. They're saying "no one can see this video on our platform." Do you really want to get in some silly semantic debate when you know full well what I meant? It seems you're being purposefully uncharitable.
- afarrell 5y ago
  > when you know full well what I meant Reminder: humans lack telepathy and so they depend on body language, tone, and word semantics to communicate.
  - unishark 5y ago
    It isn't really a semantic issue. Removing someone's communication from your site is literally censorship. It's simply not govt censorship. But censorship by quasi-monopolies makes it arguably similar and still an ethics debate.
    - onion2k 5y ago
      Are you suggesting that video platforms should not be allowed to remove content? That's a very extreme point of view.
      - unishark 5y ago
        There are two sides in the debate I referred to and my post did not advocate either side.
      - MikeUt 5y ago
        I want to highlight the very common, very dishonest debate trick you just performed. You skipped right past calling the thing for what it is (censorship), past admitting it's a problem, and you're now arguing against a hypothetical extreme solution no-one has yet proposed. Can we assume that you now agree it's censorship, and that it's a problem, and it's only the solution you proposed that bothers you?

Wowfunhappy 5y ago

The trouble is, manual moderating systems are also a no-go at scale. So what do you do?

chmod775 5y ago
You revert back to becoming a 'dumb pipe' and make moderating someone else's problem. Your ISP doesn't moderate for you either.
- xvector 5y ago
  Okay, now we are back where we started. How does that 'someone else' moderate content at scale? Eventually someone has to take the burden.
  - curryst 5y ago
    I think it basically looks like Reddit. You carve up the site into fiefdoms and then have users self-moderate. Moderation that happens at the site layer largely targets fiefdoms rather than users. New accounts are easy to create, but new communities are harder because you have to wait for members. Plus most of your moderators are free. I wouldn't be surprised if Reddit only pays for people to moderate the default subs since advertisers and new users are more likely to see them.
  - toast0 5y ago
    If we're all bunched into small communities, there is no 'at scale'. I regularly hang out in several different communities of thousands of people, including this one. All of them, except the really small ones, have one or a few people who moderate the forums, and ways to report things that are posted. If someone posts a link to disturbing video to HN, it's going to disappear right quick; even if the source lives on. If I try out a community where there's a lot of disturbing videos posted, I'm going to stop going there; but other people might go there specifically because of those video links. If the videos are legal, that's their business and none of mine; I don't want to see it, and by avoiding their community, I won't.
- 5y ago

creato 5y ago

I think there's less difference than you think. Big social networks employ moderators too. People complain about the moderators on small forums too. I think the only difference is that politicians don't even pretend to care about the complaints of moderators on small forums.

bccdee 5y ago
The thing is, small forums can wind up being completely good or completely bad, because the net quality of a small mod team has a high variance. So if you look properly, you can find the good ones. On a big platform, you wind up with a very uniform mediocrity.

ergot_vacation 5y ago

Back when Net Neutrality was hot topic, the nightmare scenario that was always trotted out was that the Internet would become cable television, devoid of the wide diversity of content and voices it has today. Every time I see a story like this (and there are a LOT of them) I half-jokingly wonder if we'll reach that future not because of the collapse of NN, but simply because everyone other than corporations will have been banned from all major platforms. With that in mind, consider that this "failure" of scale may be less of a bug and more of a feature as far as the platforms are concerned. All the big players HATE the Internet as it originally existed. All those independent voices talking all over the place were difficult or impossible to monetize, and the legal hazard of trying to do business near anything resembling a truly free discussion forum is a nightmare. Far better to do away with all that pesky multi-directional communication and go back to the good old days: big company make product, big company market product, little people buy product. No need for discussion or community. That just gets in the way. Maybe, if your lords are feeling generous, they'll give you a comment section where you can talk about how great product is. Any other posts will of course be scrubbed.

AussieWog93 5y ago
I think if all independent voices were banned from large platforms, people would just move elsewhere. I'm not saying that the large platform would collapse, but that a more independent platform would operate in parallel and take care of peoples' desire to both create and consume independent content. It's a cycle that's been going on since at least the 19th century (via pamphlets), and I don't see it stopping any time soon.
- trhway 5y ago
  the key difference of this time is network effect. People aren't going to move elsewhere, and instead they (more precisely - their brains) will be slow boiled like that frog.
- taneq 5y ago
  Of course, the draw card is that people can add their comments and feel like they're involved. You don't want anyone else actually reading the great unwashed's comments, though!

darig 5y ago

> it seems that automated (or semi-automated) moderating is absolutely a no-go at scale. It follows that society of any kind is a no-go at scale.

nradov 5y ago

Does is actually matter for Facebook as a business? Outside the HN bubble, very few users will actually quit just because some random page was suspended. Most users won't even notice.

kevin_thibedeau 5y ago
It matters when someone with a high enough profile can trigger an investigation into their business practices.

lettergram 5y ago

IMO it’s because he’s the owner of theDonald.win (community migrated to patriots.win). My bet is his voice triggered the system.

belatw 5y ago
His voice, tone and mannerisms are pretty triggering for sure. They trigger my gag reflexes.
fersc 5y ago
where does it say he's the owner of thedonald.win?

halflings 5y ago

From another one of your comments: > I'm arguing that you can probably have both -- by using human moderators. I don't think human moderators can review 720K hours of video per day. (on YouTube) You would need to hire >100,000 moderators to continuously watch videos all day and have some redundancy in reviews, then 1,000s of supervisors to make sure moderators are doing their job correctly. (almost impossible when you review videos all day with no break) Let's not even get into comment moderation...

hutzlibu 5y ago
It would be enough, if human moderators would decide over "flagged" content - or in general - review the KI decisions.
- andreareina 5y ago
  And flags from accounts that routinely flag ok content should count less.
  - hutzlibu 5y ago
    Or not at all obviously. And hundreds of other fine tuning measures, sure.

phonebucket 5y ago

> Even here, I'm pretty sure @dang is basically reading HN and cleaning it up all day. I'm not sure about this. I believe there are automated systems and shadow bans on HN (not entirely sure how they work). There's certainly community moderation via flagging and downvoting, which is essentially how the FB video was removed as well.

fsflover 5y ago
https://news.ycombinator.com/item?id=25048415 https://news.ycombinator.com/item?id=25048415