3 ms·

This is the trouble with automation. It's clear this isn't a malicious post, it just matched some keywords their moderation bot identified as such. I think a l

This is the trouble with automation. It's clear this isn't a malicious post, it just matched some keywords their moderation bot identified as such. I think a lot of the censorship problems would be resolved if they just shut the bots off and relied on user flagging. Does that require a lot more people? Sure. But the long-run result would be far more people would use and trust these networks (covering the revenue of hiring moderators). I know I'd be a lot happier if there was a thinking human deciding my fate than a random script that only a few people know the inner-workings of. As-is, it seems like a lot of these social networks are just shooting themselves in the foot just to avoid costs and get a false sense of control over the problem.

maybesomaybenot 2y ago
Um, no. I don't want to see pics of NSFL gore before the userbase has had a chance to remove them. Which is what most moderators spend time removing from FB, to the point where it psychologically traumatizes them.
- rglover 2y ago
  You don't have to. That's actually a place where automation could help. You could just use image detection and auto-tag stuff as to what you think it contains. Then, have a list of sensitive tags that are automatically blurred out in the feed (and let users customize the list as they see fit). If it's something trending towards illegal, toss it into an "emergency" queue for moderators to hand-verify and don't make it visible until it's been checked. So in your example, if someone uploads war imagery, it would be tagged as "war," "violence," "gore" and be auto-blurred for users. That doesn't mean the post or account needs to be outright nuked, just treated differently from SFW stuff.
  - maybesomaybenot 2y ago
    You assume automation solves the problem. If it did, Facebook wouldn't be hiring close to 100,000 humans to inspect content.
    - rglover 2y ago
      Automation + human intervention, yes. In the setup I described, worst case scenario something gets blurred out that's benign, but it doesn't create a press/support nightmare for Meta. Considering they've open sourced one of their image detection API [1], I'd imagine it's more a problem of accuracy and implementation at scale than a serious technical hurdle. [1] https://github.com/facebookresearch/detectron2 https://github.com/facebookresearch/detectron2
  - notfed 2y ago
    Those are subjective clarifications, and so will differ between each person. And models are pre-trained to recognize these classifications. Since you mentioned war, I'm reminded of Black Mirror episode "Men Against Fire", where an army of soldiers have eye implants that cause them to visually see enemy soldiers as unsightly. (My point being this is effectively what Facebook can do.)
- BlueTemplar 2y ago
  Is there really no legal way to go after gore posters (in spaces banning gore) ? There should be - after all, this is akin to graffiti, which is typically fined. What is not acceptable, is a platform creating a paralegal environment.

maybesomaybenot 2y ago

Um, no. I don't want to see pics of NSFL gore before the userbase has had a chance to remove them. Which is what most moderators spend time removing from FB, to the point where it psychologically traumatizes them.

rglover 2y ago
You don't have to. That's actually a place where automation could help. You could just use image detection and auto-tag stuff as to what you think it contains. Then, have a list of sensitive tags that are automatically blurred out in the feed (and let users customize the list as they see fit). If it's something trending towards illegal, toss it into an "emergency" queue for moderators to hand-verify and don't make it visible until it's been checked. So in your example, if someone uploads war imagery, it would be tagged as "war," "violence," "gore" and be auto-blurred for users. That doesn't mean the post or account needs to be outright nuked, just treated differently from SFW stuff.
- maybesomaybenot 2y ago
  You assume automation solves the problem. If it did, Facebook wouldn't be hiring close to 100,000 humans to inspect content.
  - rglover 2y ago
    Automation + human intervention, yes. In the setup I described, worst case scenario something gets blurred out that's benign, but it doesn't create a press/support nightmare for Meta. Considering they've open sourced one of their image detection API [1], I'd imagine it's more a problem of accuracy and implementation at scale than a serious technical hurdle. [1] https://github.com/facebookresearch/detectron2 https://github.com/facebookresearch/detectron2
- notfed 2y ago
  Those are subjective clarifications, and so will differ between each person. And models are pre-trained to recognize these classifications. Since you mentioned war, I'm reminded of Black Mirror episode "Men Against Fire", where an army of soldiers have eye implants that cause them to visually see enemy soldiers as unsightly. (My point being this is effectively what Facebook can do.)
BlueTemplar 2y ago
Is there really no legal way to go after gore posters (in spaces banning gore) ? There should be - after all, this is akin to graffiti, which is typically fined. What is not acceptable, is a platform creating a paralegal environment.