3 ms·

I saw a heartbreaking YouTube mini-documentary on Facebook moderators.[1] The American workers seem to get the "lighter" flags like animal abuse, which is more

I saw a heartbreaking YouTube mini-documentary on Facebook moderators.[1] The American workers seem to get the "lighter" flags like animal abuse, which is more than enough to cause trauma with daily exposure. I shudder to think what the offshore moderators go through when dealing with human/child abuse flags. In my view, Facebook clearly wants this to be a temporary evil, so they can use the data from human moderators as a training set for automated ML-based moderation. But I wonder how long people will have to endure this for those models to reach an acceptable efficacy. [1] https://www.youtube.com/watch?v=bDnjiNCtFk4 https://www.youtube.com/watch?v=bDnjiNCtFk4

munk-a 7y ago
> In my view, Facebook clearly wants this to be a temporary evil, so they can use the data from human moderators as a training set for automated ML-based moderation. But I wonder how long people will have to endure this for those models to reach an acceptable efficacy. Alternatively I wonder if it's worth it to put any number of human beings through this kind of suffering for something as worthless as social networking.
- malandrew 7y ago
  Let's assume social networking is worthless per your judgment: Do you honestly think this problem will go away if centralized social networks go away?
  - munk-a 7y ago
    Yes, this is a new thing AFAICT. Magazines that frequently published reader content didn't have to sift through quite as vile content when going through letters - death threats and the like have always been a thing but somehow we've lost the cost of creating vile content in this modern era. I've never heard of anyone getting a visit from the FBI over vague death threats on facebook.
    - celtain 7y ago
      This kind of content came with the internet, not social networks. Traditional forum moderators have had to deal with really vile content for a long time now, along with the operators of almost any other media sharing website. Unless you want to make it illegal for individuals to widely share content without going through an editor, this kind of problem isn't going away any time soon.
      - jessaustin 7y ago
        It really depends on how any particular site functions. Some sites function better than others. For example, "this kind of problem" doesn't seem to plague us here on HN.
    - Retric 7y ago
      You might find this interesting as it relates to a 44-month imprisonment and Facebook. https://www.forbes.com/sites/kashmirhill/2014/06/16/supreme-court-facebook-death-threats/#a0288081e617 https://www.forbes.com/sites/kashmirhill/2014/06/16/supreme-... On June 1, 2015, the Supreme Court reversed Elonis's conviction in an 8-1 decision. https://en.m.wikipedia.org/wiki/Elonis_v._United_States#Decision https://en.m.wikipedia.org/wiki/Elonis_v._United_States#Deci...
- jessaustin 7y ago
  You, too, can walk away from Omelas.
  - michaelmrose 7y ago
    It's a beautiful story. https://www.utilitarianism.com/nu/omelas.pdf https://www.utilitarianism.com/nu/omelas.pdf The Ones Who Walk Away from Omelas By Ursula LeGuin
benumbed 7y ago
So, without a doubt, there is some chilling, mind altering content out there. Videos providing evidence of criminal acts of violence that inspire a deep disturbance of emotion lasting days, even among hardened viewers. But the truth is that it's all a pathetic joke. You can't police telephone calls, and the conference lines have to police themselves. This is the price of decentralization and the democratization of content creation. There can no longer persist any such dictatorship of standards and practices. There's only one maxim that applies: Don't talk to strangers. End of story. There are certain manifolds and convolutions that are unhygienic, and if you have a problem with that, don't build them and don't explore them. But, a network is a network is a network. It's a system of passages. Hallways with doors. In some respects, these policing activities and attempts to moderate a mob represent an obvious over-extension beyond the normal scope of operations. It's silly to even think it's possible. But hey, maybe Facebook is catching quite the government hand out, just to experiment with some "DARPA hard" problem solving. Personally, I find it laughable that anyone imagines this problem to be even remotely generalizable enough to fit a statistical model that the current state of "AI" buzzword juicing vaporware purveyors and hucksters might jazz hands a pitch deck with. Imagine: AI staring at clouds, noticing familiar shapes among the drifting cotton balls. How productive. You could waste an entire nuclear power plant's total output on such a thing, and get nothing for it. Attack the right problem at the root. These violent, malignant people torturing the animals and murdering on livestreams. Some will get through, and the internet will simply reflect reality. But make the world a better place. Don't spin your wheels papering over the blood stains. The reality is that these images videos are going to be shared and shared again. The cat's out of the bag. Accept the bald face of the human condition, warts and all.
tjpnz 7y ago
I'm guessing that most of the stuff heard about Facebook and ML is basically hype. Even after Christchurch the best they could do to counteract the spread was content fingerprinting. Human moderators are likely their longterm solution.

munk-a 7y ago

> In my view, Facebook clearly wants this to be a temporary evil, so they can use the data from human moderators as a training set for automated ML-based moderation. But I wonder how long people will have to endure this for those models to reach an acceptable efficacy. Alternatively I wonder if it's worth it to put any number of human beings through this kind of suffering for something as worthless as social networking.

malandrew 7y ago
Let's assume social networking is worthless per your judgment: Do you honestly think this problem will go away if centralized social networks go away?
- munk-a 7y ago
  Yes, this is a new thing AFAICT. Magazines that frequently published reader content didn't have to sift through quite as vile content when going through letters - death threats and the like have always been a thing but somehow we've lost the cost of creating vile content in this modern era. I've never heard of anyone getting a visit from the FBI over vague death threats on facebook.
  - celtain 7y ago
    This kind of content came with the internet, not social networks. Traditional forum moderators have had to deal with really vile content for a long time now, along with the operators of almost any other media sharing website. Unless you want to make it illegal for individuals to widely share content without going through an editor, this kind of problem isn't going away any time soon.
    - jessaustin 7y ago
      It really depends on how any particular site functions. Some sites function better than others. For example, "this kind of problem" doesn't seem to plague us here on HN.
  - Retric 7y ago
    You might find this interesting as it relates to a 44-month imprisonment and Facebook. https://www.forbes.com/sites/kashmirhill/2014/06/16/supreme-court-facebook-death-threats/#a0288081e617 https://www.forbes.com/sites/kashmirhill/2014/06/16/supreme-... On June 1, 2015, the Supreme Court reversed Elonis's conviction in an 8-1 decision. https://en.m.wikipedia.org/wiki/Elonis_v._United_States#Decision https://en.m.wikipedia.org/wiki/Elonis_v._United_States#Deci...
jessaustin 7y ago
You, too, can walk away from Omelas.
- michaelmrose 7y ago
  It's a beautiful story. https://www.utilitarianism.com/nu/omelas.pdf https://www.utilitarianism.com/nu/omelas.pdf The Ones Who Walk Away from Omelas By Ursula LeGuin

benumbed 7y ago

So, without a doubt, there is some chilling, mind altering content out there. Videos providing evidence of criminal acts of violence that inspire a deep disturbance of emotion lasting days, even among hardened viewers. But the truth is that it's all a pathetic joke. You can't police telephone calls, and the conference lines have to police themselves. This is the price of decentralization and the democratization of content creation. There can no longer persist any such dictatorship of standards and practices. There's only one maxim that applies: Don't talk to strangers. End of story. There are certain manifolds and convolutions that are unhygienic, and if you have a problem with that, don't build them and don't explore them. But, a network is a network is a network. It's a system of passages. Hallways with doors. In some respects, these policing activities and attempts to moderate a mob represent an obvious over-extension beyond the normal scope of operations. It's silly to even think it's possible. But hey, maybe Facebook is catching quite the government hand out, just to experiment with some "DARPA hard" problem solving. Personally, I find it laughable that anyone imagines this problem to be even remotely generalizable enough to fit a statistical model that the current state of "AI" buzzword juicing vaporware purveyors and hucksters might jazz hands a pitch deck with. Imagine: AI staring at clouds, noticing familiar shapes among the drifting cotton balls. How productive. You could waste an entire nuclear power plant's total output on such a thing, and get nothing for it. Attack the right problem at the root. These violent, malignant people torturing the animals and murdering on livestreams. Some will get through, and the internet will simply reflect reality. But make the world a better place. Don't spin your wheels papering over the blood stains. The reality is that these images videos are going to be shared and shared again. The cat's out of the bag. Accept the bald face of the human condition, warts and all.

tjpnz 7y ago

I'm guessing that most of the stuff heard about Facebook and ML is basically hype. Even after Christchurch the best they could do to counteract the spread was content fingerprinting. Human moderators are likely their longterm solution.