6 ms·

As with Blizzard, Facebook seem to have some deep aversion to manual moderation, and will spend millions on getting developers to make half-hearted attempts to

As with Blizzard, Facebook seem to have some deep aversion to manual moderation, and will spend millions on getting developers to make half-hearted attempts to automate it without actually solving the problem. This is a real band aid that will do nothing to dissuade people from continuing their abusive behaviour.

post-it 5y ago
Manual moderation scales linearly: for X posts per day, you need nX moderators, forever. Silicon Valley execs are allergic to anything that doesn't scale sub-linearly. It makes them feel like the singularity might not be around the corner.
smeyer 5y ago
>will spend millions on getting developers to make half-hearted attempts What do you think the annual cost of manual moderation would be?
- hellomyguys 5y ago
  Facebook can afford manual moderation. What do you think the mental health cost of waiting for AI moderation to be good enough in 111 languages that understand the nuances of speech in text/audio/video?
  - smeyer 5y ago
    I'm not saying that the expense of manual moderation would excuse doing a poor job of moderating, but the top comment talked about spending "millions" on trying to automate moderation as if manual moderation wouldn't cost as much. They're certainly spending many millions on both manual and automatic moderation as it currently stands. They've spent tens of millions of dollars just settling with the human moderators for their job-related mental health issues https://www.theverge.com/2020/5/12/21255870/facebook-content-moderator-settlement-scola-ptsd-mental-health https://www.theverge.com/2020/5/12/21255870/facebook-content... .
MattGaiser 5y ago
Manual moderation doesn't scale. It needs to be automated.
- whateveracct 5y ago
  If scales just fine. It's embarrassingly parallel even. It's not a scaling problem - it's a provisioning problem. And Facebook isn't willing to pay for it.
thoughtstheseus 5y ago
Manuel moderation is not the solution. The issue is FB (insta) is the one moderating. Give moderation powers to the users, create a market for moderation. Some people will want whitelists, others AI screening models, others “manual moderation”. Let users decide what they want.
- namlem 5y ago
  YouTube tried that and people found ways to abuse it immediately. Organized groups started going around getting chennels they don't like banned on false pretenses.
  - thoughtstheseus 5y ago
    YouTube had differentiated moderation for each user based on personal preferences set by the user? I don’t recall that.
    - namlem 5y ago
      No, they tried just one of the methods you listed. Having it be set by user preference is an interesting idea, but whatever the default setting is will be used by the overwhelming majority of users and will be quickly subverted by attackers.
      - thoughtstheseus 5y ago
        Gotcha. I agree any default setting needs to expect attackers. I’d set the default to whitelist content from Insta (so they manually approve it) and verified real people “friends” can white list additional content for you. If you want to open it up more you can and layer in content moderators for hire, AI models to try, dashboards to optimize what you want, etc.
  - biomcgary 5y ago
    The GP was probably suggesting letting bubbles of people moderate for those who opt in to the moderation. Thus, a channel would only be banned for people who are in the group. Analogously to real life, one doesn't have the option to choose all of ones neighbors, but you can choose the neighborhood you live in (assuming you have adequate wealth, redlining is illegal, etc). Moderation bubbles would enforce content rules much like an HOA enforces grass cutting, etc. I'm not sure how it would work out, but I'm curious if there are existing examples.
KaiserPro 5y ago
> This is a real band aid that will do nothing to dissuade people from continuing their abusive behaviour. Band aid, yes. The only thing that will change behaviour is consequences. I don't verbally abuse people face to face because there is a non zero chance of being punched. I am almost entirely liberated from consequences in the online world. Online I can't see the person I'm abusing, I don't see their pain, I have zero chance of empathising with them. Even verified accounts wont help, if there are no consequences to me abusing someone. The next best thing is to hellban. (in my opinion.) The problem is, doing that at scale is exceptionally hard.
- nova22033 5y ago
  I don't verbally abuse people face to face because there is a non zero chance of being punched Just so we're clear...if you live in the US, you absolutely have a right to stand on the street corner and abuse me as long as you're not likely to cause imminent lawless action...all of that without getting punched. You absolutely can call for all people of my ethnic/racial group to be deported(or worse). What you can't do is call for your followers to go kill me or burn down my house or any other lawless action. You DO NOT have a right to say those things on facebook/youtube. In case anyone was thinking of bring up the "fighting words exception", please read this https://www.thefire.org/misconceptions-about-the-fighting-words-exception/ https://www.thefire.org/misconceptions-about-the-fighting-wo...
  - amanaplanacanal 5y ago
    He wasn't talking about legal rights, he was talking about real consequences.
    - KaiserPro 5y ago
      exactly this, I'm not talking about legal consequences. I'm talking about literally being punched in the face for saying something that you'd hear a 14 year old gamer shout over the microphone. In the real world, they'd never say it to my face, because they know I might tell their parents
- 5y ago
namlem 5y ago
The aversion is due to cost. Simple as that. Manual moderation is simply too expensive.
mgraczyk 5y ago
There are literally not enough humans in the global workforce to moderate all the content that goes through Facebook's properties. Instagram alone has over 20k things posted per second.
caymanjim 5y ago
Facebook processes literally hundreds of millions of posts per day. The idea that it can be human-moderated is laughable.
- minikites 5y ago
  So we should involve no humans at all? I think many people are clamoring for something instead of the status quo of nothing.
- chrchang523 5y ago
  Not an endorsement, but you might want to take a look at the scale that human moderation is currently applied at in China.
- hodgesrm 5y ago
  If the alternative is doing nothing, then maybe we need to change the legal framework to make businesses like Facebook unsustainable.
- InitialLastName 5y ago
  Why should the size of a business be an excuse for that business to operate antisocially? We don't say "Dow chemical processes millions of gallons of waste fluid every day. The idea that they can avoid dumping most of it in a river is laughable". If Facebook can't profitably operate their business in a way that is acceptable to the society they operate within, they don't have some inherent right to a profit.
  - caymanjim 5y ago
    I didn't say it was acceptable. I hope they all implode.
- _ph_ 5y ago
  It can be. There is a button to flag posts. Moderation can prioritize by the flagging and the amount of flags a post gets. Considering how many analysis is done onto the facebook users, it should be relatively easy to come up with a scoring system. Could be as simply as how often a post by a certain user is flagged (and potentially removed by moderators). Also just randomly check posts can be effective, if consequential action is taken. Yes, this is some effort and will cost money. But we are talking about hugely large and profitable companies here. While it might not be possible to "solve" the issue entirely, it could get a lot better if done thoroughly.
strangeattractr 5y ago
Everything is a bandaid when you're policing what people say. Manual moderation is unlikely to solve the issue to people's liking either, it doesn't on here or anywhere else I see it done. Moderation on social media isn't something that can be solved satisfactorily, everyone is always unhappy with it. Take the lab leak theory with COVID, originally Facebook came under fire for allowing misinformation to spread so was blocking content which contained this claim because it was considered harmful but they reversed course once traditional media such as Vanity Fair began reporting it. Regardless of where you stand on this, you're unhappy with what they did. Even hate speech is pretty ambiguous. The issue is one of subjective interpretation. Some comments are unambiguously hateful, others are hateful to some and not to others due to their cultural context which is important when you consider that these are global platforms. I've found it much easier to not use them than worry about what's on them.
- elliekelly 5y ago
  > Manual moderation is unlikely to solve the issue to people's liking either, it doesn't on here or anywhere else I see it done. I wholeheartedly disagree. I think the human-approach of HN mods is _the_ reason this site is better and different from other social/forum sites.
  - throwaway0a5e 5y ago
    HN is a fairly ideologically homogeneous community. Moderating it is "easy mode" compared to FB or somewhere else that has a broader cross section of the population.
- jakelazaroff 5y ago
  The point of moderation is not to make everyone happy. It’s to maximize profit, which is a second order effect of the size and engagement of the user base. If assholes are being toxic on your platform, that might be good for a while — until people start leaving and the media starts reporting on it. If someone is being an asshole to other patrons in a bar, they get thrown out. No one worries about whether they’re happy with the decision. No one says “actually their comments are only subjectively asshole-ish.” We make these judgments constantly, in basically every arena in life. Why do we start wringing our hands when it comes to online platforms?
Kalium 5y ago
How many people do you think it would take to meaningfully apply manual moderation at the scale of Blizzard, Facebook, or Instagram? How many more people in support staff? How much do you think this would cost? How well do you think that cost would stack up against spending millions on half-hearted automated moderation?
13years 5y ago
The real problem is that no company is qualified to be the moral arbiter of all society. All such attempts are doomed to fail as the only success will be the furtherance of our dystopia.
- amanaplanacanal 5y ago
  They don't have to be the moral arbiter of all society, just moderate their own users. The problem is that they want all society to become their users. They are biting off more than they can chew, to the detriment of civil society.
  - krapp 5y ago
    >The problem is that they want all society to become their users. That's not a problem. Even if all society became Instagram users, Instagram couldn't act as the moral arbiters of all society, they can still only moderate their own users on their own platform. Substitute that with any platform, including Facebook and its billions of followers. Facebook isn't society. Facebook hasn't replaced or supplanted governments, doesn't control the media, doesn't burn books or dissidents, doesn't censor any of the countless alternative means of communication that exist. No matter how big Facebook gets, Facebook can only control Facebook.
    - 13years 5y ago
      Even Facebook itself doesn't have this viewpoint. They are openly acknowledging they want to influence behavior in the real world outside of the mere use of just the Facebook app.
      - krapp 5y ago
        I don't know what specifically you're referring to. A lot of companies make bold claims about wanting to "change the world" and whatnot. I'm not aware of any public Facebook policy where they acknowledge wanting to take over all governments, control all thought and communication, and act as the sole arbiters of truth and morality for society.
zucker42 5y ago
I hardly think Facebook has a deep aversion to manual moderation, considering that I've read they employ around 15,000 manual moderators.[1] However, you may still be right that this is the wrong approach in this case. [1] https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-facebook-content-moderation-mistakes-daily-report-says/?sh=4c48454154d0 https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-...

post-it 5y ago

Manual moderation scales linearly: for X posts per day, you need nX moderators, forever. Silicon Valley execs are allergic to anything that doesn't scale sub-linearly. It makes them feel like the singularity might not be around the corner.

smeyer 5y ago

>will spend millions on getting developers to make half-hearted attempts What do you think the annual cost of manual moderation would be?

hellomyguys 5y ago
Facebook can afford manual moderation. What do you think the mental health cost of waiting for AI moderation to be good enough in 111 languages that understand the nuances of speech in text/audio/video?
- smeyer 5y ago
  I'm not saying that the expense of manual moderation would excuse doing a poor job of moderating, but the top comment talked about spending "millions" on trying to automate moderation as if manual moderation wouldn't cost as much. They're certainly spending many millions on both manual and automatic moderation as it currently stands. They've spent tens of millions of dollars just settling with the human moderators for their job-related mental health issues https://www.theverge.com/2020/5/12/21255870/facebook-content-moderator-settlement-scola-ptsd-mental-health https://www.theverge.com/2020/5/12/21255870/facebook-content... .

MattGaiser 5y ago

Manual moderation doesn't scale. It needs to be automated.

whateveracct 5y ago
If scales just fine. It's embarrassingly parallel even. It's not a scaling problem - it's a provisioning problem. And Facebook isn't willing to pay for it.

thoughtstheseus 5y ago

Manuel moderation is not the solution. The issue is FB (insta) is the one moderating. Give moderation powers to the users, create a market for moderation. Some people will want whitelists, others AI screening models, others “manual moderation”. Let users decide what they want.

namlem 5y ago
YouTube tried that and people found ways to abuse it immediately. Organized groups started going around getting chennels they don't like banned on false pretenses.
- thoughtstheseus 5y ago
  YouTube had differentiated moderation for each user based on personal preferences set by the user? I don’t recall that.
  - namlem 5y ago
    No, they tried just one of the methods you listed. Having it be set by user preference is an interesting idea, but whatever the default setting is will be used by the overwhelming majority of users and will be quickly subverted by attackers.
    - thoughtstheseus 5y ago
      Gotcha. I agree any default setting needs to expect attackers. I’d set the default to whitelist content from Insta (so they manually approve it) and verified real people “friends” can white list additional content for you. If you want to open it up more you can and layer in content moderators for hire, AI models to try, dashboards to optimize what you want, etc.
- biomcgary 5y ago
  The GP was probably suggesting letting bubbles of people moderate for those who opt in to the moderation. Thus, a channel would only be banned for people who are in the group. Analogously to real life, one doesn't have the option to choose all of ones neighbors, but you can choose the neighborhood you live in (assuming you have adequate wealth, redlining is illegal, etc). Moderation bubbles would enforce content rules much like an HOA enforces grass cutting, etc. I'm not sure how it would work out, but I'm curious if there are existing examples.

KaiserPro 5y ago

> This is a real band aid that will do nothing to dissuade people from continuing their abusive behaviour. Band aid, yes. The only thing that will change behaviour is consequences. I don't verbally abuse people face to face because there is a non zero chance of being punched. I am almost entirely liberated from consequences in the online world. Online I can't see the person I'm abusing, I don't see their pain, I have zero chance of empathising with them. Even verified accounts wont help, if there are no consequences to me abusing someone. The next best thing is to hellban. (in my opinion.) The problem is, doing that at scale is exceptionally hard.

nova22033 5y ago
I don't verbally abuse people face to face because there is a non zero chance of being punched Just so we're clear...if you live in the US, you absolutely have a right to stand on the street corner and abuse me as long as you're not likely to cause imminent lawless action...all of that without getting punched. You absolutely can call for all people of my ethnic/racial group to be deported(or worse). What you can't do is call for your followers to go kill me or burn down my house or any other lawless action. You DO NOT have a right to say those things on facebook/youtube. In case anyone was thinking of bring up the "fighting words exception", please read this https://www.thefire.org/misconceptions-about-the-fighting-words-exception/ https://www.thefire.org/misconceptions-about-the-fighting-wo...
- amanaplanacanal 5y ago
  He wasn't talking about legal rights, he was talking about real consequences.
  - KaiserPro 5y ago
    exactly this, I'm not talking about legal consequences. I'm talking about literally being punched in the face for saying something that you'd hear a 14 year old gamer shout over the microphone. In the real world, they'd never say it to my face, because they know I might tell their parents
5y ago

namlem 5y ago

The aversion is due to cost. Simple as that. Manual moderation is simply too expensive.

mgraczyk 5y ago

There are literally not enough humans in the global workforce to moderate all the content that goes through Facebook's properties. Instagram alone has over 20k things posted per second.

caymanjim 5y ago

Facebook processes literally hundreds of millions of posts per day. The idea that it can be human-moderated is laughable.

minikites 5y ago
So we should involve no humans at all? I think many people are clamoring for something instead of the status quo of nothing.
chrchang523 5y ago
Not an endorsement, but you might want to take a look at the scale that human moderation is currently applied at in China.
hodgesrm 5y ago
If the alternative is doing nothing, then maybe we need to change the legal framework to make businesses like Facebook unsustainable.
InitialLastName 5y ago
Why should the size of a business be an excuse for that business to operate antisocially? We don't say "Dow chemical processes millions of gallons of waste fluid every day. The idea that they can avoid dumping most of it in a river is laughable". If Facebook can't profitably operate their business in a way that is acceptable to the society they operate within, they don't have some inherent right to a profit.
- caymanjim 5y ago
  I didn't say it was acceptable. I hope they all implode.
_ph_ 5y ago
It can be. There is a button to flag posts. Moderation can prioritize by the flagging and the amount of flags a post gets. Considering how many analysis is done onto the facebook users, it should be relatively easy to come up with a scoring system. Could be as simply as how often a post by a certain user is flagged (and potentially removed by moderators). Also just randomly check posts can be effective, if consequential action is taken. Yes, this is some effort and will cost money. But we are talking about hugely large and profitable companies here. While it might not be possible to "solve" the issue entirely, it could get a lot better if done thoroughly.

strangeattractr 5y ago

Everything is a bandaid when you're policing what people say. Manual moderation is unlikely to solve the issue to people's liking either, it doesn't on here or anywhere else I see it done. Moderation on social media isn't something that can be solved satisfactorily, everyone is always unhappy with it. Take the lab leak theory with COVID, originally Facebook came under fire for allowing misinformation to spread so was blocking content which contained this claim because it was considered harmful but they reversed course once traditional media such as Vanity Fair began reporting it. Regardless of where you stand on this, you're unhappy with what they did. Even hate speech is pretty ambiguous. The issue is one of subjective interpretation. Some comments are unambiguously hateful, others are hateful to some and not to others due to their cultural context which is important when you consider that these are global platforms. I've found it much easier to not use them than worry about what's on them.

elliekelly 5y ago
> Manual moderation is unlikely to solve the issue to people's liking either, it doesn't on here or anywhere else I see it done. I wholeheartedly disagree. I think the human-approach of HN mods is _the_ reason this site is better and different from other social/forum sites.
- throwaway0a5e 5y ago
  HN is a fairly ideologically homogeneous community. Moderating it is "easy mode" compared to FB or somewhere else that has a broader cross section of the population.
jakelazaroff 5y ago
The point of moderation is not to make everyone happy. It’s to maximize profit, which is a second order effect of the size and engagement of the user base. If assholes are being toxic on your platform, that might be good for a while — until people start leaving and the media starts reporting on it. If someone is being an asshole to other patrons in a bar, they get thrown out. No one worries about whether they’re happy with the decision. No one says “actually their comments are only subjectively asshole-ish.” We make these judgments constantly, in basically every arena in life. Why do we start wringing our hands when it comes to online platforms?

Kalium 5y ago

How many people do you think it would take to meaningfully apply manual moderation at the scale of Blizzard, Facebook, or Instagram? How many more people in support staff? How much do you think this would cost? How well do you think that cost would stack up against spending millions on half-hearted automated moderation?

13years 5y ago

The real problem is that no company is qualified to be the moral arbiter of all society. All such attempts are doomed to fail as the only success will be the furtherance of our dystopia.

amanaplanacanal 5y ago
They don't have to be the moral arbiter of all society, just moderate their own users. The problem is that they want all society to become their users. They are biting off more than they can chew, to the detriment of civil society.
- krapp 5y ago
  >The problem is that they want all society to become their users. That's not a problem. Even if all society became Instagram users, Instagram couldn't act as the moral arbiters of all society, they can still only moderate their own users on their own platform. Substitute that with any platform, including Facebook and its billions of followers. Facebook isn't society. Facebook hasn't replaced or supplanted governments, doesn't control the media, doesn't burn books or dissidents, doesn't censor any of the countless alternative means of communication that exist. No matter how big Facebook gets, Facebook can only control Facebook.
  - 13years 5y ago
    Even Facebook itself doesn't have this viewpoint. They are openly acknowledging they want to influence behavior in the real world outside of the mere use of just the Facebook app.
    - krapp 5y ago
      I don't know what specifically you're referring to. A lot of companies make bold claims about wanting to "change the world" and whatnot. I'm not aware of any public Facebook policy where they acknowledge wanting to take over all governments, control all thought and communication, and act as the sole arbiters of truth and morality for society.

zucker42 5y ago

I hardly think Facebook has a deep aversion to manual moderation, considering that I've read they employ around 15,000 manual moderators.[1] However, you may still be right that this is the wrong approach in this case. [1] https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-facebook-content-moderation-mistakes-daily-report-says/?sh=4c48454154d0 https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-...