10 ms·

At Facebook's scale, there is not a good way to do it. Even if they were able to scale it, it would not be good enough anyway. You can automate it, but clearly

At Facebook's scale, there is not a good way to do it. Even if they were able to scale it, it would not be good enough anyway. You can automate it, but clearly that doesn't catch everything and automated systems can result in just as much bias as humans doing the job. You can hire humans to do it and make them work fast, because the amount of posts per day is extremely high. You can use some automation to help them, but that will combine the problems of human moderation and automated moderation. You can hire lots of humans to work at a normal pace and give them psychological counseling, and spend so much on their salaries that you'll go out of business, and even then they won't catch everything and will be frequently accused of bias. It's a no-win situation for Facebook because even if they were able to objectively and fairly moderate every single post, they would be accused of bias by politically motivated people anyway. This is not a technology problem, it's a social and political problem and Facebook will never be able to please everyone. Twitter is in the same boat.

tyingq 9y ago
>At Facebook's scale, there is not a good way to do it. Even if they were able to scale it, it would not be good enough anyway Facebook is picking the quota for decisions per hour or whatever, and choosing whatever help is available to reviewers. Certainly they have some culpability for the outcome.
- twblalock 9y ago
  I wasn't trying to apportion blame, because I don't see how it's useful to do so when there is no possibility of a good outcome. If Facebook adjusts the quotas, they will just be doing a more effective job at pursuing an unworkable goal.
mschuster91 9y ago
> You can hire lots of humans to work at a normal pace and give them psychological counseling, and spend so much on their salaries that you'll go out of business, and even then they won't catch everything and will be frequently accused of bias. Then you should not fucking be in the business at all! What FB does is externalizing the cost onto societies and skimming the profit! Cost here being two things: first the cost of mentally fucking up the moderators (so an extensive amount of psychological healthcare has to be shouldered by society) and second the cost to society that arises from letting Nazis and jihadists spread their propaganda (including the cost from real-world violence enabled by said propaganda).
- twblalock 9y ago
  Any online open forum would have the same problems no matter how hard they tried to prevent it. We can either accept that some level of hate speech is inevitably going to slip past the moderators, or we can give up on having open forums altogether. There is no perfect world with perfect moderation of online content. I wish Facebook had the courage to say this themselves.
  - ben_w 9y ago
    Is that provably inevitable for all online forms, or is “nobody knows how to yet”? Governments act like all tech problems are the latter, but I would like to know if this is like encryption that only friends can break (never), or like self driving cars (most people call it impossible until five years after it’s been demonstrated).
    - dfox 9y ago
      For me moderating what amounts to open communication platform falls into third category: it is impossible and you should not try to do that. Especially in FB's case where it is quite obvious that various opposing "hate groups" have learned how to game the system, and also how to present results of that such that it supports their rhetoric.
      - jacquesm 9y ago
        > it is impossible and you should not try to do that. No, it is merely extremely expensive. But that's something that you need to tackle head on when your business model is to provide a service for free to a large chunk of the planet's population. The 'free' bit doesn't absolve you from your responsibility as an operator.
        jjeaff 9y ago
        Seems like their recent move to get news off the feeds and focus on family and friends should solve the content issue. If someone is posting or liking hate speech or offensive content, I can just block them. I wouldn't add someone as a Facebook friend that would do that kind of thing anyway.
maxxxxx 9y ago
"You can hire lots of humans to work at a normal pace and give them psychological counseling, and spend so much on their salaries that you'll go out of business, " They could hire 10000 people at 100k each. This would cost 1 billion per year making 3 billion every quarter.
- pdpi 9y ago
  And how much infrastructure would be required to keep those 10000 going? How much office space, IT equipment, managers, etc. You’re suggesting that the company should grow its headcount by around 40%!
- jjeaff 9y ago
  Plus, somehow in all this discussion, people seem to be accepting the idea on its face that somehow psychological counseling would fix said issues. Maybe in some cases it would, but we are far from being able to cure many psychological problems consistently.
ams6110 9y ago
Why do they need to moderate every post. The problem posts will be coming from a minority of accounts. Ban them on first offense, or maybe allow one warning, and people will figure out pretty quickly that Facebook is not the place to post that sort of thing.
- ben_w 9y ago
  1) Because you don’t know who the offenders are until you show their content to a moderator. 2) Because zero-tolerance tends to result in over-blocking, and they already have a problem with banning content that shouldn’t be banned.
  - ams6110 9y ago
    I think Facebook knows enough about its users to precisely identify the accounts that are likely to be offenders.
    - deleted 9y ago
      [deleted]
    - ben_w 9y ago
      Perhaps, but to the extent that it does that’s a combination of moderation and machine learning — exactly what post several layers above was saying is insufficient.
- twblalock 9y ago
  Bans are easy to circumvent, and the worst trolls tend to be technically savvy enough to create new accounts. They'll do it forever. They'll hide behind VPNs and anonymizing proxies if that's what it takes. Also, what do you do if the hate speech is posted or linked to by a notable public figure, such as the President of the United States or a member of Congress?
  - Spivak 9y ago
    So? The system doesn't have to be perfect, it only needs to make trolling time consuming enough to reduce it to tolerable levels. > hate speech is posted or linked to by a notable public figure Here's how I would approach it. Accounts that are tied to real identities have a much higher threshold for bans -- public figure or not. Sans a credible threat of harm to specific individuals they can largely say whatever they would like, hateful or not. When pseudoanonymous accounts get flagged their choice is to attach a publicly visible identity to the account or be banned.
tw04 9y ago
>At Facebook's scale, there is not a good way to do it. Even if they were able to scale it, it would not be good enough anyway. Then maybe we need to place limits on their scale if they can't handle the problems their scale has created.
jancsika 9y ago
> At Facebook's scale, there is not a good way to do it. That seems quite unpersuasive, especially with no citations given. Just off the top of my head-- email exists at roughly the same scale as Facebook. Yet those shady "fwd:fwd:fwd" stories do not propagate over email at anywhere near the velocity of the propaganda campaigns which saturate Facebook. And at the end of the day those "fwd:fwd:fwd" stories don't reach nearly the number of end users as the propaganda campaigns do because of the critical mass of email users unwilling to forward that crap on to their contacts.
- dna_polymerase 9y ago
  > That seems quite unpersuasive, especially with no citations given. Well there is no good approach to censorship, it's censorship after all, you will always piss people off. People need to grow up, there is stuff out there that may dicomfort your world view but get used to it. Also only because you are deleting comments on facebook does not mean - at all - that you are tackling the cause, you just swipe the dust under the carpet.
- noway421 9y ago
  Well, maybe because email is by default designed to be 1 to 1, and 1 to many is considered to be borderline spam? While facebook is always 1 to many.
- TeMPOraL 9y ago
  > Yet those shady "fwd:fwd:fwd" stories do not propagate over email at anywhere near the velocity of the propaganda campaigns which saturate Facebook. Because people no longer use e-mail for private communication. I remember the times of "fwd:fwd:fwd" stories - they were popular before IMs and social media. And let's not forget the ultimate, age-old source of "fake news" - a real-life, face-to-face conversation with a friend or family member that starts with "did you hear that $<fake bullshit news of the day>".
mirimir 9y ago
So why not provide multiple Facebook views? That is, put everything in the same databases. Automate creation of flags for posts and accounts. Then let users set moderation level as they desire. Some would want everything. The rest would specify what they don't want to see. And they'd be warned about false positives. Some governments would limit options, I'm sure. But we already have that, so hey.
- askafriend 9y ago
  That's exactly what they do...the algorithmic feed is basically what you described except instead of it being very explicit upfront, the signals are a combination of explicit and implicit and the resulting view is always evolving independently for each person.
  - mirimir 9y ago
    OK, I get that. But there's no option for zero filtering. I'm thinking of the old talk.masked Tor onion service. Except near the end, there was zero censorship. But users could set their own filters.
    - askafriend 9y ago
      I kinda get what you're saying but in practice zero filtering would be a horrible horrible user experience. The content velocity on Facebook is so very high that unless you filter the feed in some way - it becomes a different product entirely, or perhaps even unusable. And in the case of "Let the user choose the filters however they want", the business portion of the company takes on a massive risk. Remember this isn't a charity. Facebook wants to be able to make a lot of money and exist independently as an organization. Furthermore, they want to be able to predictably grow the business. The reason I push back against your suggestions is because I see them often - solutions that swing too far into the realm of idealism and entitlement. Facebook is a business, so you have to be able to provide solutions that recognize the importance of the business side of things.
      - 9y ago
itronitron 9y ago
There are certain videos that contain extremely graphic violent content and could easily be automatically detected and blocked because they are reposts of the same video. That is a simple problem for which Facebook should be able to apply existing technology. There is a lot Facebook can do, but they are not, because they are lazy.
- bertil 9y ago
  > apply existing technology. There is a lot Facebook can do, but they are not, because they are lazy. Mmmh. You do realise that the state of the art technology that you refer to was developed by Facebook, right? I mean, there are no public communications around whether and which type of image recognition is used to detect harmful content at scale for obvious reasons but… take a wild guess. Do you _really_ think engineers at Facebook who love to solve problems at scale would prefer to have an army of contractors handle something hurtful if there was a solution involving smart code and a battery of servers?
  - itronitron 9y ago
    So, if FB has the state of the art and they are still employing armies of contractors then maybe they are just using said contractors to create their training data set.
    - bertil 9y ago
      The contractors’ feedback is certainly considered a high-signal dataset. I am not directly familiar with that aspect of the company (I worked on things somehow related but not this) but I suspect there are several ways of training and assessing estimations that certain content breaks certain rules (gore, direct threat, child endangerment, etc.). There could be challenges (legal, quality issues) of using automated decision to block or not block different types of flagged content. One thing that, as a data scientist, I noticed: none of those reports complains about seeing the same image repeatedly. I understand that posters are often tempted to spread new content, so I would expect there to be many repeats. It could be because they see so many similar and shocking things they don’t recognise. Or companies could use techniques to pool together content that is identical or similar enough.

tyingq 9y ago

>At Facebook's scale, there is not a good way to do it. Even if they were able to scale it, it would not be good enough anyway Facebook is picking the quota for decisions per hour or whatever, and choosing whatever help is available to reviewers. Certainly they have some culpability for the outcome.

twblalock 9y ago
I wasn't trying to apportion blame, because I don't see how it's useful to do so when there is no possibility of a good outcome. If Facebook adjusts the quotas, they will just be doing a more effective job at pursuing an unworkable goal.

mschuster91 9y ago

> You can hire lots of humans to work at a normal pace and give them psychological counseling, and spend so much on their salaries that you'll go out of business, and even then they won't catch everything and will be frequently accused of bias. Then you should not fucking be in the business at all! What FB does is externalizing the cost onto societies and skimming the profit! Cost here being two things: first the cost of mentally fucking up the moderators (so an extensive amount of psychological healthcare has to be shouldered by society) and second the cost to society that arises from letting Nazis and jihadists spread their propaganda (including the cost from real-world violence enabled by said propaganda).

twblalock 9y ago
Any online open forum would have the same problems no matter how hard they tried to prevent it. We can either accept that some level of hate speech is inevitably going to slip past the moderators, or we can give up on having open forums altogether. There is no perfect world with perfect moderation of online content. I wish Facebook had the courage to say this themselves.
- ben_w 9y ago
  Is that provably inevitable for all online forms, or is “nobody knows how to yet”? Governments act like all tech problems are the latter, but I would like to know if this is like encryption that only friends can break (never), or like self driving cars (most people call it impossible until five years after it’s been demonstrated).
  - dfox 9y ago
    For me moderating what amounts to open communication platform falls into third category: it is impossible and you should not try to do that. Especially in FB's case where it is quite obvious that various opposing "hate groups" have learned how to game the system, and also how to present results of that such that it supports their rhetoric.
    - jacquesm 9y ago
      > it is impossible and you should not try to do that. No, it is merely extremely expensive. But that's something that you need to tackle head on when your business model is to provide a service for free to a large chunk of the planet's population. The 'free' bit doesn't absolve you from your responsibility as an operator.
      - jjeaff 9y ago
        Seems like their recent move to get news off the feeds and focus on family and friends should solve the content issue. If someone is posting or liking hate speech or offensive content, I can just block them. I wouldn't add someone as a Facebook friend that would do that kind of thing anyway.

maxxxxx 9y ago

"You can hire lots of humans to work at a normal pace and give them psychological counseling, and spend so much on their salaries that you'll go out of business, " They could hire 10000 people at 100k each. This would cost 1 billion per year making 3 billion every quarter.

pdpi 9y ago
And how much infrastructure would be required to keep those 10000 going? How much office space, IT equipment, managers, etc. You’re suggesting that the company should grow its headcount by around 40%!
jjeaff 9y ago
Plus, somehow in all this discussion, people seem to be accepting the idea on its face that somehow psychological counseling would fix said issues. Maybe in some cases it would, but we are far from being able to cure many psychological problems consistently.

ams6110 9y ago

Why do they need to moderate every post. The problem posts will be coming from a minority of accounts. Ban them on first offense, or maybe allow one warning, and people will figure out pretty quickly that Facebook is not the place to post that sort of thing.

ben_w 9y ago
1) Because you don’t know who the offenders are until you show their content to a moderator. 2) Because zero-tolerance tends to result in over-blocking, and they already have a problem with banning content that shouldn’t be banned.
- ams6110 9y ago
  I think Facebook knows enough about its users to precisely identify the accounts that are likely to be offenders.
  - deleted 9y ago
    [deleted]
  - ben_w 9y ago
    Perhaps, but to the extent that it does that’s a combination of moderation and machine learning — exactly what post several layers above was saying is insufficient.
twblalock 9y ago
Bans are easy to circumvent, and the worst trolls tend to be technically savvy enough to create new accounts. They'll do it forever. They'll hide behind VPNs and anonymizing proxies if that's what it takes. Also, what do you do if the hate speech is posted or linked to by a notable public figure, such as the President of the United States or a member of Congress?
- Spivak 9y ago
  So? The system doesn't have to be perfect, it only needs to make trolling time consuming enough to reduce it to tolerable levels. > hate speech is posted or linked to by a notable public figure Here's how I would approach it. Accounts that are tied to real identities have a much higher threshold for bans -- public figure or not. Sans a credible threat of harm to specific individuals they can largely say whatever they would like, hateful or not. When pseudoanonymous accounts get flagged their choice is to attach a publicly visible identity to the account or be banned.

tw04 9y ago

>At Facebook's scale, there is not a good way to do it. Even if they were able to scale it, it would not be good enough anyway. Then maybe we need to place limits on their scale if they can't handle the problems their scale has created.

jancsika 9y ago

> At Facebook's scale, there is not a good way to do it. That seems quite unpersuasive, especially with no citations given. Just off the top of my head-- email exists at roughly the same scale as Facebook. Yet those shady "fwd:fwd:fwd" stories do not propagate over email at anywhere near the velocity of the propaganda campaigns which saturate Facebook. And at the end of the day those "fwd:fwd:fwd" stories don't reach nearly the number of end users as the propaganda campaigns do because of the critical mass of email users unwilling to forward that crap on to their contacts.

dna_polymerase 9y ago
> That seems quite unpersuasive, especially with no citations given. Well there is no good approach to censorship, it's censorship after all, you will always piss people off. People need to grow up, there is stuff out there that may dicomfort your world view but get used to it. Also only because you are deleting comments on facebook does not mean - at all - that you are tackling the cause, you just swipe the dust under the carpet.
noway421 9y ago
Well, maybe because email is by default designed to be 1 to 1, and 1 to many is considered to be borderline spam? While facebook is always 1 to many.
TeMPOraL 9y ago
> Yet those shady "fwd:fwd:fwd" stories do not propagate over email at anywhere near the velocity of the propaganda campaigns which saturate Facebook. Because people no longer use e-mail for private communication. I remember the times of "fwd:fwd:fwd" stories - they were popular before IMs and social media. And let's not forget the ultimate, age-old source of "fake news" - a real-life, face-to-face conversation with a friend or family member that starts with "did you hear that $<fake bullshit news of the day>".

mirimir 9y ago

So why not provide multiple Facebook views? That is, put everything in the same databases. Automate creation of flags for posts and accounts. Then let users set moderation level as they desire. Some would want everything. The rest would specify what they don't want to see. And they'd be warned about false positives. Some governments would limit options, I'm sure. But we already have that, so hey.

askafriend 9y ago
That's exactly what they do...the algorithmic feed is basically what you described except instead of it being very explicit upfront, the signals are a combination of explicit and implicit and the resulting view is always evolving independently for each person.
- mirimir 9y ago
  OK, I get that. But there's no option for zero filtering. I'm thinking of the old talk.masked Tor onion service. Except near the end, there was zero censorship. But users could set their own filters.
  - askafriend 9y ago
    I kinda get what you're saying but in practice zero filtering would be a horrible horrible user experience. The content velocity on Facebook is so very high that unless you filter the feed in some way - it becomes a different product entirely, or perhaps even unusable. And in the case of "Let the user choose the filters however they want", the business portion of the company takes on a massive risk. Remember this isn't a charity. Facebook wants to be able to make a lot of money and exist independently as an organization. Furthermore, they want to be able to predictably grow the business. The reason I push back against your suggestions is because I see them often - solutions that swing too far into the realm of idealism and entitlement. Facebook is a business, so you have to be able to provide solutions that recognize the importance of the business side of things.
    - 9y ago

itronitron 9y ago

There are certain videos that contain extremely graphic violent content and could easily be automatically detected and blocked because they are reposts of the same video. That is a simple problem for which Facebook should be able to apply existing technology. There is a lot Facebook can do, but they are not, because they are lazy.

bertil 9y ago
> apply existing technology. There is a lot Facebook can do, but they are not, because they are lazy. Mmmh. You do realise that the state of the art technology that you refer to was developed by Facebook, right? I mean, there are no public communications around whether and which type of image recognition is used to detect harmful content at scale for obvious reasons but… take a wild guess. Do you _really_ think engineers at Facebook who love to solve problems at scale would prefer to have an army of contractors handle something hurtful if there was a solution involving smart code and a battery of servers?
- itronitron 9y ago
  So, if FB has the state of the art and they are still employing armies of contractors then maybe they are just using said contractors to create their training data set.
  - bertil 9y ago
    The contractors’ feedback is certainly considered a high-signal dataset. I am not directly familiar with that aspect of the company (I worked on things somehow related but not this) but I suspect there are several ways of training and assessing estimations that certain content breaks certain rules (gore, direct threat, child endangerment, etc.). There could be challenges (legal, quality issues) of using automated decision to block or not block different types of flagged content. One thing that, as a data scientist, I noticed: none of those reports complains about seeing the same image repeatedly. I understand that posters are often tempted to spread new content, so I would expect there to be many repeats. It could be because they see so many similar and shocking things they don’t recognise. Or companies could use techniques to pool together content that is identical or similar enough.