9 ms·

I think FB has a fundamental issue here, which is that: 1) they don't feel they can NOT police content 2) they are wayyyyy too big to police content manually

I think FB has a fundamental issue here, which is that: 1) they don't feel they can NOT police content 2) they are wayyyyy too big to police content manually 3) any AI they train to police content, is not going to be able to state clearly why a given post/account was banned, and anyway... 4) if you state objective, clear rules that are automated, it will be pretty easy to game them (e.g. using euphemisms or code words for objectionable content) I don't see how FB can get out of this situation. Any social network that is as large as it is, which relies on advertising (and thus cannot be as content-neutral as e.g. phone companies are regarding what you talk about on the phone), is going to face the same situation. Of course, if FB ends up getting way smaller as a result, that would be fine by me. But I'm not holding my breath...

falcolas 6y ago
3 and 4 are easy - have a manual review of automatically flagged content. You get clear reasons, better resolution of false positives, and it’s not simple to game. And, in another plus, it can feed back into the AI training.
- g_p 6y ago
  I think part of the problem at FB's scale is getting any level of consistency across human moderation. If you ask 10 good, experienced, human moderators whether a controversial post is allowed, you'll probably get at least 15 different answers and justifications. FB would probably find it harder to train from this, as each moderator will be using different justifications, and reaching different decisions. At FB's scale, they want something cheap and straightforward which they can implement automatically and go Kafka on the lack of appeals. The reality people need to wake up to is that when a business is at FB scale, people who fall through the cracks due to a broken algorithm simply don't matter. Google doesn't care if you get locked out of your Gmail, in the same way FB doesn't care if you get spuriously banned. When users place more value on a service than the service places on them as a customer, it feels like a market failure scenario that leads to these kinds of outcomes. When the go-to price is $0 though, it isn't clear how you resolve this.
  - samvher 6y ago
    Any justice system also has a variety of different judges. Of course in an ideal world there would be perfect consistency, but imperfect consistency would still be a lot better than just letting an algorithm decide. Of course they want something cheap and straightforward, so that's where regulation should step in to make them go against their own interests, in the interest of population at large.
- KineticLensman 6y ago
  > 3 and 4 are easy But 2 (scale) makes this solution to 3 and 4 expensive / impractical.
  - samvher 6y ago
    Facebook has money, and automated filtering of content (rather than manually scanning everything) can make it scalable. Given their monopoly it should be possible to hold them accountable / there should be transparency.
  - Delk 6y ago
    While this certainly makes it much less likely that costly massive human moderation would actually be implemented, I don't think that really qualifies as an argument for why it shouldn't be done. If you think rossdavidh is right about those issues Facebook is having (and I think he's right), then that's the reality in which the company has to operate. If a business model leads to infeasible costs due to fundamental issues, maybe that business model shouldn't exist as such. Obviously Facebook (or any other company) doesn't want to do anything that significantly increases their costs without providing revenue if they can avoid it, but that doesn't mean it shouldn't be expected of them.
- Aunche 6y ago
  Facebook has nothing to gain from transparency. Inevitably, they are going to be inconsistent when it comes to edge cases, which will be used against them.
fiddlerwoaroof 6y ago
Or, perhaps, these problems are reasons why large, centralized social networks shouldn’t be allowed to exist.
- root_axis 6y ago
  They exist because they're popular, you can't dictate what is popular among the masses without authoritarianism.
  - anp 6y ago
    They exist because they’re popular and because the currently successful ones acquired or strangled all of their smaller meaningful competitors using investment capital and money from first mover advantages.
    - root_axis 6y ago
      Can you give some examples?
      - fakedang 6y ago
        Haha, are you serious? Instagram, WhatsApp, Snapchat... Instagram's Systrom explicitly said that he sold because otherwise Facebook would snuff him away.
        root_axis 6y ago
        Facebook never acquired Snapchat so I'm not sure what you're talking about there. Instagram and WhatsApp have continued to thrive since their acquisition, so from a user perspective nothing really changed, it's not like you need a FB account to use insta or whatsapp.
        fakedang 6y ago
        GP: > They exist because they’re popular and because the currently successful ones acquired or strangled all of their smaller meaningful competitors using investment capital and money from first mover advantages. Facebook literally copied Snapchat features into Instagram rightaway - Systrom even openly admitted it. That fits the bill of "strangled". Snapchat is in stagnation now, because everything it has is there on Insta. Instagram and WhatsApp thrived only because Facebook acquired them, else both companies new they would be smothered to death. That pretty much fits the bill of "acquired... Using investment capital and money from first mover advantages". They don't need you to use your FB account because they cross share data anyways. It's not like it's a painstaking operation to link different accounts, once you have all sorts of device data (which Facebook had, even on Apple devices up until recently).
inglor_cz 6y ago
> 2) they are wayyyyy too big to police content manually To be more exact: they do not want to spend money on competent policing. They probably have capability to employ ten times as many human moderators as they do now, thus reducing the blatant error rate (at least in the case of highly influential pages and people), but that would eat into their profits.
- jl6 6y ago
  Assuming a user:moderator ratio of 10,000:1, they would need to hire 200,000 moderators to cover their 2 billion users. Then at that scale, they’d need to hire thousands of meta-moderators and meta-meta-moderators too.
  - ivalm 6y ago
    If meta moderators are at the same scale then that’s only 200 meta moderators and only 1 meta meta. The base of a log scaling pyramid is approximately as large as the entire pyramid.
    - justinclift 6y ago
      Wouldn't that be 20 meta moderators (10k:1 scale), rather than 200?
      - ivalm 6y ago
        You’re right, apologies!
  - mschuster91 6y ago
    Where's the problem in that? FB makes 18.5 billion $ a year in profit. 250k employees @ 30k would work out to 7.5 billion $ in cost a year, it would hardly leave them bankrupt. Also, Facebook could offset a part of that cost by charging users a token amount - say, five dollars or something like that.
    - 6y ago
theknocker 6y ago
If only someone had figured out hundreds of years ago that policing speech is the purview of idiots and the dictators who refuse to educate them.
mschuster91 6y ago
> 2) they are wayyyyy too big to police content manually Oh hell no they aren't too big, they just don't want to reduce their profits from the current 18.5 billion US $ (per https://www.statista.com/statistics/277229/facebooks-annual-revenue-and-net-income/ https://www.statista.com/statistics/277229/facebooks-annual-...). If Facebook wanted, they could massively ramp up staffing, hire them themselves, and pay them a livable wage instead of outsourcing the cost that their business occurs on society (who has to pick up the slack from conspiracy myths, fascism and other shit), on the users who are mistakenly banned and lose their primary contact e.g. to overseas family, and on the moderation staff that is hired via third parties and exploited.
- rossdavidh 6y ago
  I would shed no tears if FB profits went down, but...I think you're underestimating how big they are. Two BILLION users, in ~100 countries (each with its own standards on what is acceptable, legal, repulsive, etc.). If you had one moderator per 100 users, that would 20 million moderators. I don't think they could hire 20 million high quality moderators (there's only one dang, and we're not giving him up), and certainly not with 18.5 billion.
simple_phrases 6y ago
They certainly generate enough revenue to moderate content manually.
- ben509 6y ago
  But they may not be able to do it well enough that it will be notably better than an AI. Scaling manual labor can't work indefinitely due to the scarcity of talent. Keep in mind that while a human being is very good at categorizing things, they're terrible at doing it over and over again, so the scarce talents are likely consistency and conscientiousness. You should expect that, overall, the (n+1)th person you hire is marginally less talented than the nth. And while people are constantly entering and exiting the labor pool, this is negated by the fact that you're trying to expand your share of it. So while there's a lot of randomness in recruiting, if you enforce any kind of standard, you'll hit a point where not enough candidates meet that standard, so you have to drop it. Once it is below the threshold of "better than AI" you have no reason to hire more people. And you'll try to expand your eligible pool by raising wages, but everyone else wants consistent, conscientious people, too, so you're liable to run into the economic constraint: the amount you have to pay someone will exceed the value they're adding as a moderator.
jancsika 6y ago
> 1) they don't feel they can NOT police content True. But we should be explicit here-- FB isn't some neat, nerdy social graph software that just happened to get really popular, like git for regular people or something. They have a business model to shake up their social graph to maximize engagement, and the implementation achieves this by keeping everyone maximally mad as hell at each other. It spreads conspiracy theories efficiently. It and its users are easily exploited, and they continue to be exploited by any nation-state, nefarious actors, or even just some dude looking to make a buck spreading fake news. That all feeds into the same destructive behaviors for the users as the system desperately tries to keep them engaging more and more. Facebook's own software and UI exacerbates these problems. With that starting point in mind, I agree with your 1, 2, 3, and 4 above. But this is less like email having to deal with new sets of problems as it scaled to the entire world. It's more like a drug cartel trying to figure out the maximum amount of their fundamentally destructive product they can sell before the government finally decides to comes in and break up their enterprise.
- kangaroozach 6y ago
  Censoring content rather than refuting its claims is counterproductive. If skeptics are not able to be argumentatively refute a claim, they will not succeed in changing minds. The current strategy of trying to silence view points will eventually fail miserably. The issue is that when a platform outsources its censorship (“moderation”) to a third party that it still controls, rather than a third party that is actually perceived to be neutral by most people, then you create distrust which further fuels conspiracy theories. Also a conspiracy theory may actually be true. Jeffrey Epstein was not discussed during the 2016 election aside from a few fringe articles. It was too toxic for mainstream to touch. And everyone in Hollywood knew about Harvey Weinstein and even joked about it. These topics were covered up only for so long. Today there are lots of insane and farfetched theories being floated. But the theories being censored today that are indeed based in reality, will eventually become common knowledge and the censors will be implicated in the coverup thereby harming their trust. Discussion, debate, argument, evidence, analysis lead us towards truth. Mute only appears to work and can be a useful political tool for only so long. Today our public forum is virtual, but still a public forum. Trump wasn’t allowed to block certain followers because of their right to be in the forum and hear from their president. So the question is whether governments will acknowledge this fact and limit the power of our big tech group communication platforms and enforce transparency and non-biased appeals process. Or will they cede total control of our public forums to Big Tech out of short term political convenience.
xerxespoy 6y ago
They can solve this issue by adequately providing for self-moderation. Avoiding this approach reveals an intention to impose. Users leave once they realise the violation.
dustingetz 6y ago
It's inherent to centralized power that opposing interests will attack it. Principle agent problem strikes again
Balgair 6y ago
Thank you for stating these points. I believe that FB is going to be cut up due to these issues and the interplay with politics, just like Bell was. Per my views on the situation, Zuck is cooked. When he went before congress to testify, he was being given the classic robber-baron deal that the US has used a few times now. As an analogy to actual robber-baron, at first when the railroads were new, Congress didn't care much about them. Then when all the businesses and politicians started to use them, they started to notice that the railroads were very important. When beef suspiciously didn't get to Chicago on time, or when trains delayed representatives to important votes up in Albany, then the politicians got legislative. A similar scaling of importance has occurred with big tech. The US then typically takes these newly vital sectors and then gives them the deal: Fix it or face jail time. Compliance means that your family for countless generations is never poor again. You go kite-surfing for the rest of your days. Congress works with you to write the laws in such a way that you get a monopoly on the sector with a nice 3% above inflation growth baked in. Congress and the US get a sector that isn't scammy and just works (albeit poorly at times). Congress gets you to testify for a few days, ensures that you have lied to congress in that time, and holds that jail time over you as leverage to comply. Zuck was given this deal. Per the points OP outlines, Zuck cannot follow through on his end of the bargain. He cannot get FB to work the way Congress is demanding that he do. Zuck may not be facing jail time (yet), but may when he gets in front of Congress again. After Dec. 37th / Jan. 6th (because 2020 never ends), Congress likely sees Big Tech as a direct personal threat (not existential) and is likely to move directly against that threat with due haste.
- rossdavidh 6y ago
  While I think Mark Zuckerberg is unlikely to face jail time, there are some points to your post I agree with. Especially the analogy with railroads. The other point I would add, is that it's not just Congress. I have heard government leaders in Mexico, Poland, Hungary, and Turkey all voicing similar concerns, and in some cases actually passing legislation already. I think more will come.
lamontcg 6y ago
They need to be forced, by government regulations, to have to talk to people who have had their accounts banned and to provide them some kind of legal recourse. The law needs to recognize that for massive social networks like Facebook and Twitter and for the Apple and Google App stores that people are actually entitled to those services, and have a right to use them, and shouldn't be kicked off of them without any form of recourse. They've grown too big to just consider them like the corner coffee shop and allow them to kick anyone off of their platform for whatever reasons they cook up. And that means that they will need to staff up and that will come out of their profit margins, and that's okay. That's why they need to be forced to do it by regulations.
Thorentis 6y ago
Which is why social media networks should make the far easier rule that they will only ban illegal content. If people want something censored, make it illegal. If people don't like they things are being censored, take it up with their local representative.

falcolas 6y ago

3 and 4 are easy - have a manual review of automatically flagged content. You get clear reasons, better resolution of false positives, and it’s not simple to game. And, in another plus, it can feed back into the AI training.

g_p 6y ago
I think part of the problem at FB's scale is getting any level of consistency across human moderation. If you ask 10 good, experienced, human moderators whether a controversial post is allowed, you'll probably get at least 15 different answers and justifications. FB would probably find it harder to train from this, as each moderator will be using different justifications, and reaching different decisions. At FB's scale, they want something cheap and straightforward which they can implement automatically and go Kafka on the lack of appeals. The reality people need to wake up to is that when a business is at FB scale, people who fall through the cracks due to a broken algorithm simply don't matter. Google doesn't care if you get locked out of your Gmail, in the same way FB doesn't care if you get spuriously banned. When users place more value on a service than the service places on them as a customer, it feels like a market failure scenario that leads to these kinds of outcomes. When the go-to price is $0 though, it isn't clear how you resolve this.
- samvher 6y ago
  Any justice system also has a variety of different judges. Of course in an ideal world there would be perfect consistency, but imperfect consistency would still be a lot better than just letting an algorithm decide. Of course they want something cheap and straightforward, so that's where regulation should step in to make them go against their own interests, in the interest of population at large.
KineticLensman 6y ago
> 3 and 4 are easy But 2 (scale) makes this solution to 3 and 4 expensive / impractical.
- samvher 6y ago
  Facebook has money, and automated filtering of content (rather than manually scanning everything) can make it scalable. Given their monopoly it should be possible to hold them accountable / there should be transparency.
- Delk 6y ago
  While this certainly makes it much less likely that costly massive human moderation would actually be implemented, I don't think that really qualifies as an argument for why it shouldn't be done. If you think rossdavidh is right about those issues Facebook is having (and I think he's right), then that's the reality in which the company has to operate. If a business model leads to infeasible costs due to fundamental issues, maybe that business model shouldn't exist as such. Obviously Facebook (or any other company) doesn't want to do anything that significantly increases their costs without providing revenue if they can avoid it, but that doesn't mean it shouldn't be expected of them.
Aunche 6y ago
Facebook has nothing to gain from transparency. Inevitably, they are going to be inconsistent when it comes to edge cases, which will be used against them.

fiddlerwoaroof 6y ago

Or, perhaps, these problems are reasons why large, centralized social networks shouldn’t be allowed to exist.

root_axis 6y ago
They exist because they're popular, you can't dictate what is popular among the masses without authoritarianism.
- anp 6y ago
  They exist because they’re popular and because the currently successful ones acquired or strangled all of their smaller meaningful competitors using investment capital and money from first mover advantages.
  - root_axis 6y ago
    Can you give some examples?
    - fakedang 6y ago
      Haha, are you serious? Instagram, WhatsApp, Snapchat... Instagram's Systrom explicitly said that he sold because otherwise Facebook would snuff him away.
      - root_axis 6y ago
        Facebook never acquired Snapchat so I'm not sure what you're talking about there. Instagram and WhatsApp have continued to thrive since their acquisition, so from a user perspective nothing really changed, it's not like you need a FB account to use insta or whatsapp.
        fakedang 6y ago
        GP: > They exist because they’re popular and because the currently successful ones acquired or strangled all of their smaller meaningful competitors using investment capital and money from first mover advantages. Facebook literally copied Snapchat features into Instagram rightaway - Systrom even openly admitted it. That fits the bill of "strangled". Snapchat is in stagnation now, because everything it has is there on Insta. Instagram and WhatsApp thrived only because Facebook acquired them, else both companies new they would be smothered to death. That pretty much fits the bill of "acquired... Using investment capital and money from first mover advantages". They don't need you to use your FB account because they cross share data anyways. It's not like it's a painstaking operation to link different accounts, once you have all sorts of device data (which Facebook had, even on Apple devices up until recently).

inglor_cz 6y ago

> 2) they are wayyyyy too big to police content manually To be more exact: they do not want to spend money on competent policing. They probably have capability to employ ten times as many human moderators as they do now, thus reducing the blatant error rate (at least in the case of highly influential pages and people), but that would eat into their profits.

jl6 6y ago
Assuming a user:moderator ratio of 10,000:1, they would need to hire 200,000 moderators to cover their 2 billion users. Then at that scale, they’d need to hire thousands of meta-moderators and meta-meta-moderators too.
- ivalm 6y ago
  If meta moderators are at the same scale then that’s only 200 meta moderators and only 1 meta meta. The base of a log scaling pyramid is approximately as large as the entire pyramid.
  - justinclift 6y ago
    Wouldn't that be 20 meta moderators (10k:1 scale), rather than 200?
    - ivalm 6y ago
      You’re right, apologies!
- mschuster91 6y ago
  Where's the problem in that? FB makes 18.5 billion $ a year in profit. 250k employees @ 30k would work out to 7.5 billion $ in cost a year, it would hardly leave them bankrupt. Also, Facebook could offset a part of that cost by charging users a token amount - say, five dollars or something like that.
  - 6y ago

theknocker 6y ago

If only someone had figured out hundreds of years ago that policing speech is the purview of idiots and the dictators who refuse to educate them.

mschuster91 6y ago

> 2) they are wayyyyy too big to police content manually Oh hell no they aren't too big, they just don't want to reduce their profits from the current 18.5 billion US $ (per https://www.statista.com/statistics/277229/facebooks-annual-revenue-and-net-income/ https://www.statista.com/statistics/277229/facebooks-annual-...). If Facebook wanted, they could massively ramp up staffing, hire them themselves, and pay them a livable wage instead of outsourcing the cost that their business occurs on society (who has to pick up the slack from conspiracy myths, fascism and other shit), on the users who are mistakenly banned and lose their primary contact e.g. to overseas family, and on the moderation staff that is hired via third parties and exploited.

rossdavidh 6y ago
I would shed no tears if FB profits went down, but...I think you're underestimating how big they are. Two BILLION users, in ~100 countries (each with its own standards on what is acceptable, legal, repulsive, etc.). If you had one moderator per 100 users, that would 20 million moderators. I don't think they could hire 20 million high quality moderators (there's only one dang, and we're not giving him up), and certainly not with 18.5 billion.

simple_phrases 6y ago

They certainly generate enough revenue to moderate content manually.

ben509 6y ago
But they may not be able to do it well enough that it will be notably better than an AI. Scaling manual labor can't work indefinitely due to the scarcity of talent. Keep in mind that while a human being is very good at categorizing things, they're terrible at doing it over and over again, so the scarce talents are likely consistency and conscientiousness. You should expect that, overall, the (n+1)th person you hire is marginally less talented than the nth. And while people are constantly entering and exiting the labor pool, this is negated by the fact that you're trying to expand your share of it. So while there's a lot of randomness in recruiting, if you enforce any kind of standard, you'll hit a point where not enough candidates meet that standard, so you have to drop it. Once it is below the threshold of "better than AI" you have no reason to hire more people. And you'll try to expand your eligible pool by raising wages, but everyone else wants consistent, conscientious people, too, so you're liable to run into the economic constraint: the amount you have to pay someone will exceed the value they're adding as a moderator.

jancsika 6y ago

> 1) they don't feel they can NOT police content True. But we should be explicit here-- FB isn't some neat, nerdy social graph software that just happened to get really popular, like git for regular people or something. They have a business model to shake up their social graph to maximize engagement, and the implementation achieves this by keeping everyone maximally mad as hell at each other. It spreads conspiracy theories efficiently. It and its users are easily exploited, and they continue to be exploited by any nation-state, nefarious actors, or even just some dude looking to make a buck spreading fake news. That all feeds into the same destructive behaviors for the users as the system desperately tries to keep them engaging more and more. Facebook's own software and UI exacerbates these problems. With that starting point in mind, I agree with your 1, 2, 3, and 4 above. But this is less like email having to deal with new sets of problems as it scaled to the entire world. It's more like a drug cartel trying to figure out the maximum amount of their fundamentally destructive product they can sell before the government finally decides to comes in and break up their enterprise.

kangaroozach 6y ago
Censoring content rather than refuting its claims is counterproductive. If skeptics are not able to be argumentatively refute a claim, they will not succeed in changing minds. The current strategy of trying to silence view points will eventually fail miserably. The issue is that when a platform outsources its censorship (“moderation”) to a third party that it still controls, rather than a third party that is actually perceived to be neutral by most people, then you create distrust which further fuels conspiracy theories. Also a conspiracy theory may actually be true. Jeffrey Epstein was not discussed during the 2016 election aside from a few fringe articles. It was too toxic for mainstream to touch. And everyone in Hollywood knew about Harvey Weinstein and even joked about it. These topics were covered up only for so long. Today there are lots of insane and farfetched theories being floated. But the theories being censored today that are indeed based in reality, will eventually become common knowledge and the censors will be implicated in the coverup thereby harming their trust. Discussion, debate, argument, evidence, analysis lead us towards truth. Mute only appears to work and can be a useful political tool for only so long. Today our public forum is virtual, but still a public forum. Trump wasn’t allowed to block certain followers because of their right to be in the forum and hear from their president. So the question is whether governments will acknowledge this fact and limit the power of our big tech group communication platforms and enforce transparency and non-biased appeals process. Or will they cede total control of our public forums to Big Tech out of short term political convenience.

xerxespoy 6y ago

They can solve this issue by adequately providing for self-moderation. Avoiding this approach reveals an intention to impose. Users leave once they realise the violation.

dustingetz 6y ago

It's inherent to centralized power that opposing interests will attack it. Principle agent problem strikes again

Balgair 6y ago

Thank you for stating these points. I believe that FB is going to be cut up due to these issues and the interplay with politics, just like Bell was. Per my views on the situation, Zuck is cooked. When he went before congress to testify, he was being given the classic robber-baron deal that the US has used a few times now. As an analogy to actual robber-baron, at first when the railroads were new, Congress didn't care much about them. Then when all the businesses and politicians started to use them, they started to notice that the railroads were very important. When beef suspiciously didn't get to Chicago on time, or when trains delayed representatives to important votes up in Albany, then the politicians got legislative. A similar scaling of importance has occurred with big tech. The US then typically takes these newly vital sectors and then gives them the deal: Fix it or face jail time. Compliance means that your family for countless generations is never poor again. You go kite-surfing for the rest of your days. Congress works with you to write the laws in such a way that you get a monopoly on the sector with a nice 3% above inflation growth baked in. Congress and the US get a sector that isn't scammy and just works (albeit poorly at times). Congress gets you to testify for a few days, ensures that you have lied to congress in that time, and holds that jail time over you as leverage to comply. Zuck was given this deal. Per the points OP outlines, Zuck cannot follow through on his end of the bargain. He cannot get FB to work the way Congress is demanding that he do. Zuck may not be facing jail time (yet), but may when he gets in front of Congress again. After Dec. 37th / Jan. 6th (because 2020 never ends), Congress likely sees Big Tech as a direct personal threat (not existential) and is likely to move directly against that threat with due haste.

rossdavidh 6y ago
While I think Mark Zuckerberg is unlikely to face jail time, there are some points to your post I agree with. Especially the analogy with railroads. The other point I would add, is that it's not just Congress. I have heard government leaders in Mexico, Poland, Hungary, and Turkey all voicing similar concerns, and in some cases actually passing legislation already. I think more will come.

lamontcg 6y ago

They need to be forced, by government regulations, to have to talk to people who have had their accounts banned and to provide them some kind of legal recourse. The law needs to recognize that for massive social networks like Facebook and Twitter and for the Apple and Google App stores that people are actually entitled to those services, and have a right to use them, and shouldn't be kicked off of them without any form of recourse. They've grown too big to just consider them like the corner coffee shop and allow them to kick anyone off of their platform for whatever reasons they cook up. And that means that they will need to staff up and that will come out of their profit margins, and that's okay. That's why they need to be forced to do it by regulations.

Thorentis 6y ago

Which is why social media networks should make the far easier rule that they will only ban illegal content. If people want something censored, make it illegal. If people don't like they things are being censored, take it up with their local representative.