5 ms·

Have we reached a decade of "AI will fix it" yet, or are we still a couple of years off? I feel like so many issues around content moderation have been waved o

Have we reached a decade of "AI will fix it" yet, or are we still a couple of years off? I feel like so many issues around content moderation have been waved off by the big tech companies as being solved by algorithms and yet time and time again it's made very clear that algorithms alone aren't going to solve these problems. I used to think it was because the engineers and managers involved believed so deeply in the potential but now I'm pretty sure it's because higher ups know the alternative (e.g. hiring humans, or abandoning money-generating algorithmic feeds) would be expensive and they'll throw anything and everything they can at the wall to avoid having to do it.

temp8964 5y ago
> "AI will fix it" What is the "it"? A society problem or a company problem? I don't think AI is designed to fix society problems. They are designed to make more money for the company, even with the cost of the society.
- Nevermark 5y ago
  While AI isn’t a wand for society’s problems, it isn’t a wand for business problems either. In both cases AI will do whatever task you design or train it to do.
jasonshaev 5y ago
I don't love defending Facebook, but as of last year, they employed 15 THOUSAND content moderators (https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-facebook-content-moderation-mistakes-daily-report-says/?sh=1f862a7354d0 https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-...) Content moderation at Facebook scale is really, really hard. Even for real people. Again I hate to defend Facebook, but your comment is very misleading by ignoring that Facebook HAS hired humans to help solve the content moderation problem. A lot of them.
- afavour 5y ago
  The clear implication here is surely that 15,000 is not enough. Which is maybe not surprising when Facebook claims to have 2.85 billion active users. By that metric 15k isn't really all that much. > Content moderation at Facebook scale is really, really hard. Even for real people. Maybe you could even argue that it's impossible. The question then is what to do about it. One answer could be "stop using recommendation algorithms you do not control", but that would harm Facebook's profits.
  - jasonshaev 5y ago
    My guess (and this is purely a guess) is that it's "solvable" to within some threshold. But what is that threshold? I have no idea. If you turn the knobs too far, you'll presumably end up with more false positives. What's an acceptable threshold for false positives? I have no idea. If you asked me 3 years ago, I would have naively answered that technology can do a much better job consistently enforcing moderation rules than our current technology actually can. Realizing how hard it is, even for "trained" humans, eviscerated that naivety real fast. I definitely agree it's, literally, impossible for 15k moderators to accurately review anywhere near enough posts to get to whatever that acceptable threshold is. I'm also not convinced doubling that number is enough either. There's always a certain arbitrariness and subjective interpretation in content moderation. Someone will always end up unhappy with the result.
  - beerandt 5y ago
    Plus the number of moderators is irrelevant if they're trained to only remove posts from non-advertisers, or base their decisions on biased "fact-checker" articles.
  - karmelapple 5y ago
    Using algorithms to determine the order someone sees things in is, in my view, a social media company applying editorial content. And therefore, if their editor (which is an algorithm for Facebook and Twitter) promotes something that’s slanderous or libelous, Facebook should be held to the same standards as an editor who could be prosecuted for that. Same with someone who writes something on Facebook. Write something slanderous? You are like a newspaper reporter at this point. Facebook promotes it in the algorithm? Now their algorithm is responsible, too. Facebook, nor its users, have any significant negative consequences for publishing anything that is false or misleading. Until that changes, I doubt the company’s behavior, nor individual posters, will change.
- hetspookjee 5y ago
  Yes, 15000 is a lot. But >2 billion people is a lot more. Wikipedia has far fewer people employed on content moderation yet they seem to fair just fine. This number throwing on Facebook "doing an effort" is an attempt to keep the toxic business model that they have a live at an acceptable rate. How many people would they save for hiring if they'd change the timeline into a chronological one instead of their current one? It'd be much harder to get as much attention than, but also less addictive, which is a net good for every human except for Facebook as a business.
  - discobot2 5y ago
    Wikipedia has much less content
    - TheOtherHobbes 5y ago
      Yes. Perhaps that's a clue? FB's groups are terribly designed if you want to have a conversation and keep a collection of useful information. They're cleverly designed to keep people scrolling forever on a content treadmill - in no way the same as optimising them for usefulness.
      - Spivak 5y ago
        I mean TikTok is the perfectly optimized content treadmill and compared to Facebook and people rave about how good it is. You forget that a lot of the value of social media is passive entertainment to unwind.
    - colejohnson66 5y ago
      Wikipedia can also be “moderated” by anyone. If I see spam or incorrect info on Wikipedia, I can edit it myself. Maybe reword things, add references, etc. On social media (FB/IG/etc), I don’t have that choice. My only option is to report it and hope the moderators don’t say “this [x] doesn’t violate our community standards” on a clearly spam/scam account/comment/post. I’ve reported so much garbage on social media only to be told it’s not a scam or whatever, I’ve practically given up trying.
- arrosenberg 5y ago
  That being the case, it seems like Facebook in it's current form shouldn't exist then. It has no natural right to exist, and the harms clearly outweigh any benefits that could be argued.
- ElViajero 5y ago
  > I don't love defending Facebook, but as of last year, they employed 15 THOUSAND content moderators If all that discussions in Facebook happened in cafes, libraries and schools there will be way more "moderators" in that spaces. 15,000 is nothing for a business with 2.85 billion users. That is a little more than 1 moderator for each 200,000 users.
- mdoms 5y ago
  > I don't love defending Facebook, but as of last year, they employed 15 THOUSAND content moderators Great. Keep going. They clearly need many thousands more - and they can afford it[0]. [0] https://www.nytimes.com/2021/04/28/business/facebook-earnings-profit.html https://www.nytimes.com/2021/04/28/business/facebook-earning...
- yy727 5y ago
  Content moderation is only viable for small-scale communities, we're just yet to accept that.
mabub24 5y ago
Most social networks operate with a business model/philosophy that the bare minimum of moderation will promote uploading and engagement, capturing the biggest user base, while also saving on expenses. Moderation is just for when you achieve market dominance or attention, and the bare minimum is shifted slightly towards a bit more moderation to appease any critical comments, outcries, or changes in expectations. AI moderation is, and, I think, always has been, a cost saving measure. The effectiveness of it for the health of a platform has always been secondary.
paulpauper 5y ago
Even as early as 2013 Facebook was using advanced image recognition to moderate content. They use both humans and sutomated screening
slg 5y ago
From the article. >we used keyword-based classification to assess whether they contained support for politicians, movements, parties, or ideologies They don't need AI to improve this. A boring old keyword search would improve things. Sure, classifying groups perfectly is a difficult problem especially at Facebook's scale. But this shows almost a complete lack of effort when groups like "Bernie Sanders for President 2020" and "Liberty lovers for Ted Cruz" are recommended.
PragmaticPulp 5y ago
One of my extended family members works for a FAANG company (not Facebook) on a team that moderates user-generated content. They are actually using AI to great effect, with great results. They have large amounts of manual review and moderation as well, but having AI augment the human intervention and also continuously trained based on human overrides of AI decisions makes the system much more efficient. What many people don't understand is that even if Facebook employed human moderators to individually review every group, there would still be isolated examples of policy violations slipping through. Human moderators are far from perfect. They can and do make mistakes. At scale, you get some moderators who don't actually care about doing the work, so they start letting things slip through instead of reviewing them. AI is also helpful in flagging human moderator decisions that disagree strongly with AI predictions, which can then be used to catch moderator errors or improve the AI, depending on which is ultimately deemed correct. It's easy to forget that Facebook has almost 3 billion users. At scales that large, even 99.99% correct content moderation (human or otherwise) will still result in a lot of incorrectly moderated content slipping through. More commonly, it may not be immediately obvious that content breaks policies, or in this case that a group is political. Or maybe the group started as one thing, but then evolved over time to become highly political. Or maybe they chose a name that sounds innocuous, but is actually very offensive given some obscure context or lingo. The problem is impossible to solve perfectly, so we need to instead focus on setting realistic expectations for what can be done.
philovivero 5y ago
What makes you think we're not already in that decade, and this is what it looks like when AI "fixes" things? Are you familiar with AI Alignment? When AI does work, it does not always align with human values and motives. And even if it did, it only aligns with the values and motives of its creators.
- Nevermark 5y ago
  Yes. I think it’s a common view that realigning Facebook’s values and motives is part of any improvement. Thus the public criticism.
AnimalMuppet 5y ago
The problem is the humans - the users. They keep clicking on outrage-bait, and re-posting it. AIs are not going to fix the humans. They might turn the gain down a bit, they might cut off some of the worst stuff, but they're not going to fix the problem. The problem is us.
- cvwright 5y ago
  Sure, the root of the problem is the humans. But IMO you minimize the importance of controlling that gain. HN is full of humans, too, but here the worst/craziest stuff usually gets reduced visibility. On the big "social" platforms, they crank that crap up to 11.
jensensbutton 5y ago
I'd love to see more discussion about what success actually looks like for content moderation on facebook (or the internet, really). We don't seem to care that the legal system doesn't get remotely close to catching everyone who commits an ACTUAL crime so why do we need to be perfect (whatever that even means) for content moderation?

temp8964 5y ago

> "AI will fix it" What is the "it"? A society problem or a company problem? I don't think AI is designed to fix society problems. They are designed to make more money for the company, even with the cost of the society.

Nevermark 5y ago
While AI isn’t a wand for society’s problems, it isn’t a wand for business problems either. In both cases AI will do whatever task you design or train it to do.

jasonshaev 5y ago

I don't love defending Facebook, but as of last year, they employed 15 THOUSAND content moderators (https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-facebook-content-moderation-mistakes-daily-report-says/?sh=1f862a7354d0 https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-...) Content moderation at Facebook scale is really, really hard. Even for real people. Again I hate to defend Facebook, but your comment is very misleading by ignoring that Facebook HAS hired humans to help solve the content moderation problem. A lot of them.

afavour 5y ago
The clear implication here is surely that 15,000 is not enough. Which is maybe not surprising when Facebook claims to have 2.85 billion active users. By that metric 15k isn't really all that much. > Content moderation at Facebook scale is really, really hard. Even for real people. Maybe you could even argue that it's impossible. The question then is what to do about it. One answer could be "stop using recommendation algorithms you do not control", but that would harm Facebook's profits.
- jasonshaev 5y ago
  My guess (and this is purely a guess) is that it's "solvable" to within some threshold. But what is that threshold? I have no idea. If you turn the knobs too far, you'll presumably end up with more false positives. What's an acceptable threshold for false positives? I have no idea. If you asked me 3 years ago, I would have naively answered that technology can do a much better job consistently enforcing moderation rules than our current technology actually can. Realizing how hard it is, even for "trained" humans, eviscerated that naivety real fast. I definitely agree it's, literally, impossible for 15k moderators to accurately review anywhere near enough posts to get to whatever that acceptable threshold is. I'm also not convinced doubling that number is enough either. There's always a certain arbitrariness and subjective interpretation in content moderation. Someone will always end up unhappy with the result.
- beerandt 5y ago
  Plus the number of moderators is irrelevant if they're trained to only remove posts from non-advertisers, or base their decisions on biased "fact-checker" articles.
- karmelapple 5y ago
  Using algorithms to determine the order someone sees things in is, in my view, a social media company applying editorial content. And therefore, if their editor (which is an algorithm for Facebook and Twitter) promotes something that’s slanderous or libelous, Facebook should be held to the same standards as an editor who could be prosecuted for that. Same with someone who writes something on Facebook. Write something slanderous? You are like a newspaper reporter at this point. Facebook promotes it in the algorithm? Now their algorithm is responsible, too. Facebook, nor its users, have any significant negative consequences for publishing anything that is false or misleading. Until that changes, I doubt the company’s behavior, nor individual posters, will change.
hetspookjee 5y ago
Yes, 15000 is a lot. But >2 billion people is a lot more. Wikipedia has far fewer people employed on content moderation yet they seem to fair just fine. This number throwing on Facebook "doing an effort" is an attempt to keep the toxic business model that they have a live at an acceptable rate. How many people would they save for hiring if they'd change the timeline into a chronological one instead of their current one? It'd be much harder to get as much attention than, but also less addictive, which is a net good for every human except for Facebook as a business.
- discobot2 5y ago
  Wikipedia has much less content
  - TheOtherHobbes 5y ago
    Yes. Perhaps that's a clue? FB's groups are terribly designed if you want to have a conversation and keep a collection of useful information. They're cleverly designed to keep people scrolling forever on a content treadmill - in no way the same as optimising them for usefulness.
    - Spivak 5y ago
      I mean TikTok is the perfectly optimized content treadmill and compared to Facebook and people rave about how good it is. You forget that a lot of the value of social media is passive entertainment to unwind.
  - colejohnson66 5y ago
    Wikipedia can also be “moderated” by anyone. If I see spam or incorrect info on Wikipedia, I can edit it myself. Maybe reword things, add references, etc. On social media (FB/IG/etc), I don’t have that choice. My only option is to report it and hope the moderators don’t say “this [x] doesn’t violate our community standards” on a clearly spam/scam account/comment/post. I’ve reported so much garbage on social media only to be told it’s not a scam or whatever, I’ve practically given up trying.
arrosenberg 5y ago
That being the case, it seems like Facebook in it's current form shouldn't exist then. It has no natural right to exist, and the harms clearly outweigh any benefits that could be argued.
ElViajero 5y ago
> I don't love defending Facebook, but as of last year, they employed 15 THOUSAND content moderators If all that discussions in Facebook happened in cafes, libraries and schools there will be way more "moderators" in that spaces. 15,000 is nothing for a business with 2.85 billion users. That is a little more than 1 moderator for each 200,000 users.
mdoms 5y ago
> I don't love defending Facebook, but as of last year, they employed 15 THOUSAND content moderators Great. Keep going. They clearly need many thousands more - and they can afford it[0]. [0] https://www.nytimes.com/2021/04/28/business/facebook-earnings-profit.html https://www.nytimes.com/2021/04/28/business/facebook-earning...
yy727 5y ago
Content moderation is only viable for small-scale communities, we're just yet to accept that.

mabub24 5y ago

Most social networks operate with a business model/philosophy that the bare minimum of moderation will promote uploading and engagement, capturing the biggest user base, while also saving on expenses. Moderation is just for when you achieve market dominance or attention, and the bare minimum is shifted slightly towards a bit more moderation to appease any critical comments, outcries, or changes in expectations. AI moderation is, and, I think, always has been, a cost saving measure. The effectiveness of it for the health of a platform has always been secondary.

paulpauper 5y ago

Even as early as 2013 Facebook was using advanced image recognition to moderate content. They use both humans and sutomated screening

slg 5y ago

From the article. >we used keyword-based classification to assess whether they contained support for politicians, movements, parties, or ideologies They don't need AI to improve this. A boring old keyword search would improve things. Sure, classifying groups perfectly is a difficult problem especially at Facebook's scale. But this shows almost a complete lack of effort when groups like "Bernie Sanders for President 2020" and "Liberty lovers for Ted Cruz" are recommended.

PragmaticPulp 5y ago

One of my extended family members works for a FAANG company (not Facebook) on a team that moderates user-generated content. They are actually using AI to great effect, with great results. They have large amounts of manual review and moderation as well, but having AI augment the human intervention and also continuously trained based on human overrides of AI decisions makes the system much more efficient. What many people don't understand is that even if Facebook employed human moderators to individually review every group, there would still be isolated examples of policy violations slipping through. Human moderators are far from perfect. They can and do make mistakes. At scale, you get some moderators who don't actually care about doing the work, so they start letting things slip through instead of reviewing them. AI is also helpful in flagging human moderator decisions that disagree strongly with AI predictions, which can then be used to catch moderator errors or improve the AI, depending on which is ultimately deemed correct. It's easy to forget that Facebook has almost 3 billion users. At scales that large, even 99.99% correct content moderation (human or otherwise) will still result in a lot of incorrectly moderated content slipping through. More commonly, it may not be immediately obvious that content breaks policies, or in this case that a group is political. Or maybe the group started as one thing, but then evolved over time to become highly political. Or maybe they chose a name that sounds innocuous, but is actually very offensive given some obscure context or lingo. The problem is impossible to solve perfectly, so we need to instead focus on setting realistic expectations for what can be done.

philovivero 5y ago

What makes you think we're not already in that decade, and this is what it looks like when AI "fixes" things? Are you familiar with AI Alignment? When AI does work, it does not always align with human values and motives. And even if it did, it only aligns with the values and motives of its creators.

Nevermark 5y ago
Yes. I think it’s a common view that realigning Facebook’s values and motives is part of any improvement. Thus the public criticism.

AnimalMuppet 5y ago

The problem is the humans - the users. They keep clicking on outrage-bait, and re-posting it. AIs are not going to fix the humans. They might turn the gain down a bit, they might cut off some of the worst stuff, but they're not going to fix the problem. The problem is us.

cvwright 5y ago
Sure, the root of the problem is the humans. But IMO you minimize the importance of controlling that gain. HN is full of humans, too, but here the worst/craziest stuff usually gets reduced visibility. On the big "social" platforms, they crank that crap up to 11.

jensensbutton 5y ago

I'd love to see more discussion about what success actually looks like for content moderation on facebook (or the internet, really). We don't seem to care that the legal system doesn't get remotely close to catching everyone who commits an ACTUAL crime so why do we need to be perfect (whatever that even means) for content moderation?