5 ms·

> In 2016, internal analysis at Facebook found 64% of all extremist group joins were due to their own recommendation tools. Yet repeated attempts to counteract

• emilsedgh 6y ago
  Well, based on the original quote, it's not that they passively did nothing about it. The consciously and proactively blocked attempts to fix it.
  • eximius 6y ago
    > attempts to counteract this problem were ignored or shut down. I think you misinterpreted this?
    • deleted 6y ago
      [deleted]
    • emilsedgh 6y ago
      Maybe. How do you interpret it?
      • whymauri 6y ago
        I'm interpreting that the business development culture at Facebook disincentivized taking action against extremist groups.
        • lallysingh 6y ago
          Attempts to prevent it were shut down. Shutting something down is an activity, thus /active/.
          • whymauri 6y ago
            What? This literally makes no sense, lol. Edit: I see the confusion now. emilsedgh, you, and I all agree. I thought emilsedgh was saying the opposite of what they wrote.
      • ghayes 6y ago
        Your original statement can be casually read to seem like your disagreeing with OP’s message (that Facebook did not quell extremist groups), due to the structure of your message (a lead statement followed by a delayed contradiction). You could make it clearer with better emphasis that you are stating “agree and” instead of “no because”.
• ballenf 6y ago
  How did they define "extremist" in that analysis? And how many total people are we talking about? Seems like the relevance of that line really depends on answers to both. I.e., if extremist is super narrow we may be talking about 64 people out of 100. If extremist is overly broad, then maybe all the recommendations were for groups that a majority of the population would not find offensive. Just saying the line by itself without context doesn't convey as much information as it first appears.
• johnchristopher 6y ago
  I don't like Facebook but why would they configure a recommendation engine to stop suggesting extremist groups to people with extremist affinities ? There are no humans behind those wheels. What else beside outright banning should have they done ? (I think banning extremists wouldn't have impacted their revenues much so they should have but that's another debate)
  • rsynnott 6y ago
    > I don't like Facebook but why would they configure a recommendation engine to stop suggesting extremist groups to people with extremist affinities That's almost certainly not what they did. When you see someone ranting about the 5G turning the coronavirus communist or whatever, that person didn't generally come up with that idea themselves; they were exposed to it online, either via friends, or via this. Their algorithm is likely pushing extremist nonsense on people which it determines are vulnerable to believing it, which isn't the same as having an affinity for it. Obviously this isn't what they set out to do; they presumably set out to increase engagement, and if that happens to increase engagement, well...
• bostonvaulter2 6y ago
  Do we know what percentage of group joins in general are due to recommendations? Would be an interesting data point to have.
• rsynnott 6y ago
  But doing so would hurt engagement, and hence the bottom line! Facebook have, perhaps accidentally, created a monster of perverse incentives. Not sure what the solution is, besides regulation (which would be extremely difficult).
  • goatinaboat 6y ago
    Not sure what the solution is, besides regulation (which would be extremely difficult). The solution is only difficult if you start from the basis that Facebook must continue to exist. If they cannot run a profitable business that isn’t harmful, that’s noone’s problem but theirs.
    • dylan604 6y ago
      I would love to see Facebook labeled as a domestic terrorist group for how much they have aided and abetted, and sanctioned accordingly. Make it illegal for companies to do business with them. You don't have to shut them down, but if you make it so they can't earn enough to survive, then oh well. Sorry, not sorry, you're business model wasn't a good one.
  • shawn-furyan 6y ago
    I would be very careful about considering any actions that Facebook, in particular, takes to be accidental. From the very beginning, intentional recklessness ("Move fast and break things") has been their credo. When you're being reckless on purpose, none of the damage you create is accidental.
    • bognition 6y ago
      Move fast and break things out another way is: hurry up let’s undermine democracy
  • deleted 6y ago
    [deleted]
• justicezyx 6y ago
  Well, I dont think anyone should be surprised... Why anyone think capitalists can actually practice morality? That's never been done in the hundreds of years of history of capitalism. And capitalists can be quite moral personally. Across the history, the rich and powerful have always had a positive image. But their enterprises have always been requiring regulations.
• ipsum2 6y ago
  According to Tim Kendall's LinkedIn, he stopped working at Facebook in 2010. So it's interesting that he claims to have internal information from 2016.
  • save_ferris 6y ago
    Maybe, but it's pretty common for people to keep up with former coworkers and happenings in the company, especially since he was an early employee with (I'm assuming) a fair amount of equity. I could see an employee giving him that data out of concern, but that's a fair point.
  • evgen 6y ago
    It is really not that surprising. People talk. When you work for a company like this you end up with a large portion of your circle of friends being current or former co-workers. I knew things there were not NDA-cleared for years after I left various startups because people chat and if you know the right questions to ask or are reasonably good at appearing to know just a bit more than you happen to know then people will often fill in the blanks for you. The well really only runs dry when most of that cohort have also left the company.
• dcolkitt 6y ago
  On the surface it sounds pretty outrageous. My question would be though, what should Facebook do instead? A recommendation engine is just an algorithm to maximize an objective function. That objective function being matching users with content that they enjoy and engage with. The algorithm has no in-built notion of political extremism. It almost assuredly seems to be the case that people with radical opinions prefer to consume media that matches their views. If Bob is a three-percenters, it's highly unlikely he'd prefer to read the latest center-left think piece from The Atlantic. Unless you're willing to ban recommend engines entirely, the only possible alternative I can see is for Facebook to intentionally tip the scales. Extremist political opinions would have to be explicitly penalized in the objective function. But now you've turned Facebook from a neutral platform into an explicit arbiter of political opinion. It means some humans at Facebook are intentionally deciding what people should and should not read, watch and listen to. Remember Facebook as an organization is not terribly representative of the country as a whole. Fewer than 5% of Facebook employees vote Republican, compared to 50% of the country. Virtually no one is over 50. Males are over-represented relative to females. Blacks and hispanics are heavily under-represented. And that doesn't even get into international markets, where the Facebook org is even less representative. The cure sounds worse than the disease. I really think it's a bad idea to pressure Facebook into the game of explicitly picking political winners and losers. A social media platform powerful enough to give you everything you want is strong enough to destroy everything you value.
  • JoshTko 6y ago
    We probably should ban recommendation engines for content.
    • nitrogen 6y ago
      Social platform recommendation engines are designed to optimize "revealed preference." I've commented in the past that "revealed preference" is just a new name for exploitation of vice. People's higher goals are often counter to their day-to-day instinctive behaviors. We should find ways to optimize those goals, rather than momentary metrics.
  • 6y ago
• anigbrowl 6y ago
  Sadly unsurprising. I have Facebook sockpuppet accounts that I use just for researching extremist types and I am constantly amazed at how much of the work FB does for me.
• chinathrow 6y ago
  Who will sue the execs Zuckerberg/Sandberg at this point? It's about time.
• antpls 6y ago
  Having all extremists in one directory must be handy for FBI/police's investigation