9 ms·

The spread of low-credibility content by social bots

120 points by infodocket 8y ago

thrower123 8y ago
Wiping out the bots would absolutely destroy the user counts of most social networks, so it's just not going to happen. On the other end, there are a lot of formerly credible sources of information that have stepped in it badly and repeatedly, and tarnished their aura of authority. The fourth estate is looking more and more bought and paid for, and the academy has their own issues there, not to mention a mother of a reproducibility problem.
- thoughtexplorer 8y ago
  What % are bots?
  - thrower123 8y ago
    With Twitter and Instagram, it feels like there are more bots than real people. Facebook, I'm less sure of, but with as many random friend requests as I get from unknown "people" with half-baked profiles and no connections, it's happening there too.
  - gaius 8y ago
    I would estimate at least 50%.
- CathyWest 8y ago
  And now it's a TOS violation to "dehumanise" people on Twitter, and what's more dehumanising than calling a user a bot?
webwanderings 8y ago
> curbing social bots may be an effective strategy for mitigating the spread of online misinformation. Why not turn them off entirely? The least of self regulation these popular social media companies can do for humans.
- tree_of_item 8y ago
  You cannot just "turn them off". What exactly are you going to do?
- asaph 8y ago
  Not all social media bots are malicious. And not all social media bots are posting content from low credibility sources.
  - eiaoa 8y ago
    > Not all social media bots are malicious. And not all social media bots are posting content from low credibility sources. The collateral damage from banning them all may be worth it, if they're a major means for pushing low credibility content and disinformation.
    - robotresearcher 8y ago
      Eliminating all the internet bots would be the mundane real life first step of the Butlerian Jihad of the Dune universe. In the stories, they eliminate all machine computers in order to prevent AIs destroying society, and go back to human computers. The idea that AIs (broadly defined to include simple bots) are net more trouble than they are worth has been kicked around in literature forever. It’s fun to live in the age where we get to wrestle with it in practice.
  - undreren 8y ago
    This is not an argument. Not all drunk drivers kill people either. A good argument must imply that social media bots are a net benefit to society and democracy, not that some good ones exist.
- matt4077 8y ago
  I believe "curbing" is simply a nod to the fact that the people/groups controlling these bots have no interest in turning them off, and are unlikely to change their opinion. The harms this article shows are, after all, their purpose. The actors who may be motivated to mitigate these harms, i. e. the social networks and possibly governments would face problems very similar to those with (other) spam, namely identification. Of course the history of spam filters makes me optimistic that it's feasible put a rather significant dent in the problem with even just modest efforts. Summary: It's more of a semantic difference, arguably more precise because there is no off switch as such.
hlecuanda 8y ago
I have been feeling depressed for a few days pondering the general credibility crisis that prevales over human communications as of late. The spread of low quality content or plain disinformation is just an aspect of the problem. We have those who stand to gain from, politically or financially from influencing the global conversation to their advantage, and we've seen an endless parade of examples, that have elevated Astroturfing and manipulative deceit from a curiosity into an exact science and art form: Cambridge analytica, Facebook and their HR firm, Bannon and Breitbart, the Russian troll farms, twitterbots and fake profiles, the inexplicable fact that FB knows and tells advertisers exactly how many times their ad was shown and bills them accordingly, but they need to develop new technology to figure out exactly how this very same features were used to needle in an election, sow discord and amplify animosity. OTOH I can read and hear all sorts of opinions from all kinds of experts on every field who I no longer trust, since I don't know if they are using their audience for profit and ever so subtly change the perception of those who listen: it puzzles me to no end that nowdays Microsoft is considered open source's best new friend, the same Microsoft that no more than 15 years ago was covertly funding SCO's Lawsuits vs Linux. I feel like advertising has seemed to the very fabric of human communications, not to better humanity but for the short term gain and selfish goals of a few who can afford such services. The internet used to be an Electronic Frontier where everybody could be who they really were and speak their minds. Now it's a poisoned cesspit where everybody lies about everything and those who do it better get to sell you stuff. <//3
- elg0nz 8y ago
  "The world is going to crack wide open. There is something on the horizon. A massive connectivity. The barriers between us will disappear, and we’re not ready. We’ll hurt each other in new ways, we’ll sell and be sold, we’ll expose our most tender selves only to be mocked and destroyed. We’ll be so vulnerable, and we’ll pay the price. We won’t be able to pretend that we can protect ourselves anymore. It’s a huge danger, a gigantic risk, but it’s worth it, if only we can learn to take care of each other, then this awesome destructive new connection won’t isolate us. It won’t leave us in the end so totally alone." - https://medium.com/@chrstphrmllr/you-are-not-safe-bebb0538e16e https://medium.com/@chrstphrmllr/you-are-not-safe-bebb0538e1...
- 8y ago
706f6f70 8y ago
It strikes me as somewhat ironic that Nature would public this given that they recently tweeted: > Editorial: The US Department of Health and Human Services proposes to establish a legal definition of whether someone is male or female based on the genitals they are born with. This proposal has no foundation in science and should be abandoned. [1] That last sentence just absolutely boggles the mind and I'm not entirely sure how this journal retains any semblance of reputation when its editorial board is so clearly willing to put ideology over science. --- [1] https://twitter.com/nature/status/1064694083090812928 https://twitter.com/nature/status/1064694083090812928
- yesenadam 8y ago
  It doesn't 'boggle' my mind. It would be helpful, I suppose, if you could explain how the last sentence shows they are 'so clearly willing to put ideology over science' - nothing could be less clear to me, although I'm far from expert. Otherwise it will seem like that's precisely what you're doing yourself, which I suspect is why you've been heavily downvoted.
  - thoughtexplorer 8y ago
    Because Nature is claiming that science is not science. That should concern everyone. The fact is that anatomy predicts gender over 99% of the time. https://twitter.com/BretWeinstein/status/1064937624329048064 https://twitter.com/BretWeinstein/status/1064937624329048064 https://www.playboy.com/read/the-difference-between-sex-and-gender-explained-1 https://www.playboy.com/read/the-difference-between-sex-and-...
    - notahacker 8y ago
      If the purpose of proposed legislation is to eliminate all consideration of exceptions, surely science demonstrating that exceptions exist is the pertinent factor, and not science demonstrating that exceptions are rare.
  - tree_of_item 8y ago
    The last sentence shows that they, Nature, are seriously suggesting that there is something wrong with defining sex based on observable sexual characteristics. This shows they are willing to put ideology over science: the only reason to suggest such a thing is ideology.
olivermarks 8y ago
I think the massively successful clickbait junk/fake news purveyors like Taboola and Outbrain are equally damaging because they are openly publishing garbage on the sites people visit to read 'news' and form opinions
thoughtexplorer 8y ago
Stuff like this is scary. Check out this video about the caravan and migrants climbing the border wall. Strangely edited. Presented as breaking news. Music as though it's a suspenseful movie. Yet looking at the comments everyone seems to be buying into it. https://www.youtube.com/watch?v=2GuuNzR_AZg https://www.youtube.com/watch?v=2GuuNzR_AZg It has 2M+ views in just a few days.
- jimrandomh 8y ago
  Neither the comments nor the view count are necessarily legit; both can be created by bot traffic.
elvinyung 8y ago
Shameless self-promotion: I wrote down some thoughts a while ago about this: https://www.notion.so/The-Hyperreality-of-Big-Data-ae0aebb358c34d8a9b01eb61d3b77b48 https://www.notion.so/The-Hyperreality-of-Big-Data-ae0aebb35... It almost feels like the only solution (for one aspect of the problem) is for the "good guys" to outmeme the "bad guys". But of course, that's much, much, much easier said than done.
- ghthor 8y ago
  This is a very good short term option, that I hope others have been working on. My personal actions have been to encourage my family member to stop using social media, point them to top quality content producers like Joe Rogan & redlettermedia. And most importantly, I help them to calm down, the information available is exhausting, but the world around us isn't changing that fast, I remind them to use that as a reference, talk to people I person, get involved in local politics, take a nature walk, life is short.
  - WhompingWindows 8y ago
    Is Joe Rogan truly the highest quality content producer? I've seen/heard some of his content, it's quite diffuse wrt information density. Do I really want to listen to 2.5 hour podcast to get what I can in 30 minutes from other podcasters? I also heard him asking Neil deGrasse Tyson about whether the moon landing was faked with a lot of skepticism in his voice...
    - ghthor 8y ago
      I think hes a quality content source at the very least. You can usually find an abridged or best of for the most interesting podcasts. And you can reliably fall down to a complete unedited interview to get more balanced observation 9f his or guests opinions. He sticks out to me as one of my most interesting and he shares similar levels of openness and drive for the truth compared to the other independent producers I am aware of it frequent. There are other quality producers that do better jobs at interviewing different niches. The truth is always changing, entertaining controversial hypotheses is reinforcing or exposing the truth. As my awareness, research, observational and social skills have expanded, so too has my worldview. If it we represent a worldview as a polygon, where each time you accept a new story into your worldview you add that point as a new vertex to your shape. What happens as you validate and accept more points outside your ahapn? Your total perimeter likely increases is length, ever increasing the total number of potentially mysterious stories available to you. The larger your worldview, the larger set of "what ifs" become available to you, "what ifs" produce larger impacts requiring larger and larger changes to your worldview. If you dont entertain the what ifs and discuss potentially impactful what ipfs, the more cataclysmic the potential change you'll have to make. He likely asked In jest, but I feel it was probably healthily for Niel to retell the facts for himself as it was for Joe to entertain the unlikely idea, but incredibly impactful the story is if it's not the whole truth.
sjeohp 8y ago
The assumptions are: 1) third party fact-checkers (eg. snopes) are credible and reliable. 2) the third party "Botometer" (machine-learning model) is valid and was trained on an accurately classified corpus. This type of study always relies on the elevation of someone's subjective assessment (eg. "this account is a bot/human", "this fact-checker is honest") to the status of objective truth. It is a serious weakness when studying subjects loaded with political consequence, wherein well-resourced stakeholders are constantly attempting to shape competing narratives by any means available to them.
- ImaCake 8y ago
  EDIT: just to be clear, I think it's important to note that Nature and these authors likely have a bias towards what they consider more truthful news and that will be reflected in their choices of material and tools for this study. >2) the third party "Botometer" (machine-learning model) is valid and was trained on an accurately classified corpus. For anyone curious enough to follow up on how effective their "botometer" tool was, the methods section [0] details what they used and links to said tools. The language used is simple enough for most people to follow. I am not great at following their statistics, but the botometer seems to be at least decently accurate. Certainly, figure 2d seems pretty convincing to me for validating the botometer. >1) third party fact-checkers (eg. snopes) are credible and reliable. The methods section talks about how their model works with Onion articles. The great thing about The Onion is that it is a perfect positive control i.e it is always false. The authors state the following in the methods section: >Many low-credibility sources label their content as satirical, and viral satire is sometimes mistaken for real news. For these reasons, satire sites are not excluded from the list of low-credibility sources. However, our findings are robust with respect to this choice. The Onion is the satirical source with the highest total volume of shares. We repeated our analyses of most viral articles (e.g., Fig. 3a) with articles from theonion.com excluded and the results were not affected. 0. https://www.nature.com/articles/s41467-018-06930-7#Sec8 https://www.nature.com/articles/s41467-018-06930-7#Sec8
dstroot 8y ago
“Successful low-credibility sources are heavily supported by social bots. These results suggest that curbing social bots may be an effective strategy for mitigating the spread of online misinformation.” No, really? Sorry - go ahead and vote me down. I couldn’t resist. Sometimes science uncovers facts that seem obvious.
m302449 8y ago
How do I know you are not a bot?

120 points by infodocket 8y ago

Wiping out the bots would absolutely destroy the user counts of most social networks, so it's just not going to happen. On the other end, there are a lot of formerly credible sources of information that have stepped in it badly and repeatedly, and tarnished their aura of authority. The fourth estate is looking more and more bought and paid for, and the academy has their own issues there, not to mention a mother of a reproducibility problem.

thoughtexplorer 8y ago
What % are bots?
- thrower123 8y ago
  With Twitter and Instagram, it feels like there are more bots than real people. Facebook, I'm less sure of, but with as many random friend requests as I get from unknown "people" with half-baked profiles and no connections, it's happening there too.
- gaius 8y ago
  I would estimate at least 50%.
CathyWest 8y ago
And now it's a TOS violation to "dehumanise" people on Twitter, and what's more dehumanising than calling a user a bot?

webwanderings 8y ago

> curbing social bots may be an effective strategy for mitigating the spread of online misinformation. Why not turn them off entirely? The least of self regulation these popular social media companies can do for humans.

tree_of_item 8y ago
You cannot just "turn them off". What exactly are you going to do?
asaph 8y ago
Not all social media bots are malicious. And not all social media bots are posting content from low credibility sources.
- eiaoa 8y ago
  > Not all social media bots are malicious. And not all social media bots are posting content from low credibility sources. The collateral damage from banning them all may be worth it, if they're a major means for pushing low credibility content and disinformation.
  - robotresearcher 8y ago
    Eliminating all the internet bots would be the mundane real life first step of the Butlerian Jihad of the Dune universe. In the stories, they eliminate all machine computers in order to prevent AIs destroying society, and go back to human computers. The idea that AIs (broadly defined to include simple bots) are net more trouble than they are worth has been kicked around in literature forever. It’s fun to live in the age where we get to wrestle with it in practice.
- undreren 8y ago
  This is not an argument. Not all drunk drivers kill people either. A good argument must imply that social media bots are a net benefit to society and democracy, not that some good ones exist.
matt4077 8y ago
I believe "curbing" is simply a nod to the fact that the people/groups controlling these bots have no interest in turning them off, and are unlikely to change their opinion. The harms this article shows are, after all, their purpose. The actors who may be motivated to mitigate these harms, i. e. the social networks and possibly governments would face problems very similar to those with (other) spam, namely identification. Of course the history of spam filters makes me optimistic that it's feasible put a rather significant dent in the problem with even just modest efforts. Summary: It's more of a semantic difference, arguably more precise because there is no off switch as such.

hlecuanda 8y ago

I have been feeling depressed for a few days pondering the general credibility crisis that prevales over human communications as of late. The spread of low quality content or plain disinformation is just an aspect of the problem. We have those who stand to gain from, politically or financially from influencing the global conversation to their advantage, and we've seen an endless parade of examples, that have elevated Astroturfing and manipulative deceit from a curiosity into an exact science and art form: Cambridge analytica, Facebook and their HR firm, Bannon and Breitbart, the Russian troll farms, twitterbots and fake profiles, the inexplicable fact that FB knows and tells advertisers exactly how many times their ad was shown and bills them accordingly, but they need to develop new technology to figure out exactly how this very same features were used to needle in an election, sow discord and amplify animosity. OTOH I can read and hear all sorts of opinions from all kinds of experts on every field who I no longer trust, since I don't know if they are using their audience for profit and ever so subtly change the perception of those who listen: it puzzles me to no end that nowdays Microsoft is considered open source's best new friend, the same Microsoft that no more than 15 years ago was covertly funding SCO's Lawsuits vs Linux. I feel like advertising has seemed to the very fabric of human communications, not to better humanity but for the short term gain and selfish goals of a few who can afford such services. The internet used to be an Electronic Frontier where everybody could be who they really were and speak their minds. Now it's a poisoned cesspit where everybody lies about everything and those who do it better get to sell you stuff. <//3

elg0nz 8y ago
"The world is going to crack wide open. There is something on the horizon. A massive connectivity. The barriers between us will disappear, and we’re not ready. We’ll hurt each other in new ways, we’ll sell and be sold, we’ll expose our most tender selves only to be mocked and destroyed. We’ll be so vulnerable, and we’ll pay the price. We won’t be able to pretend that we can protect ourselves anymore. It’s a huge danger, a gigantic risk, but it’s worth it, if only we can learn to take care of each other, then this awesome destructive new connection won’t isolate us. It won’t leave us in the end so totally alone." - https://medium.com/@chrstphrmllr/you-are-not-safe-bebb0538e16e https://medium.com/@chrstphrmllr/you-are-not-safe-bebb0538e1...
8y ago

706f6f70 8y ago

It strikes me as somewhat ironic that Nature would public this given that they recently tweeted: > Editorial: The US Department of Health and Human Services proposes to establish a legal definition of whether someone is male or female based on the genitals they are born with. This proposal has no foundation in science and should be abandoned. [1] That last sentence just absolutely boggles the mind and I'm not entirely sure how this journal retains any semblance of reputation when its editorial board is so clearly willing to put ideology over science. --- [1] https://twitter.com/nature/status/1064694083090812928 https://twitter.com/nature/status/1064694083090812928

yesenadam 8y ago
It doesn't 'boggle' my mind. It would be helpful, I suppose, if you could explain how the last sentence shows they are 'so clearly willing to put ideology over science' - nothing could be less clear to me, although I'm far from expert. Otherwise it will seem like that's precisely what you're doing yourself, which I suspect is why you've been heavily downvoted.
- thoughtexplorer 8y ago
  Because Nature is claiming that science is not science. That should concern everyone. The fact is that anatomy predicts gender over 99% of the time. https://twitter.com/BretWeinstein/status/1064937624329048064 https://twitter.com/BretWeinstein/status/1064937624329048064 https://www.playboy.com/read/the-difference-between-sex-and-gender-explained-1 https://www.playboy.com/read/the-difference-between-sex-and-...
  - notahacker 8y ago
    If the purpose of proposed legislation is to eliminate all consideration of exceptions, surely science demonstrating that exceptions exist is the pertinent factor, and not science demonstrating that exceptions are rare.
- tree_of_item 8y ago
  The last sentence shows that they, Nature, are seriously suggesting that there is something wrong with defining sex based on observable sexual characteristics. This shows they are willing to put ideology over science: the only reason to suggest such a thing is ideology.

olivermarks 8y ago

I think the massively successful clickbait junk/fake news purveyors like Taboola and Outbrain are equally damaging because they are openly publishing garbage on the sites people visit to read 'news' and form opinions

thoughtexplorer 8y ago

Stuff like this is scary. Check out this video about the caravan and migrants climbing the border wall. Strangely edited. Presented as breaking news. Music as though it's a suspenseful movie. Yet looking at the comments everyone seems to be buying into it. https://www.youtube.com/watch?v=2GuuNzR_AZg https://www.youtube.com/watch?v=2GuuNzR_AZg It has 2M+ views in just a few days.

jimrandomh 8y ago
Neither the comments nor the view count are necessarily legit; both can be created by bot traffic.

elvinyung 8y ago

Shameless self-promotion: I wrote down some thoughts a while ago about this: https://www.notion.so/The-Hyperreality-of-Big-Data-ae0aebb358c34d8a9b01eb61d3b77b48 https://www.notion.so/The-Hyperreality-of-Big-Data-ae0aebb35... It almost feels like the only solution (for one aspect of the problem) is for the "good guys" to outmeme the "bad guys". But of course, that's much, much, much easier said than done.

ghthor 8y ago
This is a very good short term option, that I hope others have been working on. My personal actions have been to encourage my family member to stop using social media, point them to top quality content producers like Joe Rogan & redlettermedia. And most importantly, I help them to calm down, the information available is exhausting, but the world around us isn't changing that fast, I remind them to use that as a reference, talk to people I person, get involved in local politics, take a nature walk, life is short.
- WhompingWindows 8y ago
  Is Joe Rogan truly the highest quality content producer? I've seen/heard some of his content, it's quite diffuse wrt information density. Do I really want to listen to 2.5 hour podcast to get what I can in 30 minutes from other podcasters? I also heard him asking Neil deGrasse Tyson about whether the moon landing was faked with a lot of skepticism in his voice...
  - ghthor 8y ago
    I think hes a quality content source at the very least. You can usually find an abridged or best of for the most interesting podcasts. And you can reliably fall down to a complete unedited interview to get more balanced observation 9f his or guests opinions. He sticks out to me as one of my most interesting and he shares similar levels of openness and drive for the truth compared to the other independent producers I am aware of it frequent. There are other quality producers that do better jobs at interviewing different niches. The truth is always changing, entertaining controversial hypotheses is reinforcing or exposing the truth. As my awareness, research, observational and social skills have expanded, so too has my worldview. If it we represent a worldview as a polygon, where each time you accept a new story into your worldview you add that point as a new vertex to your shape. What happens as you validate and accept more points outside your ahapn? Your total perimeter likely increases is length, ever increasing the total number of potentially mysterious stories available to you. The larger your worldview, the larger set of "what ifs" become available to you, "what ifs" produce larger impacts requiring larger and larger changes to your worldview. If you dont entertain the what ifs and discuss potentially impactful what ipfs, the more cataclysmic the potential change you'll have to make. He likely asked In jest, but I feel it was probably healthily for Niel to retell the facts for himself as it was for Joe to entertain the unlikely idea, but incredibly impactful the story is if it's not the whole truth.

sjeohp 8y ago

The assumptions are: 1) third party fact-checkers (eg. snopes) are credible and reliable. 2) the third party "Botometer" (machine-learning model) is valid and was trained on an accurately classified corpus. This type of study always relies on the elevation of someone's subjective assessment (eg. "this account is a bot/human", "this fact-checker is honest") to the status of objective truth. It is a serious weakness when studying subjects loaded with political consequence, wherein well-resourced stakeholders are constantly attempting to shape competing narratives by any means available to them.

ImaCake 8y ago
EDIT: just to be clear, I think it's important to note that Nature and these authors likely have a bias towards what they consider more truthful news and that will be reflected in their choices of material and tools for this study. >2) the third party "Botometer" (machine-learning model) is valid and was trained on an accurately classified corpus. For anyone curious enough to follow up on how effective their "botometer" tool was, the methods section [0] details what they used and links to said tools. The language used is simple enough for most people to follow. I am not great at following their statistics, but the botometer seems to be at least decently accurate. Certainly, figure 2d seems pretty convincing to me for validating the botometer. >1) third party fact-checkers (eg. snopes) are credible and reliable. The methods section talks about how their model works with Onion articles. The great thing about The Onion is that it is a perfect positive control i.e it is always false. The authors state the following in the methods section: >Many low-credibility sources label their content as satirical, and viral satire is sometimes mistaken for real news. For these reasons, satire sites are not excluded from the list of low-credibility sources. However, our findings are robust with respect to this choice. The Onion is the satirical source with the highest total volume of shares. We repeated our analyses of most viral articles (e.g., Fig. 3a) with articles from theonion.com excluded and the results were not affected. 0. https://www.nature.com/articles/s41467-018-06930-7#Sec8 https://www.nature.com/articles/s41467-018-06930-7#Sec8

dstroot 8y ago

“Successful low-credibility sources are heavily supported by social bots. These results suggest that curbing social bots may be an effective strategy for mitigating the spread of online misinformation.” No, really? Sorry - go ahead and vote me down. I couldn’t resist. Sometimes science uncovers facts that seem obvious.

m302449 8y ago

How do I know you are not a bot?