5 ms·

It seems that is reasonable to claim that Botometer has not proven its effectiveness. But that does not prove the opposite. That a police department is unable

It seems that is reasonable to claim that Botometer has not proven its effectiveness. But that does not prove the opposite. That a police department is unable to distinguish thieves from the rest of citizens do not prove that there are no thieves but just that the police department is incompetent.

mike_hearn 5y ago
Quite a few comments seem to be mixing up a couple of things here: 1. Do spam bots exist 2. Do "social bots" exist For (1) the answer is clearly yes and nobody is claiming otherwise. However, academics don't see to care about spambots. That's a problem industrial researchers care about but not the sort of people who publish 'scientific' papers. For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any. There's a very deep problem here which crops up in lots of other contexts e.g. why is there so much research into psionic abilities, why is COVID modelling so unreliable, why do scientific journals keep publishing auto-generated gibberish as "peer reviewed science" [1], why do so many papers in so many fields fail to replicate incl. not just social sciences but fields of critical importance like oncology? The root causes are the same, namely, academic institutions don't actually seem to care if research uses the scientific method or not. So you get things like Botometer as featured in this article, where it's just some ML model that's producing noise. It classified half of US Congress as bots and nobody cared. Thousands of papers have been published on the back of this model and zero have been retracted. It's because in that world what matters is publishing things that are interesting, not publishing things that are true. [1] https://dailysceptic.org/2021/10/03/436-randomly-generated-peer-reviewed-papers-published-by-springer-nature/ https://dailysceptic.org/2021/10/03/436-randomly-generated-p...
- quinnjh 5y ago
  >unreliable, why do scientific journals keep publishing auto-generated gibberish as "peer reviewed science" >these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. Sorry but the issue smells recursive
- yehNonsense0 5y ago
  Because our economics have nothing to do with science; we incentivize meme competition; brand versus brand. Society is currently designed to force the have-nots to LARP gladiatorial combat, not enrich each other’s lives.
- tablespoon 5y ago
  > 2. Do "social bots" exist... > For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any. I'm not sure clear on your distinction between "spam bots" and "social bots," but clearly there is reliable evidence to support "bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas," exist, like plans to procure accounts for them. https://www.nytimes.com/interactive/2021/12/20/technology/china-facebook-twitter-influence-manipulation.html https://www.nytimes.com/interactive/2021/12/20/technology/ch... > Suppliers should provide accounts on overseas social platforms to purchasers at any time. The platforms include Twitter, Facebook, etc., and the supplier should provide about 300 accounts per month on each platform. Note: Content for this project is highly time sensitive. Suppliers should be able to supply accounts at any time. I'd imagine those accounts would be deployed and managed in an automated fashion. Is someone automating accounts to send out propaganda operating a "spam bot" or a "social bot"?
  - mike_hearn 5y ago
    The article is paywalled so I can't tell if it's reliable or not. My prior expectation is that it's not reliable, because: a. The NYT has been a prime pusher of conspiracy theories about Russian foreign influence on US elections, none of which turned out to be reliable. Claims of Chinese foreign influence are very similar. b. The media have published a great number of stories on supposed Russian/foreign conspiracies to use "social bots" and on investigation they are invariably relying on third party research of the sort that this paper shows aren't reliable. c. The New York Times has become extremely biased in the last decade, especially post 2016 and especially on anything related to why people vote the way they do. I would say though, that 10 accounts per day doesn't seem like enough to have much impact on anything at the scale of social networks. The claim the paper is making is twofold: 1. The research asserting the existence of large scale social bots is all flawed and doesn't really show that. 2. Even if there are people trying to do such things, there's no evidence to suggest it would or could work. Given the saturation level media coverage of this topic and the widespread ideological belief in certain political quarters that you can rewrite people's politics with hashtags, I'm sure there must be some people, somewhere, trying it out. But if there are, the academic papers claiming to have spotted them aren't really doing so.
aaron695 5y ago

mike_hearn 5y ago

Quite a few comments seem to be mixing up a couple of things here: 1. Do spam bots exist 2. Do "social bots" exist For (1) the answer is clearly yes and nobody is claiming otherwise. However, academics don't see to care about spambots. That's a problem industrial researchers care about but not the sort of people who publish 'scientific' papers. For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any. There's a very deep problem here which crops up in lots of other contexts e.g. why is there so much research into psionic abilities, why is COVID modelling so unreliable, why do scientific journals keep publishing auto-generated gibberish as "peer reviewed science" [1], why do so many papers in so many fields fail to replicate incl. not just social sciences but fields of critical importance like oncology? The root causes are the same, namely, academic institutions don't actually seem to care if research uses the scientific method or not. So you get things like Botometer as featured in this article, where it's just some ML model that's producing noise. It classified half of US Congress as bots and nobody cared. Thousands of papers have been published on the back of this model and zero have been retracted. It's because in that world what matters is publishing things that are interesting, not publishing things that are true. [1] https://dailysceptic.org/2021/10/03/436-randomly-generated-peer-reviewed-papers-published-by-springer-nature/ https://dailysceptic.org/2021/10/03/436-randomly-generated-p...

quinnjh 5y ago
>unreliable, why do scientific journals keep publishing auto-generated gibberish as "peer reviewed science" >these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. Sorry but the issue smells recursive
yehNonsense0 5y ago
Because our economics have nothing to do with science; we incentivize meme competition; brand versus brand. Society is currently designed to force the have-nots to LARP gladiatorial combat, not enrich each other’s lives.
tablespoon 5y ago
> 2. Do "social bots" exist... > For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any. I'm not sure clear on your distinction between "spam bots" and "social bots," but clearly there is reliable evidence to support "bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas," exist, like plans to procure accounts for them. https://www.nytimes.com/interactive/2021/12/20/technology/china-facebook-twitter-influence-manipulation.html https://www.nytimes.com/interactive/2021/12/20/technology/ch... > Suppliers should provide accounts on overseas social platforms to purchasers at any time. The platforms include Twitter, Facebook, etc., and the supplier should provide about 300 accounts per month on each platform. Note: Content for this project is highly time sensitive. Suppliers should be able to supply accounts at any time. I'd imagine those accounts would be deployed and managed in an automated fashion. Is someone automating accounts to send out propaganda operating a "spam bot" or a "social bot"?
- mike_hearn 5y ago
  The article is paywalled so I can't tell if it's reliable or not. My prior expectation is that it's not reliable, because: a. The NYT has been a prime pusher of conspiracy theories about Russian foreign influence on US elections, none of which turned out to be reliable. Claims of Chinese foreign influence are very similar. b. The media have published a great number of stories on supposed Russian/foreign conspiracies to use "social bots" and on investigation they are invariably relying on third party research of the sort that this paper shows aren't reliable. c. The New York Times has become extremely biased in the last decade, especially post 2016 and especially on anything related to why people vote the way they do. I would say though, that 10 accounts per day doesn't seem like enough to have much impact on anything at the scale of social networks. The claim the paper is making is twofold: 1. The research asserting the existence of large scale social bots is all flawed and doesn't really show that. 2. Even if there are people trying to do such things, there's no evidence to suggest it would or could work. Given the saturation level media coverage of this topic and the widespread ideological belief in certain political quarters that you can rewrite people's politics with hashtags, I'm sure there must be some people, somewhere, trying it out. But if there are, the academic papers claiming to have spotted them aren't really doing so.

aaron695 5y ago