12 ms·
The Rise and Fall of 'Social Bot' Research (2021)
- caaqil 5y agoWow, this is really interesting. The research in this area, as the paper points out, is almost completely garbage. The Botometer case is especially shocking: > The total numbers of Twitter followers they analyzed during the election campaign was 838,026, which, assuming the claimed “social bot” prevalence of 9.9%, would correspond to roughly 83k “social bots”. However, the paper does not provide any examples of “social bot” accounts and the raw data was not shared. When we contacted the authors, they were unable to provide us with a single credible example of a “social bot”. > To summarize the result, not a single one of the accounts in the list was a “bot” in any meaningful sense of the word, certainly not a “social bot”. In other words, every single “bot” in our sample was a false alarm; the precision (positive predictive value) of Botometer on this task is 0%. Assuming that Botometer does not perform worse than a random number generator (as we did not check the accounts rated as human), we can also conclude that the best guess for the number of “social bots” following the political parties in Germany is not 83,000, as claimed by Keller and Klinger, but zero.
- Hokusai 5y agoIt seems that is reasonable to claim that Botometer has not proven its effectiveness. But that does not prove the opposite. That a police department is unable to distinguish thieves from the rest of citizens do not prove that there are no thieves but just that the police department is incompetent.
- mike_hearn 5y agoQuite a few comments seem to be mixing up a couple of things here: 1. Do spam bots exist 2. Do "social bots" exist For (1) the answer is clearly yes and nobody is claiming otherwise. However, academics don't see to care about spambots. That's a problem industrial researchers care about but not the sort of people who publish 'scientific' papers. For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any. There's a very deep problem here which crops up in lots of other contexts e.g. why is there so much research into psionic abilities, why is COVID modelling so unreliable, why do scientific journals keep publishing auto-generated gibberish as "peer reviewed science" [1], why do so many papers in so many fields fail to replicate incl. not just social sciences but fields of critical importance like oncology? The root causes are the same, namely, academic institutions don't actually seem to care if research uses the scientific method or not. So you get things like Botometer as featured in this article, where it's just some ML model that's producing noise. It classified half of US Congress as bots and nobody cared. Thousands of papers have been published on the back of this model and zero have been retracted. It's because in that world what matters is publishing things that are interesting, not publishing things that are true. [1] https://dailysceptic.org/2021/10/03/436-randomly-generated-peer-reviewed-papers-published-by-springer-nature/ https://dailysceptic.org/2021/10/03/436-randomly-generated-p...
- quinnjh 5y ago>unreliable, why do scientific journals keep publishing auto-generated gibberish as "peer reviewed science" >these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. Sorry but the issue smells recursive
- yehNonsense0 5y agoBecause our economics have nothing to do with science; we incentivize meme competition; brand versus brand. Society is currently designed to force the have-nots to LARP gladiatorial combat, not enrich each other’s lives.
- tablespoon 5y ago> 2. Do "social bots" exist... > For (2) the correct answer is "there is no reliable evidence to support this". The term social bot is of course vague, but these papers are usually defining it as bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas. And when you go looking for evidence that such bots exist you end up finding what Gallwitz & Kreil found - that not just some of the research is bad, but all of it is. Where are the papers proving this problem actually exists in a clearly convincing way? There aren't any. I'm not sure clear on your distinction between "spam bots" and "social bots," but clearly there is reliable evidence to support "bots that attempt to subtly manipulate political discourse for ideological or geopolitical agendas," exist, like plans to procure accounts for them. https://www.nytimes.com/interactive/2021/12/20/technology/china-facebook-twitter-influence-manipulation.html https://www.nytimes.com/interactive/2021/12/20/technology/ch... > Suppliers should provide accounts on overseas social platforms to purchasers at any time. The platforms include Twitter, Facebook, etc., and the supplier should provide about 300 accounts per month on each platform. Note: Content for this project is highly time sensitive. Suppliers should be able to supply accounts at any time. I'd imagine those accounts would be deployed and managed in an automated fashion. Is someone automating accounts to send out propaganda operating a "spam bot" or a "social bot"?
- mike_hearn 5y agoThe article is paywalled so I can't tell if it's reliable or not. My prior expectation is that it's not reliable, because: a. The NYT has been a prime pusher of conspiracy theories about Russian foreign influence on US elections, none of which turned out to be reliable. Claims of Chinese foreign influence are very similar. b. The media have published a great number of stories on supposed Russian/foreign conspiracies to use "social bots" and on investigation they are invariably relying on third party research of the sort that this paper shows aren't reliable. c. The New York Times has become extremely biased in the last decade, especially post 2016 and especially on anything related to why people vote the way they do. I would say though, that 10 accounts per day doesn't seem like enough to have much impact on anything at the scale of social networks. The claim the paper is making is twofold: 1. The research asserting the existence of large scale social bots is all flawed and doesn't really show that. 2. Even if there are people trying to do such things, there's no evidence to suggest it would or could work. Given the saturation level media coverage of this topic and the widespread ideological belief in certain political quarters that you can rewrite people's politics with hashtags, I'm sure there must be some people, somewhere, trying it out. But if there are, the academic papers claiming to have spotted them aren't really doing so.
- aaron695 5y ago
- fourthark 5y agoThe word seems to be used as an insult equivalent to “NPC”.
- Mizza 5y agoI think it's funny how the language of the terminally online political partisans is mirrored. NPC/Russian Bot, SJW/Chud, Libtard/Nazi, Snowflake/Fragile, etc.. round and round it goes.
- JasonFruit 5y agoThose names are reflective of various ways you can dismiss people without considering their positions — or more charitably, categorize and dismiss people espousing positions you've already considered and dismissed. It shows that those techniques are non-ideological, independent of the views of the person employing them. Use of language like that of a sign that if you're looking for substantive discussion of ideas, you're in the wrong place.
- hunterb123 5y agoThose terms have differing reasons to why they are said, they aren't mirrored, just grouped. They don't really have to be partisan terms either. Some are dangerous and dehumanize the other side, not just insult. Some are just stupid. Some are rude but accurate. - NPC = Person who isn't thinking for themselves, follows the group mindlessly. Russian Bot = Term used to dehumanize and attribute real dissent to a disinformation campaign. SJW = Social justice warrior. An overly PC person who goes on social media cancel culture brigades in the name of fixing social injustices. Chud = Foreign or strange in Slavic or slang for "unattractive"? Seems to be an insult at appearance, not intelligence. Libtard = An insult at liberals/leftists directed at their mental capability. Nazi = Term used to dehumanize. Usually implies the person is racist. Snowflake = Someone who gets offended easily Fragile = Never heard this one used, usage is older than snowflake, maybe just used now because one side doesn't want to use the same term?
- whatshisface 5y ago
- r721 5y agoTwitter thread from author: https://twitter.com/FlorianGallwitz/status/1376167308834529280 https://twitter.com/FlorianGallwitz/status/13761673088345292...
- Wronnay 5y agoThe author also made a talk about it: https://www.youtube.com/watch?v=Qwiss4qWuTs https://www.youtube.com/watch?v=Qwiss4qWuTs May be easier to consume than reading a scientific paper...
- m4lvin 5y agohttps://media.ccc.de/v/rc3-2021-chaoszone-297-the-rise-and-fall https://media.ccc.de/v/rc3-2021-chaoszone-297-the-rise-and-f... and there you can watch it without tracking :-)
- escalt 5y agoThe other author also did a talk (in German) on social bots at the end of 2017 https://media.ccc.de/v/34c3-9268-social_bots_fake_news_und_filterblasen https://media.ccc.de/v/34c3-9268-social_bots_fake_news_und_f... https://youtu.be/6jNWl5d_DOk https://youtu.be/6jNWl5d_DOk
- iratewizard 5y agoThe CIA used white papers during the Cold War for psyops against adversarial nations. This is just one of many examples to show that feds are doing it domestically as well.
- WpgRubyDev 5y agoNo, quite the opposite. The contention that 'psyops' were being conducted using bots is now in serious doubt because of this new research. And far from such 'psyops' being the act of the American federal government, as you suggest, instead it was argued that the Russian government was responsible. And the accusation that there was collusion with Russians to achieve this was the central driving force behind the impeachment and near-removal of the U.S. president. It's completely irrelevant and nonsensical to respond to this research by claiming it proves "the feds" are "doing psyops" domestically.
- radarsat1 5y agoI thought the accusations of collusion were more about one-sided release of information from hacked servers? and that the bots accusation (which is not disproved here afaict?) was more directed at the russians _in addition_ to the hacking accusations. but i always understood that the collusion was strictly about the hacking. correct me if i'm wrong.
- iratewizard 5y agoSo first the evil Russian government (with a gdp smaller than several US states) was behind the bots that were reported by white papers funded domestically. Now that the bots are seen as fake, it was the evil Russians again that faked the domestically funded reports! It's amazing how competent their government is at espionage and how grossly incompetent ours is on every level, according to you.
- WpgRubyDev 5y ago> Now that the bots are seen as fake, it was the evil Russians again that faked the domestically funded reports! You misunderstood my comment because I never suggested that. I don't know where you got that from.
- deleted 5y ago[deleted]
- pthread_t 5y agoWhat about "trolls farms" [0]? [0] https://www.technologyreview.com/2021/09/16/1035851/facebook-troll-farms-report-us-2020-election/ https://www.technologyreview.com/2021/09/16/1035851/facebook...
- r721 5y agoThere's quite a few of undercover journalistic investigations too: >Meet The Activist Who Uncovered The Russian Troll Factory Named In The Mueller Probe >Internet activist Lyudmila Savchuk spent two months working undercover at the troll factory in 2015 https://www.npr.org/sections/parallels/2018/03/15/594062887/some-russians-see-u-s-investigation-into-russian-election-meddling-as-a-soap-ope https://www.npr.org/sections/parallels/2018/03/15/594062887/... (2018) >Inside a Ukrainian Troll Farm >To understand how these powerful mechanisms function, OCCRP member center Slidstvo.Info sent a journalist undercover to work at a Ukrainian troll farm in advance of this summer’s parliamentary elections. https://www.occrp.org/en/investigations/inside-a-ukrainian-troll-farm https://www.occrp.org/en/investigations/inside-a-ukrainian-t... (2019) >Undercover reporter reveals life in a Polish troll farm >Katarzyna Pruszkiewicz spent six months running fake social media accounts at self-described ‘ePR firm’ in Wrocław https://www.theguardian.com/world/2019/nov/01/undercover-reporter-reveals-life-in-a-polish-troll-farm https://www.theguardian.com/world/2019/nov/01/undercover-rep... (2019)
- JohnHaugeland 5y agoThis paper is deeply flawed. It basically says "we rebuked one study, therefore bots aren't real." Anyone who has been on Reddit for the last year has seen government run botnets posting spintext by the hundreds. Thinking there are no bots is wildly naive
- Jiro 5y agoIt's the most common bot detection software and was used by over a thousand research papers.
- freebreakfast 5y agoThe paper does not claim "bots aren't real." It claims: > The field of “social bot” research is fundamentally flawed. While “social bot” researchers have received an enormous amount of public attention, their methods are highly dubious. And, as we and others have demonstrated, they fail miserably and consistently when evaluated under real-world conditions. Studies claiming to investigate the prevalence, properties, or influence of “social bots” have, in reality, just investigated false positives and artifacts of the flawed detection methods employed.
- mypastself 5y agoThe paper found that the vast majority of Twitter users classified as bots were actually human. They analyzed data sets previously used by three organizations responsible for the majority of available papers on the topic. The authors of the paper concluded that, in the future, any claims that withhold the list of account handles classified as bots should be ignored because the claims can’t be independently verified. So they didn’t rebuke just one study but datasets underlying most studies on the topic, they analyzed Twitter rather than Reddit, and they did not claim that bots don’t exist. If I am mistaken, someone who’s analyzed the paper in more depth than I did can correct me.
- elcapitan 5y agoI think this is specific about the notion of "social bots", i.e. bots that are mostly indistinguishable from humans or could even interact with them in a realistic way. Obviously everybody can write a spambot, or click a "post this on twitter" button (which is 90% of what people mistake for "bot posts", anyway).
- taubek 5y agoSo there are no bots on Twitter?
- lmeyerov 5y agoThis article is good to see but frustrating as similarly weak in execution due to cherry picking and selection bias. People will mistake the review's sloppiness and issues with some papers with "social bots don't exist and detection is mostly false positives". It's not like Facebook is spending ~$1B every year here to ban 0 accounts. - Manipulated accounts are prevalent: People sell manipulation tools, and once you identify the tool fingerprints, you can often precisely identify the accounts using them. For example, one of our team members published on NBC etc on a 'cyborg' (mixed use) tool used by Qanon + Trump's social media manager which, initially, self-identified itself via a unique user agent tag. Those resulted in high volume same-text tweets & fake retweets as they'd blast out via members who gave personal account access, which is huge for scales like MAGA Twitter. We have plenty of examples like this, and reading the paper gives a weasel-worded impression that it's not happening. - The paper explicitly did not check accounts Twitter already removed for likewise suspicion of being bots. Note that Twitter is generally slower and and more conservative here than a researcher -- they have money at stake that really likes bots -- so when even Twitter agrees it's a TOS-abusing bot, that's generally a good sign. Ex: One of our collaborators was reporting on Saudi bots, and it took a year for Twitter to take them down, namely around the time Twitter acknowledged the Saudi gov had corrupted one of their employees and they announced a sweep. Skipping those accounts for validation is massive selection bias. Bans take awhile, and frustratingly generally too late as only well after the accounts are used for social media manipulation. But looking at old papers and ignoring the confirmed findings and then claiming no bots is bad science. - The original botometer paper was years ago, which matched a different era in Twitter use, Twitter moderation, and attacker patterns. The paper seems similar to evaluating a HFT algorithm from 10 years ago in the current stock market, and worse, with bad settings, and being surprised they lost money. You wouldnt invest your savings that way. Likewise, who would use a 5yr old virus scanner? It's a more boring paper when you phrase it this way. There are good takeaways from this paper for folks in the field, but they give a quite warped impression if you don't have the context for these Ex: Simple rules and features like in botometer can work, but settings matter (drift, ...) Ex: While we and others are moving to graph neural networks and other continuing advances in account/event analytics, we need to make sure these tools are accessible else social scientists will be stuck with old algorithms, crufty implementations, and confusing settings. (GNNs are super exciting in general, see our recent review on industry adoption of more modern graph intelligence w/ gradientflow .) Ex: Twitter's rules for accessing+ publishing data (... you largely can't) hurts infodemic research, forcing deep hacks in what should have been a simpler reproduction study like this Ex: Based on reception and writing of this paper, the term bots may be confusing the public on what social media manipulation looks like Edit: They could have used clean methodology and written a paper "The Botometer tool is increasingly unreliable; X% of papers relying on it are likely incorrect". I think they should, because it's probably higher than people suspect. But it's not what they did, so I wouldn't conclude much from it until they do. The (mis)reproduction study is not even close to the standards my (peer-reviewed) peers regularly work for.
- autopilot23 5y agoGreat paper, thanks for your effort!
- deleted 5y ago[deleted]
- dr_dshiv 5y agoI remember the social bots on MySpace. It turns out that it is easy to pass the Turing Test when you have big boobs. Although... wasn't the original Turing test actually involving whether a man would beat a bot in pretending to be a woman? Edit: yes, it was about gender detection: https://en.m.wikipedia.org/wiki/Turing_test#:~:text=The%20Turing%20test%2C%20originally%20called,from%2C%20that%20of%20a%20human.&text=If%20the%20evaluator%20cannot%20reliably,to%20have%20passed%20the%20test https://en.m.wikipedia.org/wiki/Turing_test#:~:text=The%20Tu....
- jszymborski 5y agoI will say a lot of the examples of false-positives for bots are true-negatives for BotSentinel. Many of the handles are deactivated or suspended, but here are some examples of some live ones. - @ErikLuczak https://botsentinel.com/profile/1077274080984862720 https://botsentinel.com/profile/1077274080984862720 - @pdmonna https://botsentinel.com/profile/1200756272942829575 https://botsentinel.com/profile/1200756272942829575 - @CTO1ChipNagel https://botsentinel.com/profile/1411284846 https://botsentinel.com/profile/1411284846 Some accounts get a "disruptive" score - @cj_panirman https://botsentinel.com/profile/719017787805577220 https://botsentinel.com/profile/719017787805577220 But another question is, do we care if these bots are actually automated, or whether they are individuals that behave like them. Manually tweeting a single, political hashtag 854 times in a six day period is a spammer, regardless if they are a human. The result is little different than if they had programmed a bot to do so. Another was spamming using an IFTT trigger. If I was on the twitter team, I'd have no problem if that person was flagged as a bot.
- mike_hearn 5y agoOf course we care. Claiming an account is a bot is very different to claiming it's a real but disruptive person. Claims of botting are ultimately always claims of a conspiracy (by the bot master). Many of these conspiracies are merely commercial in nature and so we don't normally use the term, but social bot claims are insidious because they're invariably assertions of political or geopolitical conspiracy. If it's not true it's deeply corrosive to real personal relationships, trust in electoral integrity, international relations and so on. Even putting the wider social effects of a false classification to one side, a disruptive person is not necessarily misrepresenting anything and can possibly be made to follow the rules of the community with temporary timeouts, notification messages, etc. They may not have realized they were being disruptive. A spammer already knows they're breaking the rules of the platform and don't care, so the countermeasures need to be different. Twitter themselves have pointed this out quite eloquently, and made it clear that they think BotSentinel is no better than Botometer: "What about tools like Botometer & Bot Sentinel? .... Binary judgments of who’s a bot or not have real potential to poison our public discourse — particularly when they are pushed out through the media ... the narrative on what’s actually going on is increasingly behind the curve."
- pphysch 5y agoThe entire "online manipulation" research field is deeply compromised and unserious. Because there are enormous political ramifications to it. You can't just come out with a rigorous & factual study of the "Russiagate" hoax or Julian Assange defamation campaign. That would be career suicide if it ended up anywhere but a right-wing or Russian blog. One of the most glaring flaws of the research is the lack of effort put into verifying attribution. Some suspicious accounts are pushing barely-literate propaganda in support of Country A? Case closed: it's a propaganda campaign by the government of Country A. These "studies" virtually never test whether the suspicious behavior was orchestrated by an enemy of Country A to generate bad press, that is, propaganda against Country A. This still happens even after the Vault 7 leaks revealed that the CIA actively plants "false flag" strings in Iranian, Russian, Chinese languages within CIA malware.
- staiz 5y agoThere is also the economic remifications, Facebook, Google, Amazon, MS, All depend upon quantity of Users and quantity of Users time. If there is no distinction between bots and real users, then bots directly increase thier revenue.
- pphysch 5y agoThere is a threshold where platform manipulation can destroy the value of the platform, so these companies definitely have an interest in keeping it in check.
- Raed667 5y agoIn places where labor is cheap, it could be more effective to employ the equivalent of a "call center" to have a large number of people animate a whole bunch of Facebook and Twitter accounts. No bots or special tooling needed.