7 ms·

CMU Researchers: Nearly Half of Accounts Tweeting About Coronavirus Are Bots

106 points by Scipio_Afri 6y ago

derivativethrow 6y ago
The study hasn't been published, so what's the story doing here? Tell me when I can read an actual paper.
bransonf 6y ago
I know a few people who do Twitter bot detection as sort of a hobby, and I would say you have to be skeptic of these numbers from both directions. Obvious bots are, well, obvious. Novice or careless implementations lead to accounts that tweet at the same time or based on the same trigger words. Tweets made on a routine basis are the easiest to detect (e.g. Tweets at 5PM CST every day) But things go much deeper. There are even companies that now develop bots tools essentially, increasing the randomness or human-like qualities of bots. These are much harder to detect with the standard methods. There are other proxies, like not getting organic replies or having no friends. But the sophisticated bot networks have already figured this out. You have to realize that sometimes human labor even goes into the effort, where someone will maintain hundreds of fake accounts, and attempt to make them look organic. From these points, it's easy to underestimate the number of actual bot accounts. On the other hand, these same methods don't usually control well for a certain type of Twitter user. There are actually some really obsessive twitter users who will spend 12+ hours a day active. They typically produce a ton of content, and a lot of replies. I notice this on celebrity/political accounts especially. (Apparently people are calling this the 'reply-guy' now) Those accounts are susceptible to appearing as bots because they deviate far from what one might imagine the 'average' Twitter user looks like. Speaking anecdotally, I think this relates closest to politics, and hot topic issues (Hence the topic in this research). Take a Tweet from the US President for example. You will notice the same few dozen accounts reply to every Tweet of his. Some may be bots, but others are most definitely human. But these often get classified as bots. If I had to make a guess, I would say the bot detection models fail more often to account for the true number of bots. At this point, it's an arms race. Obvious bots are becoming increasingly less obvious. Overall, I think bots are only going to get better, and it worries me for the fate of social networks. The thing most frightening is finding an obvious bot and seeing a large number of organic users interacting with it as if it were real. Edit: One more thing I forgot to mention. An increasing number of bots are getting verified now. I have no idea how the verification process works, but apparently the botters have figured out how to game it successfully.
- aspenmayer 6y ago
  ‘Reply-guy‘ is subjective, and a bit reductionist and has a somewhat negative connotation. It’s used to describe unwanted attention from a specific person or a specific type of person, which is also subjective. It goes further than simply replying a lot and generally being active on Twitter. Not to be a lexical prescriptivist, just aiming for clarity of description of usage and NPOV. https://twitter.com/sbarolo/status/1036685010869407744?s=20 https://twitter.com/sbarolo/status/1036685010869407744?s=20 https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_vie...
  - bransonf 6y ago
    Appreciate it. I've just been hearing the term thrown around a lot lately in context of someone who replies to every single one of someone else's tweets.
    - aspenmayer 6y ago
      I would say that’s proper usage based on context.
- metrokoi 6y ago
  > There are actually some really obsessive twitter users who will spend 12+ hours a day active. They typically produce a ton of content, and a lot of replies. I notice this on celebrity/political accounts especially. (Apparently people are calling this the 'reply-guy' now) Lol there are definitely some real people that post as if they would not pass the Turing test! Repetitive replies, etc.
  - pjc50 6y ago
    Yes - I'm not sure this actually matters for purposes of blocking though?
    - metrokoi 6y ago
elamje 6y ago
I went bot hunting a few weeks ago and it was incredibly insightful. What’s interesting is you can just drill in deeper and deeper by looking at the followers of bot accounts, which are very likely bots themselves. Many created in Feb and March 2020 with thousands of tweets already. I had a lot of fun thinking about the heuristics to detect a bot. Some common ones seemed to be they follow a bot, they have inflammatory bio, banner, or profile pictures. Many tweet really polarizing things. Here is a thread I did about it. I actually tagged a suspect account and the troll started commenting in the thread, DMed me, reported random tweets, and a bunch of other stuff. https://twitter.com/elamje/status/1259691257724579841?s=21 https://twitter.com/elamje/status/1259691257724579841?s=21 The first bot seems to launch ad hominem attacks on organic users via retweets with comments saying something terrible about them. When it doesn’t do that, it just retweets other accounts that do the same strategy.
- metrokoi 6y ago
  >Some common ones seemed to be they follow a bot, they have inflammatory bio, banner, or profile pictures. Many tweet really polarizing things. Perhaps many are real people who get banned often and create new accounts, hence the recent account created dates and inflammatory bio, banner, etc. I often hear about bots and their influence on the internet, but I don't seem to see them that often. Granted, I try not to spend a lot of time on social media, nor do I actively search for them. I suppose I have never seen a clear definition of what makes a bot a bot.
  - elamje 6y ago
    Sure. In the thread I elaborate that they typically have already cranked out thousands of tweets. Some seem more innocuous. Some seem to have already gained 50 followers in a month or two. That’s possible, but hard without a viral tweet. Some might be real users.
    - anigbrowl 6y ago
      You'll probably like the tools from the U of Indiana's social media labs: http://osome.iuni.iu.edu/tools/ http://osome.iuni.iu.edu/tools/
edyyhotteb 6y ago
When in doubt, blame the Russians and Chinese "We do know that it looks like it's a propaganda machine, and it definitely matches the Russian and Chinese playbooks, but it would take a tremendous amount of resources to substantiate that," said Kathleen Carley, a professor of computer science at Carnegie Mellon University who is conducting a study into bot-generated coronavirus activity on Twitter that has yet to be published.
- SpicyLemonZest 6y ago
  Surely you understand why it's ironic to register a new account just to post this comment.
  - edyyhotteb 6y ago
    No but i fully understand what confirmation bias is
  - buboard 6y ago
    it would be ironic if he/she is a bot. you think so? more likely, they wanted to avoid looking like straying from the usual groupthink
- sjg007 6y ago
  There is definitely a Russian disinformation campaign as well as a GOP disinformation campaign. With respect to Covid-19 these are designed to get Trump re-elected. I think the Russians also just want to sow discord as well per usual. The pitch will be that it was the democratic governors who shut down the economy and not him.
deleted 6y ago
[deleted]
fzeroracer 6y ago
This is part of the problem I have with the narrative that you must allow completely unfettered speech on any platform. That you should fight lies with the truth. That sort of argument ends up going completely out the window when you have bot accounts spun up by the thousands designed to artificially boost certain narrative for the sake of profit or to sow dissent. Much like managing your lawn, you can't let the weeds grow and expect other plants to simply out-compete them when the weeds choke out everything else around them. Unfortunately that seems to be an inevitable problem on any platform that prioritizes userbase growth above all else.
- IIAOPSW 6y ago
  Did you just make this analogy up? I ask because it sounds a lot like "Well-kept gardens die by pacifism". https://www.lesswrong.com/posts/tscc3e5eujrsEeFN4/well-kept-gardens-die-by-pacifism https://www.lesswrong.com/posts/tscc3e5eujrsEeFN4/well-kept-...
bobbiechen 6y ago
I wonder how many of these are actively trying to influence the conversation, as opposed to just piling onto a trending hashtag with irrelevant material. As an example, I picked the currently trending hashtag #YouKnowYoureASidePieceWhen and immediately found this spam: https://twitter.com/DebtReliefLoans/status/1263703393119371268?s=19 https://twitter.com/DebtReliefLoans/status/12637033931193712...
Drybones 6y ago
As someone who spends way too much time on Twitter (been using it since 2008), I see these kinds of articles often and have hardly ever encountered these types of bots. Additionally, I've had my own account locked and shadow/ghost banned on twitter for having the words coronavirus or COVID-19 in tweets. It took weeks of never tweeting those words to undo it. I've even had my personal and business twitter accounts suspended all at once for "Violating Twitter rules of using multiple accounts for targeted harassment." They restored some of these accounts after filing an appeal and they said "We're sorry, but our systems mistakenly flagged your account." They never restored all of them and the ones restored have been locked permanently again because one account logged into the app used the C words again. One of the suspended accounts never to return was a 10K follower image bot for tech and old school computer images. Another was an account for my main business to announce news and server status updates. Whether bots are mucking up Twitter or not, Twitter itself is a horrible platform and company. I can't wait for the service to disappear into oblivion.
- throwaway-2020 6y ago
  >Twitter itself is a horrible platform and company. I can't wait for the service to disappear into oblivion. You can always walk away from a service. If you are waiting for the next "Big Thing" to ditch your problems you will find that even this new shiny platform will (eventually) have the same or worse problems. The internet is a rabble that needs taming. Services automate a lot of stuff to try to tame us, but nothing is perfect. Innocent users get banned. If you are building a business on these services don't rely on just one. Build redundancy into your marketing, use many channels.
dannyeei 6y ago
You cut out the rest of the title! "... Down from the normal 78% which it's normally at"
newyankee 6y ago
Although not a fan of exposing real identity on social media i wonder if there is a mechanism where twitter can acknowledge that an account is real based on some behind the scenes Zk-snark like validation.
tiagobraw 6y ago
why don't Twitter require a capcha before tweeting? it would reduce the number of bots by a large factor... it could also have a way for legitimate bots to be registered and easily identified as such
- TomasEkeli 6y ago
  Irritate all the twitter users every time they want to give twitter free content? That would kill twitter. I'm all for it!
roenxi 6y ago
This is bizarre to the point where there are a bunch of questions raised. > Among the misinformation disseminated by bot accounts: tweeted conspiracy theories about hospitals being filled with mannequins or tweets that connected the spread of the coronavirus to 5G wireless towers > We do know that it looks like it's a propaganda machine, and it definitely matches the Russian and Chinese playbooks What exactly is mental state of the Chinese or Russian propaganda departments that they would be doing this? How could the Russians possibly benefit from US citizens aged 60+ dying off at 2-4 times the normal rate? How could the Russians possibly sow more division amongst Americans vs. the normal state of American politics? Just this week Pelosi called Trump morbidly obese, and she is the underdog of childish name-calling in that pairing. How could foreign agents trolling on twitter top that level of incivility? And who are the idiots who are looking up random twitter accounts as their information source?
- pjc50 6y ago
  The foreign intelligence services don't care about left vs. right; they care about competent vs. incompetent, and destroying the American capacity for foresight and competent, proportional response. Every time a competent person says something correct the partisans don't like and gets fired, they win a little more. There is no way that Russia could sink the USS Theodore Roosevelt; their own carrier practically sank on its own during a recent sortie. They're not responsible for the mass infection of its crew either - and it may not have been possible to prevent. But what they can do is make the firing of its captain a partisan issue, and of course ensure that the acting secretary of the Navy is operating in an environment where partisan loyalty to Trump is more important than empathy with the crew, so he gives a morale-destroying speech. An early, solid lockdown would have been short and contained the virus. A relaxed, fragmented one ensures it remains in the population, damaging the economy longer. Coronavirus isn't a biological weapon. Partisan stupidity can be weaponised, though.
  - roenxi 6y ago
    That is a totally stupid strategy that relies on a series of connections that don't make any sense and could just as easily be good decisions for America that result in the country doing better than normal. And it isn't Russia making partisan issues partisan - it is the action of the partisans themselves. It would be cheaper and more effective to bribe key officials in government. There was a conspiracy about Jeff Epstein running his paedophile ring in aid of ... Mossad or the Illuminati or whoever. It would be a bit more expensive and a lot more effective to make that conspiracy theory a reality rather than rely on bots to 'sow division' - whatever that is supposed to do that the politicians can't manage on their own.
deleted 6y ago
[deleted]
swagatkonchada 6y ago
Twitter needs to be a lot better at detecting bots.

106 points by Scipio_Afri 6y ago

The study hasn't been published, so what's the story doing here? Tell me when I can read an actual paper.

I know a few people who do Twitter bot detection as sort of a hobby, and I would say you have to be skeptic of these numbers from both directions. Obvious bots are, well, obvious. Novice or careless implementations lead to accounts that tweet at the same time or based on the same trigger words. Tweets made on a routine basis are the easiest to detect (e.g. Tweets at 5PM CST every day) But things go much deeper. There are even companies that now develop bots tools essentially, increasing the randomness or human-like qualities of bots. These are much harder to detect with the standard methods. There are other proxies, like not getting organic replies or having no friends. But the sophisticated bot networks have already figured this out. You have to realize that sometimes human labor even goes into the effort, where someone will maintain hundreds of fake accounts, and attempt to make them look organic. From these points, it's easy to underestimate the number of actual bot accounts. On the other hand, these same methods don't usually control well for a certain type of Twitter user. There are actually some really obsessive twitter users who will spend 12+ hours a day active. They typically produce a ton of content, and a lot of replies. I notice this on celebrity/political accounts especially. (Apparently people are calling this the 'reply-guy' now) Those accounts are susceptible to appearing as bots because they deviate far from what one might imagine the 'average' Twitter user looks like. Speaking anecdotally, I think this relates closest to politics, and hot topic issues (Hence the topic in this research). Take a Tweet from the US President for example. You will notice the same few dozen accounts reply to every Tweet of his. Some may be bots, but others are most definitely human. But these often get classified as bots. If I had to make a guess, I would say the bot detection models fail more often to account for the true number of bots. At this point, it's an arms race. Obvious bots are becoming increasingly less obvious. Overall, I think bots are only going to get better, and it worries me for the fate of social networks. The thing most frightening is finding an obvious bot and seeing a large number of organic users interacting with it as if it were real. Edit: One more thing I forgot to mention. An increasing number of bots are getting verified now. I have no idea how the verification process works, but apparently the botters have figured out how to game it successfully.

aspenmayer 6y ago
‘Reply-guy‘ is subjective, and a bit reductionist and has a somewhat negative connotation. It’s used to describe unwanted attention from a specific person or a specific type of person, which is also subjective. It goes further than simply replying a lot and generally being active on Twitter. Not to be a lexical prescriptivist, just aiming for clarity of description of usage and NPOV. https://twitter.com/sbarolo/status/1036685010869407744?s=20 https://twitter.com/sbarolo/status/1036685010869407744?s=20 https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_vie...
- bransonf 6y ago
  Appreciate it. I've just been hearing the term thrown around a lot lately in context of someone who replies to every single one of someone else's tweets.
  - aspenmayer 6y ago
    I would say that’s proper usage based on context.
metrokoi 6y ago
> There are actually some really obsessive twitter users who will spend 12+ hours a day active. They typically produce a ton of content, and a lot of replies. I notice this on celebrity/political accounts especially. (Apparently people are calling this the 'reply-guy' now) Lol there are definitely some real people that post as if they would not pass the Turing test! Repetitive replies, etc.
- pjc50 6y ago
  Yes - I'm not sure this actually matters for purposes of blocking though?
  - metrokoi 6y ago

elamje 6y ago

I went bot hunting a few weeks ago and it was incredibly insightful. What’s interesting is you can just drill in deeper and deeper by looking at the followers of bot accounts, which are very likely bots themselves. Many created in Feb and March 2020 with thousands of tweets already. I had a lot of fun thinking about the heuristics to detect a bot. Some common ones seemed to be they follow a bot, they have inflammatory bio, banner, or profile pictures. Many tweet really polarizing things. Here is a thread I did about it. I actually tagged a suspect account and the troll started commenting in the thread, DMed me, reported random tweets, and a bunch of other stuff. https://twitter.com/elamje/status/1259691257724579841?s=21 https://twitter.com/elamje/status/1259691257724579841?s=21 The first bot seems to launch ad hominem attacks on organic users via retweets with comments saying something terrible about them. When it doesn’t do that, it just retweets other accounts that do the same strategy.

metrokoi 6y ago
>Some common ones seemed to be they follow a bot, they have inflammatory bio, banner, or profile pictures. Many tweet really polarizing things. Perhaps many are real people who get banned often and create new accounts, hence the recent account created dates and inflammatory bio, banner, etc. I often hear about bots and their influence on the internet, but I don't seem to see them that often. Granted, I try not to spend a lot of time on social media, nor do I actively search for them. I suppose I have never seen a clear definition of what makes a bot a bot.
- elamje 6y ago
  Sure. In the thread I elaborate that they typically have already cranked out thousands of tweets. Some seem more innocuous. Some seem to have already gained 50 followers in a month or two. That’s possible, but hard without a viral tweet. Some might be real users.
  - anigbrowl 6y ago
    You'll probably like the tools from the U of Indiana's social media labs: http://osome.iuni.iu.edu/tools/ http://osome.iuni.iu.edu/tools/

edyyhotteb 6y ago

When in doubt, blame the Russians and Chinese "We do know that it looks like it's a propaganda machine, and it definitely matches the Russian and Chinese playbooks, but it would take a tremendous amount of resources to substantiate that," said Kathleen Carley, a professor of computer science at Carnegie Mellon University who is conducting a study into bot-generated coronavirus activity on Twitter that has yet to be published.

SpicyLemonZest 6y ago
Surely you understand why it's ironic to register a new account just to post this comment.
- edyyhotteb 6y ago
  No but i fully understand what confirmation bias is
- buboard 6y ago
  it would be ironic if he/she is a bot. you think so? more likely, they wanted to avoid looking like straying from the usual groupthink
sjg007 6y ago
There is definitely a Russian disinformation campaign as well as a GOP disinformation campaign. With respect to Covid-19 these are designed to get Trump re-elected. I think the Russians also just want to sow discord as well per usual. The pitch will be that it was the democratic governors who shut down the economy and not him.

deleted 6y ago

[deleted]

fzeroracer 6y ago

This is part of the problem I have with the narrative that you must allow completely unfettered speech on any platform. That you should fight lies with the truth. That sort of argument ends up going completely out the window when you have bot accounts spun up by the thousands designed to artificially boost certain narrative for the sake of profit or to sow dissent. Much like managing your lawn, you can't let the weeds grow and expect other plants to simply out-compete them when the weeds choke out everything else around them. Unfortunately that seems to be an inevitable problem on any platform that prioritizes userbase growth above all else.

IIAOPSW 6y ago
Did you just make this analogy up? I ask because it sounds a lot like "Well-kept gardens die by pacifism". https://www.lesswrong.com/posts/tscc3e5eujrsEeFN4/well-kept-gardens-die-by-pacifism https://www.lesswrong.com/posts/tscc3e5eujrsEeFN4/well-kept-...

bobbiechen 6y ago

I wonder how many of these are actively trying to influence the conversation, as opposed to just piling onto a trending hashtag with irrelevant material. As an example, I picked the currently trending hashtag #YouKnowYoureASidePieceWhen and immediately found this spam: https://twitter.com/DebtReliefLoans/status/1263703393119371268?s=19 https://twitter.com/DebtReliefLoans/status/12637033931193712...

Drybones 6y ago

As someone who spends way too much time on Twitter (been using it since 2008), I see these kinds of articles often and have hardly ever encountered these types of bots. Additionally, I've had my own account locked and shadow/ghost banned on twitter for having the words coronavirus or COVID-19 in tweets. It took weeks of never tweeting those words to undo it. I've even had my personal and business twitter accounts suspended all at once for "Violating Twitter rules of using multiple accounts for targeted harassment." They restored some of these accounts after filing an appeal and they said "We're sorry, but our systems mistakenly flagged your account." They never restored all of them and the ones restored have been locked permanently again because one account logged into the app used the C words again. One of the suspended accounts never to return was a 10K follower image bot for tech and old school computer images. Another was an account for my main business to announce news and server status updates. Whether bots are mucking up Twitter or not, Twitter itself is a horrible platform and company. I can't wait for the service to disappear into oblivion.

throwaway-2020 6y ago
>Twitter itself is a horrible platform and company. I can't wait for the service to disappear into oblivion. You can always walk away from a service. If you are waiting for the next "Big Thing" to ditch your problems you will find that even this new shiny platform will (eventually) have the same or worse problems. The internet is a rabble that needs taming. Services automate a lot of stuff to try to tame us, but nothing is perfect. Innocent users get banned. If you are building a business on these services don't rely on just one. Build redundancy into your marketing, use many channels.

dannyeei 6y ago

You cut out the rest of the title! "... Down from the normal 78% which it's normally at"

newyankee 6y ago

Although not a fan of exposing real identity on social media i wonder if there is a mechanism where twitter can acknowledge that an account is real based on some behind the scenes Zk-snark like validation.

tiagobraw 6y ago

why don't Twitter require a capcha before tweeting? it would reduce the number of bots by a large factor... it could also have a way for legitimate bots to be registered and easily identified as such

TomasEkeli 6y ago
Irritate all the twitter users every time they want to give twitter free content? That would kill twitter. I'm all for it!

roenxi 6y ago

This is bizarre to the point where there are a bunch of questions raised. > Among the misinformation disseminated by bot accounts: tweeted conspiracy theories about hospitals being filled with mannequins or tweets that connected the spread of the coronavirus to 5G wireless towers > We do know that it looks like it's a propaganda machine, and it definitely matches the Russian and Chinese playbooks What exactly is mental state of the Chinese or Russian propaganda departments that they would be doing this? How could the Russians possibly benefit from US citizens aged 60+ dying off at 2-4 times the normal rate? How could the Russians possibly sow more division amongst Americans vs. the normal state of American politics? Just this week Pelosi called Trump morbidly obese, and she is the underdog of childish name-calling in that pairing. How could foreign agents trolling on twitter top that level of incivility? And who are the idiots who are looking up random twitter accounts as their information source?

pjc50 6y ago
The foreign intelligence services don't care about left vs. right; they care about competent vs. incompetent, and destroying the American capacity for foresight and competent, proportional response. Every time a competent person says something correct the partisans don't like and gets fired, they win a little more. There is no way that Russia could sink the USS Theodore Roosevelt; their own carrier practically sank on its own during a recent sortie. They're not responsible for the mass infection of its crew either - and it may not have been possible to prevent. But what they can do is make the firing of its captain a partisan issue, and of course ensure that the acting secretary of the Navy is operating in an environment where partisan loyalty to Trump is more important than empathy with the crew, so he gives a morale-destroying speech. An early, solid lockdown would have been short and contained the virus. A relaxed, fragmented one ensures it remains in the population, damaging the economy longer. Coronavirus isn't a biological weapon. Partisan stupidity can be weaponised, though.
- roenxi 6y ago
  That is a totally stupid strategy that relies on a series of connections that don't make any sense and could just as easily be good decisions for America that result in the country doing better than normal. And it isn't Russia making partisan issues partisan - it is the action of the partisans themselves. It would be cheaper and more effective to bribe key officials in government. There was a conspiracy about Jeff Epstein running his paedophile ring in aid of ... Mossad or the Illuminati or whoever. It would be a bit more expensive and a lot more effective to make that conspiracy theory a reality rather than rely on bots to 'sow division' - whatever that is supposed to do that the politicians can't manage on their own.

deleted 6y ago

[deleted]

swagatkonchada 6y ago

Twitter needs to be a lot better at detecting bots.