3 ms·

I noticed two major things in this article. First, Twitter can't do what the students are doing - an informal, unreliable service going "hey , bot!" is fine fo

I noticed two major things in this article. First, Twitter can't do what the students are doing - an informal, unreliable service going "hey , bot!" is fine for some random people to offer, but the site owner can't chance high error rates on public statements. Twitter keeps suggesting they're doing something like this internally, which would make sense. Second, the article offers no real evidence that this service works. It's a machine learning problem with no proven classifications in the dataset, so the learner will at best reproduce the developers opinions of what a Russian bot looks like. I'll play around with the tool, but my initial guess is that it just learned "patriotic, inflammatory language, retweets only" as "bot", which isn't a breakthrough anyone can actually act on.

rohanphadte 9y ago
Hey, I'm a creator of the tool written in the article. The model looks at hundreds of features of each profile, not just a few features such as patriotic, inflammatory language, and retweets as you suggested. As a result, the model has predicted profiles with very sketchy behavior, such as accounts of normal people that were compromised and become political propaganda accounts a a few years later. We wanted to bring up these accounts and the analysis behind it so we could show the techniques behind these organizations who run these bot-like accounts. We offer our own analysis of these accounts here: https://medium.com/@robhat/an-analysis-of-propaganda-bots-on-twitter-7b7ec57256ae https://medium.com/@robhat/an-analysis-of-propaganda-bots-on...
- Bartweiss 9y ago
  Hey, thanks so much! The Wired take definitely left me cynical; it's all too easy to write a human-interest piece about ML approaches that don't actually work. But this is a much more concrete explanation of results, and I'm intrigued. If you don't mind, could you offer any more clarification on your test/training set? I see the Medium piece talks about wanting to avoid selection bias from hand-classification, but the Wired summary just described hand-selecting 100 "ground truth" bots and then adding their followers. How did you know the followers were also bots? And how did you try to ensure the bots you selected were a reasonable sample?

rohanphadte 9y ago

Hey, I'm a creator of the tool written in the article. The model looks at hundreds of features of each profile, not just a few features such as patriotic, inflammatory language, and retweets as you suggested. As a result, the model has predicted profiles with very sketchy behavior, such as accounts of normal people that were compromised and become political propaganda accounts a a few years later. We wanted to bring up these accounts and the analysis behind it so we could show the techniques behind these organizations who run these bot-like accounts. We offer our own analysis of these accounts here: https://medium.com/@robhat/an-analysis-of-propaganda-bots-on-twitter-7b7ec57256ae https://medium.com/@robhat/an-analysis-of-propaganda-bots-on...

Bartweiss 9y ago
Hey, thanks so much! The Wired take definitely left me cynical; it's all too easy to write a human-interest piece about ML approaches that don't actually work. But this is a much more concrete explanation of results, and I'm intrigued. If you don't mind, could you offer any more clarification on your test/training set? I see the Medium piece talks about wanting to avoid selection bias from hand-classification, but the Wired summary just described hand-selecting 100 "ground truth" bots and then adding their followers. How did you know the followers were also bots? And how did you try to ensure the bots you selected were a reasonable sample?