3 ms·

Note that, contrary to what the HN title implies, Twitter doesn't say it specifically targeted those hashtags or hid them entirely. The testimony mentions them

Note that, contrary to what the HN title implies, Twitter doesn't say it specifically targeted those hashtags or hid them entirely. The testimony mentions them as examples of tags that were associated with a significant number of bot/spam tweets.

SrslyJosh 9y ago
Indeed. The title is misleading. From the PDF: "In the two months preceding the election, around 57,000 users posted approximately 426,000 unique Tweets containing variations of the #PodestaEmails hashtag. Approximately one quarter (25%) of those Tweets received internal tags from our automation detection systems that hid them from searches. As described in greater detail below, our systems detected and hid just under half (48%) of the Tweets relating to variants of another notable hashtag, #DNCLeak, which concerned the disclosure of leaked emails from the Democratic National Committee. These steps were part of our general efforts at the time to fight automation and spam on our platform across all areas."
- creaghpatr 9y ago
  I'm interested to know how they determined half of all #DNCleaks tweets were spam and how that compares to other trending topics both political and apolitical. Seems like they are hiding behind the anti-spam stance without detailing how their system determines that. Removing almost 50% of communication on a topic is a pretty dramatic step, especially during an election.
  - teraflop 9y ago
    Yes, but it would be an equally dramatic step to leave their normal anti-spam system in place, but then specifically exempt a particular hashtag from filtering. But of course you're right; the core problem is that the filtering process itself is completely opaque and inscrutable. We just have to take Twitter's word that it doesn't introduce explicit or implicit biases.
  - spdustin 9y ago
    In my experience analyzing these sorts of things for my skunkworks project, the "source" (client or app) and too-quick-to-be-human retweet brigades are dead giveaways, and I'd imagine they're part of this algorithm. I also started tracking "visual hashes" of avatars (blurred and reduced to a 32x32 grid) in an effort to find re-used or stock profile pics, which has shown some promise as a spammer/bot/brigade signal. As well, existing known signals such as "user with default background color" or "no verified email address" could be part of it. For example, IFTTT was recently shown to be used by a pro-Trump "retweet brigade" attempting to boost the "Uranium One" campaign with old "RT-style" retweets that look like new tweets but all have the same text. That group wasn't terribly smart, since they all looked the same, and were broadcast in relatively short order by hundreds of accounts. I know these are guesses; Twitter likely won't ever share their secret sauce recipe.
cavanasm 9y ago
Yeah, the current title seems pretty misleading / politically slanted. The actual content of the article is a lot more. "Twitter's Testimony before the Senate Judiciary, Subcommittee on Crime and Terrorism" or something similar might make more sense. It's a 15 page document of Twitter's testimony, not a news article about the contents of that testimony.
sctb 9y ago
Yes, we've updated the title from the submitted “Twitter testifies to hiding #DNCLeak and #PodestaEmails during election”.
- iaw 9y ago
  I hit "flag" and the title was already changed.
gre 9y ago
Their filter was more than just botspam. Twitter purposely removed trending pro-Bernie and anti-Hillary hashtags during the primary.
- koolba 9y ago
  I wouldn’t put it past them but do you have evidence of that? I didn’t see it in this testimony.
  - gre 9y ago
    Other than following the election very closely and observing, no I don't have evidence. Tags like #whichhillary were very popular at times yet not in the trending tags. If you caught a tag before the censors you could see it trending before they removed it. Again, just observation, sadly. Are their trending hashtags included in the firehose data or otherwise recorded? Someone could go back and analyze it if they had the data. edit: Here you go. Also google for it to find more links. Feb 2016, well before the primary was over. https://www.dailykos.com/stories/2016/2/26/1491618/-Twitter-Exec-Censors-WhichHillary-in-advance-of-Sunday-Fundraiser-Key-Primaries https://www.dailykos.com/stories/2016/2/26/1491618/-Twitter-...

SrslyJosh 9y ago

Indeed. The title is misleading. From the PDF: "In the two months preceding the election, around 57,000 users posted approximately 426,000 unique Tweets containing variations of the #PodestaEmails hashtag. Approximately one quarter (25%) of those Tweets received internal tags from our automation detection systems that hid them from searches. As described in greater detail below, our systems detected and hid just under half (48%) of the Tweets relating to variants of another notable hashtag, #DNCLeak, which concerned the disclosure of leaked emails from the Democratic National Committee. These steps were part of our general efforts at the time to fight automation and spam on our platform across all areas."

creaghpatr 9y ago
I'm interested to know how they determined half of all #DNCleaks tweets were spam and how that compares to other trending topics both political and apolitical. Seems like they are hiding behind the anti-spam stance without detailing how their system determines that. Removing almost 50% of communication on a topic is a pretty dramatic step, especially during an election.
- teraflop 9y ago
  Yes, but it would be an equally dramatic step to leave their normal anti-spam system in place, but then specifically exempt a particular hashtag from filtering. But of course you're right; the core problem is that the filtering process itself is completely opaque and inscrutable. We just have to take Twitter's word that it doesn't introduce explicit or implicit biases.
- spdustin 9y ago
  In my experience analyzing these sorts of things for my skunkworks project, the "source" (client or app) and too-quick-to-be-human retweet brigades are dead giveaways, and I'd imagine they're part of this algorithm. I also started tracking "visual hashes" of avatars (blurred and reduced to a 32x32 grid) in an effort to find re-used or stock profile pics, which has shown some promise as a spammer/bot/brigade signal. As well, existing known signals such as "user with default background color" or "no verified email address" could be part of it. For example, IFTTT was recently shown to be used by a pro-Trump "retweet brigade" attempting to boost the "Uranium One" campaign with old "RT-style" retweets that look like new tweets but all have the same text. That group wasn't terribly smart, since they all looked the same, and were broadcast in relatively short order by hundreds of accounts. I know these are guesses; Twitter likely won't ever share their secret sauce recipe.

cavanasm 9y ago

Yeah, the current title seems pretty misleading / politically slanted. The actual content of the article is a lot more. "Twitter's Testimony before the Senate Judiciary, Subcommittee on Crime and Terrorism" or something similar might make more sense. It's a 15 page document of Twitter's testimony, not a news article about the contents of that testimony.

sctb 9y ago

Yes, we've updated the title from the submitted “Twitter testifies to hiding #DNCLeak and #PodestaEmails during election”.

iaw 9y ago
I hit "flag" and the title was already changed.

gre 9y ago

Their filter was more than just botspam. Twitter purposely removed trending pro-Bernie and anti-Hillary hashtags during the primary.

koolba 9y ago
I wouldn’t put it past them but do you have evidence of that? I didn’t see it in this testimony.
- gre 9y ago
  Other than following the election very closely and observing, no I don't have evidence. Tags like #whichhillary were very popular at times yet not in the trending tags. If you caught a tag before the censors you could see it trending before they removed it. Again, just observation, sadly. Are their trending hashtags included in the firehose data or otherwise recorded? Someone could go back and analyze it if they had the data. edit: Here you go. Also google for it to find more links. Feb 2016, well before the primary was over. https://www.dailykos.com/stories/2016/2/26/1491618/-Twitter-Exec-Censors-WhichHillary-in-advance-of-Sunday-Fundraiser-Key-Primaries https://www.dailykos.com/stories/2016/2/26/1491618/-Twitter-...