3 ms·
> The monitoring seems at odds with claims from both Twitter and Dataminr that neither company would engage in or facilitate domestic surveillance following a s
by throwawizzle 6y ago
> The monitoring seems at odds with claims from both Twitter and Dataminr that neither company would engage in or facilitate domestic surveillance following a string of 2016 controversies. Twitter, up until recently a longtime investor in Dataminr alongside the CIA, provides the company with full access to a content stream known as the “firehose” — a rare privilege among tech firms and one that lets Dataminr, recently valued at over $1.8 billion, scan every public tweet as soon as its author hits send. Both companies denied that the protest monitoring meets the definition of surveillance.
Dataminr is not the only company with access to the firehose that has law enforcement and military customers (or customers of customers, who would even know?) doing whatever they please with it.
I believe that Twitter has cracked down on this recently, or is at least beginning to ask Twitter data customers to restrict some of their customer's usage (or remove access entirely). But all of this after the fact.
Also, if you deleted something off Twitter and you think it's gone from every downstream data warehouses that captured it seconds after you tweeted, and also the people with access to those warehouses that copied it to their own storage... well, then I have a bridge to sell you.
- falcolas 6y agoLast I heard, there were less than 10 companies with access to the full firehouse (which has every tweet). Most only get a partial feed, if that. Disclaimer: Used to work for Dataminr; was laid off.
- throwawizzle 6y agoThere is partial firehose access, that's true. That's probably much more common than full access. I bet you could get some surveillance done with only partial access, though. And there is also such a thing as imports, which I think lets you pull historical data based on terms you want to find. I don't know if that gets a sample, or everything that matches.
- ashtonkem 6y agoThat’s true, but doesn’t mean quite what you’re implying. The full firehose was originally given to a small number of companies, including Gnip (eventually purchased by Twitter). What Gnip did was provide filtering for the firehose as well as some historical searches; this in theory would allow companies to receive in soft real time every tweet that was relevant to their search query. The result is that there are tons of people who don’t have access to the full firehose, but effectively have the ability to receive every single tweet that’s relevant to their search query. It wouldn’t be very hard to find every tweet relevant to a protest, as this is superficially quite similar to finding tweets relevant to a brand or marketed event. That being said, this isn’t a product that you can just sign up for and use. You had (my contacts no longer work at twitter) to negotiate a contract with Twitter directly to get access to this, pay large sums of money, and go through a fairly high touch sales and support process.
- mastazi 6y ago> Dataminr is not the only company with access to the firehose Was Twitter an investor in those other companies though? I believe this is the differentiator that the author of the article is trying to bring up.
- throwawizzle 6y agoI can't speak for all of them, but I'm pretty sure they're not an investor in the company I am familiar with. That's a fair point to bring up.