8 ms·
New leak shows NSA harvests To, From, and Bcc lines of e-mail data
- VladRussian2 13y agoyep, and due to a bug in the perl script, it harvests all the lines to the next From.
- pfortuny 13y agoYes, it is probably a regexp bug, like /^ $/ Someone pressed the space key once at the wrong place...
- webwanderings 13y agoHow the heck do you track Ad interacting habits through just an IP address? I call BS on that particular paragraph.
- schmidp 13y agoThey could track if you reply to an ad or, if they also track your IP connections, see if you click on a link in an email containing an ad.
- webwanderings 13y agoThe IP information is a common and general location, usually of your nearest Telecom tower. IP address in itself does not lead to your Internet device. So the paragraph is still inaccurate if the Ad agency is tracking through IP. Your IP was always naked and available to anyone you send email to (through the headers).
- tmzt 13y agoA couple of things: 1. That is true of some cell-based mobile data solutions, but others use an actual IPv4/v6 address assigned to each mobile session. 2. Some popular webmail systems hide the source IP address, while others include a special header with the data.
- vidarh 13y agoBut your actual IP address still does not tell them your precise location. Unless they also separately get a log of what cell tower handled traffic for what IP addresses, they'd still be left with only the location of your local internet provider.
- tmzt 13y agoI might have been mislead by the use of IP, the handset is what is tracked and it usually is more precise than just what tower the device is associated with. It can include measurements taken from multiple towers, which can be derived from data needed for CDMA to even function, or it can include government mandated E911 information which is usually derived from an internal 4-channel GPS receiver. In theory this is only supposed to be used for E911 functions when the handset is in contact with a PSAP, but we have no way as the public to know what information from these systems are collected and stored or for how long.
- webwanderings 13y agoAn IP on a mobile device is not as same as an IP on desktop. As far as desktop is concerned, unless ISPs are willing to track your exact location for the government, there is no way anyone can pinpoint your exact location through the public IP address.
- dreamfactory 13y agovia ad agencies?
- webwanderings 13y agoAnd what does the Ad agencies got to do with this NSA mess? Or in other words, is there any disclosure in this article of any Ad agency doing NSA's job?
- SoftwareMaven 13y agoIt's a stretch, admittedly, but most email marketing interactions have campaign-specific email addresses to follow conversions. They would certainly be able to see what companies you've previously interacted with by the fact that you are receiving emails from company.com.
- ChrisAntaki 13y agoNSA harvests the entire email. Source: http://www.wired.com/threatlevel/2007/05/mark_klein_docu/ http://www.wired.com/threatlevel/2007/05/mark_klein_docu/
- chakalakasp 13y agoIf I'm not mistaken, isn't fiber optic splitting done with a glass... Prism?
- deleted 13y ago[deleted]
- mpyne 13y agoCareful, you're going down the Glenn Beck-style "OLIGARHY" route.
- flyinRyan 13y agoThe person in this thread that is closest to a Fox News cast member would be the local HN government apologist. Namely, yourself.
- leecbaker 13y agoI talked to a company that did this kind of work, and they used tiny MEMS mirrors made of gold plated silicon. They wouldn't tell me much about their customers, but they specialized in switching and splitting high speed data at the physical layer. http://www.glimmerglass.com/solutions/cyber-security-and-lawful-interception/ http://www.glimmerglass.com/solutions/cyber-security-and-law...
- astrodust 13y ago"You know, in case you want a redundant signal for...redundancy...reasons."
- mortehu 13y agoIt's done by bending the fiber, causing leaks. The signal has a single wavelength, so a prism would serve no purpose.
- barredo 13y agoIs there any numbers on the amount of hard drives NSA have?
- joe_the_user 13y ago"How Much Is A Zettabyte?" http://foxnewsinsider.com/2013/06/07/how-much-zettabyte-nsa-utah-facility-can-hold-immense-amount-data http://foxnewsinsider.com/2013/06/07/how-much-zettabyte-nsa-...
- barredo 13y ago> The center will reportedly be able to store five zettabytes worth of information I am sure some day in the future there will be MicroSD cards with this storage capacity. But now it is just mindblowing ~5.500.000.000.000 Gigabytes
- moneyrich2 13y agoWhat if it was a database that stores databases. mindfuck
- DanBC 13y agoThis website makes the unsourced claim about Yottabytes that "You can compare it to the World Wide Web as the entire Internet almost takes up about a Yottabyte." (http://whatsabyte.com/ http://whatsabyte.com/) 1 Zettabyte = 1,073,741,824 terabytes. This Quora answer says that total HD supplied numbers worldwide in 2011 was 6,800,000 units. (http://www.quora.com/How-many-hard-drives-are-produced-each-year http://www.quora.com/How-many-hard-drives-are-produced-each-...) I find the 5 zettabyte figure hard to believe.
- aaronblohowiak 13y agoTapes, not disks.
- DanBC 13y ago
- zerohp 13y agoIt's been a while since I worked with email headers and smtp, but I don't think the Bcc header actually exists in transit. The mail user agent and/or the mail submission agent remove it. They could reconstruct this information from the graph.
- ChrisAntaki 13y agoConsidering they have an upstream wiretap on all US internet traffic, it seems possible they could determine the BCC.
- XorNot 13y agoWhy bother? Google knows the BCC line - your email provider by definition has to in order to BCC the message, and it very much depends on them to properly scrub it out as well (I know for a while it was pretty trivial to see BCC's from most providers).
- jlgreco 13y agoApparently not always: "In the second case, recipients specified in the "To:" and "CC:" lines each are sent a copy of the message with the "BCC:" line removed as above, but the recipients on the "BCC:" line get a separate copy of the message containing a "BCC:" line. (When there are multiple recipient addresses in the "BCC:" field, some implementations actually send a separate copy of the message to each recipient with a "BCC:" containing only the address of that particular recipient.)" http://en.wikipedia.org/wiki/Blind_carbon_copy#Visibility http://en.wikipedia.org/wiki/Blind_carbon_copy#Visibility Depending on combination and location of the MUA and MSA, it is plausible that the NSA was able to get full BCC lines.
- tracker1 13y agoIt depends on if you get the envelop. The envelop is a list of recipients for a given server to handle... the headers of the actual email are a convenience for display, not used for relay/forward/delivery.
- deleted 13y ago[deleted]
- throwaway10001 13y agoSo who the F can we trust? All those denials from everyone and now we see this, which I kinda suspected since Verizon was ordered to hand over the same for phone calls.
- nathas 13y agoYou can't trust anyone. You can trust protocols and implementations and proven math. That said, I would bet good money they already have working quantum computers, in which case current crypto may have quite a few problems.
- philh 13y ago> I would bet good money they already have working quantum computers How much, at what odds, and under what conditions of settling?
- chetanahuja 13y agoThe odds depend on whether you know the odds or not.
- nostrademons 13y agoThat's pretty restrictive. I know a large number of scientists and computer programmers who believe the only thing you can trust are protocols and math, and they're usually completely unable to function outside of the narrow domain of their work. I think a better philosophy is to trust that people will behave according to the incentives and information available to them. So if there is an organization out there, you can bet that it will act to expand the scope of the organizations' actions, because organizations that don't do this eventually get replaced by ones that do. If the organization is tasked with keeping tabs on all of America's adversaries, you can bet that they will see adversaries wherever possible to preserve a purpose for the organization.
- zachrose 13y ago"Institutions will try to preserve the problem to which they are the solution." -- Clay Shirky http://www.kk.org/thetechnium/archives/2010/04/the_shirky_prin.php http://www.kk.org/thetechnium/archives/2010/04/the_shirky_pr...
- rjbwork 13y agoSo just use the CC field, problem solved!
- hammerzeit 13y agoThis headline is misleading. It implies the program is still ongoing, where the original article clearly states that the program was shut down 2 years ago. Moreover, after a lot of cynical complaining about Obama not being meaningfully different than previous administrations, it's worth noting that Obama was the one to shut this down. I'm not interested in reflexively defending the government or Obama but we still need to pay attention to the facts at hand.
- Terretta 13y agoPrograms are often shut down when superseded by new or better tools. In any case, both the WaPo and the Guardian article on the same leak says metadata collection is ongoing: A senior administration official queried by the Washington Post denied that the Obama administration was "using this program" to "collect internet metadata in bulk", but added: "I'm not going to say we're not collecting any internet metadata." http://www.guardian.co.uk/world/2013/jun/27/nsa-data-mining-authorised-obama http://www.guardian.co.uk/world/2013/jun/27/nsa-data-mining-...
- hammerzeit 13y agoDon't you think if there was evidence that the government was still harvesting CCs,BCCs, etc en masse, that the article would have made this their central claim? The vast majority of the article is about how this stopped in 2011. One line implies they still collect "internet metadata" but not in bulk.
- rhizome 13y agoThe information that has come out over the last 12 years has taught us that they will change the name of the program, calling "this program" "shut down," while continuing with what they were already doing under another name, sometimes under different authorities (but always the same conduct).
- lemming 13y agoIf you read the article carefully, or even better the original Guardian article: http://www.guardian.co.uk/world/2013/jun/27/nsa-online-metadata-collection http://www.guardian.co.uk/world/2013/jun/27/nsa-online-metad... you'll find claims that while the original program was shut down something very like it although probably larger in scope is still ongoing - details are murky though. I instinctively like Obama, but I'm forced to admit that his policies on national security are by any objective means worse than his predecessor. He's just more eloquent when he talks about them.
- mbateman 13y agoWhat exactly is it supposed to mean that the NSA intercepts only data with one "foreign end"? That it intercepts all data that crosses e.g. a transatlantic cable? Or that it scans the IP header of absolutely everything and grabs anything with a non-US IP as either source or destination? Or something else?
- guelo 13y agoThat's where the "minimization" documents from last week come in. They actually collect everything, but then there is some kind of filter at the collection point that is supposed to remove any communication that they are certain is american-to-american. But that filter also has exceptions for things like encrypted messages or pretty much anything else they are interested in. They make the filter as loose as they can while still being able to maintain some deniability that they don't collect domestic communication.
- mpyne 13y agoThat really depends on the details of what's capable with their collection equipment. Transoceanic cables are obvious, but there are also satellite-satellite links that are possible that could terminate right in Iowa for all we know, that the NSA wouldn't be able to retain unless/until it made it to one of the Tier 1 ISPs (they might use geolocated IPs for this). And as the other comment mentions, the minimization procedures seem arranged to blacklist and discard only that data which is clearly US<->US so you could definitely end up with your data being socked away.
- deleted 13y ago[deleted]
- moneyrich2 13y agoCan I ask how? How do you have 75% of the traffic or 75% of the servers (as the article states), how the hell is that logistically possible?
- grey-area 13y agoGCHQ in the uk have been doing it for most uk traffic, and it seems the approach is to store full data as long as they can, and store headers for longer. All the content is stored for 3 days, then the headers are kept for 30 days, and shared with other agencies like the NSA, who may well keep it all indefitely if they have enough storage available. They probably do some early filtering to keep it manageable, removing duplicate content, unwanted videos etc. and the headers and metadata are probably not that large. Before reading the GCHQ docs I wouldn't have found this claim credible... If they are not collecting every communication in the world, you can be sure it is not from lack of ambition to do so. In the words of General Alexander: “Why can’t we collect all the signals all the time?” the N.S.A. director was quoted as saying. “Sounds like a good summer project for Menwith." Which is a worryng thought when you realise the implications of this ambition. We used to think that only a god could be omniscient, but that is the current ambition of our intelligence services and politicians.
- dfc 13y agoIf you don't want to use that awful doc viewer: wget http://s3.documentcloud.org/documents/719116/pages/doc03-p1-large.gif ... wget http://s3.documentcloud.org/documents/719116/pages/doc03-p52-large.gif The last time the guardian had a document up and I provided these gifs someone replied with a pdf copy. I am unsure of how to get the pdf from documentcloud. So feel free to post a pdf link and please explain where you get the URL from
- mh- 13y agoI took a guess and got it right (dont sue me plz DocumentCloud): http://s3.documentcloud.org/documents/719116/doc03.pdf
- dfc 13y agoI regret that I have but one upvote to give you. Many thanks. I swore that last time the pdf had no relation to the gif names.
- cliffu 13y agoWhat you just did is no different than the felony they got weev for. Edit: well, I guess you didn't falsify any headers, though.
- speeder 13y agoNow make a script to download several of those at once, and you are Aaron class felon... This make me depressed :/
- LukeShu 13y agoSomeone else posted the link to the PDF, but for downloading sequences: wget http://s3.documentcloud.org/documents/719116/pages/doc03-p{1..52}-large.gif
- eslaught 13y agoIs email traffic typically encrypted between major providers? E.g. could a network attacker, located between Google and Microsoft, intercept unencrypted traffic between gmail and hotmail addresses?
- SoftwareMaven 13y agoIt depends. Bigger providers like Google typically prefer to use TLS, but there are a great many smaller providers who do not. In order to keep everything flowing, there has to be the ability to fall back to unencrypted messages.
- pjqwdpjoqw 13y agoIt is often encrypted, but since the encryption is negotiated using STARTTLS it can easily be stripped by an active attacker. It works fine against passive attackers.
- kintamanimatt 13y agoSMTP relay is almost always unencrypted. The client that connects to the SMTP server may connect via an encrypted connection though, but that's mostly to prevent snooping on the client's local network.
- astangl 13y agoThey discontinued the program to save just the 3 headers because now they've got other programs that save the entire email message. And phone calls, and text messages and tweets, etc.
- jccc 13y agoWhat if the near-term result of all these revelations is that it just becomes the new normal? Is it necessarily such a bad thing for the snoops in the shadows if they know people will eventually just get used to it? After all nothing really feels different day-to-day so ... meh.
- raintrees 13y agoData is being collected on all people. All people are guilty of something. When the time comes, a case will be built for the person currently under scrutiny. http://marginalrevolution.com/marginalrevolution/2013/06/no-one-is-innocent.html http://marginalrevolution.com/marginalrevolution/2013/06/no-...
- bobsil1 13y agoIf you're a techie, knowing that every bit of data you collect from customers will eventually end up in Utah-- you have a duty to either collect the minimum data possible or encrypt both transmission and storage and demand a warrant for access.
- kyzyl 13y agoQuestion: Does anyone here understand where exactly systems that incorporate a zero-knowledge architecture fit into the recently illuminated legal framework (re: warrants, etc.)? i.e. if I implement my service so that I don't have the keys and cannot reasonably obtain them, what does that mean for my users and their data, presuming the data is stored in the US? Juicy example: Lastpass. (I am not affiliated with Lastpass.) I'm sure this has already been discussed on HN recently, but with the dizzying number of PRISM/Snowden/Leaks/Wiretapping threads flying around it's difficult to keep up.
- mike-cardwell 13y agoLastpass can access your decrypted password vault if they are compelled to. All they have to do is send you some modified JavaScript which steals your password/key. They're certainly worried about persistant XSS attacks being used to gain access to peoples vaults. There's nothing stopping them performing one of these attacks themselves, targetted to a specific user. If you think this is unlikely, look up Hushmail being compelled to send modified java applets to their users to steal their keys. It has been done before. So yeah, if the US government wants access to a list of all of your accounts, when you logged in to them, what IPs you logged in with and your usernames and passwords, they'd probably be quite pleased to find out you're using Lastpass
- kyzyl 13y agoWell Lastpass was just an example. With enough effort any service can be hacked, but if the bar is high enough it means it's more likely that the US gov can't/won't do it en masse. I would note that Lastpass allows you to implement Google Authenticator/Yubikey/One-time-pad/Biometrics to help secure your key against a simple XSS attack. I think that probably qualifies as 'setting the bar high.' In any case, my question was more towards the _legal_ situation, not the technical. Suppose you have a near-perfect no-knowledge system, how does the US gov view that entity? At least in theory, if they cannot reasonably force the company to give up the keys, what can they legally do? Can they force the company to shutdown? Can they make the company force users off the service in an attempt to get them into a less secure realm? Are such systems even legal in the current climate? Of course there is always a way to hack it, and the $5 wrench will beat anything (pun intended), but as far as the mass surveillance mandate goes those options are probably out.
- jameshart 13y agoI think we would all forgive the intrusion if, as a side effect of this program, NSA was using this data to feed a spam filtering service to which we could all subscribe.
- raintrees 13y agoNo, for me it would have to be Daniel Suarez' Daemon. Then I could skip over the forgive part, since many alphabet agencies may be rendered obsolete... And all spammers would have ceased or been desisted. Good story, if you haven't yet read/heard it.
- poster69 13y agoThis thread is big meaningless distraction... The main point is: You Are all being illegally spied on The land of the free is a big lie.
- Aqueous 13y agoHere's an idea: If you don't want to make an anxious public even more anxious, don't name your NSA surveillance program "EvilOlive." Or really, anything starting with 'evil.'
- askimto 13y agoAnyone ever hear the rumor that the reason why Google pulled out of China was because Chinese hackers had tapped into a feed of all email metadata? I heard it included subject. This news made me immediately think of that rumor.
- deleted 13y ago[deleted]
- tonyplee 13y agoWill google pull out of US now? :-) Can't fight big government. China or US, really doens't matter.
- deleted 13y ago[deleted]