59 ms·

XKeyscore: NSA program collects 'nearly everything a user does on the internet'

• ceautery 13y ago
  I wonder how they store all that. Surely a side benefit of this could be NSA contributions to CS journals about database techniques. Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? I think it's more likely this claim is made simply to generate FUD in the general population.
  • quasque 13y ago
    Another benefit could be realised in the future if historians and linguists manage to get access to all this data. I imagine that researchers of the social sciences would end up enjoying the same sort of large-scale collaborative projects that particle physicists or genomic bioinformaticians currently have with their huge datasets.
  • nikcub 13y ago
    Hadoop[0] and OpenStack[1] I think the era of government being far ahead of commercial tech capability is over. The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors. [0] http://online.wsj.com/article/SB10001424127887323495604578535290627442964.html http://online.wsj.com/article/SB1000142412788732349560457853... [1] http://www.youtube.com/watch?v=NgahKksMZis http://www.youtube.com/watch?v=NgahKksMZis
    • 286c8cb04bda 13y ago
      The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors. There are a lot of fingers in that pie. Oracle, for example, has a National Security Group, whose job is to come up with "solutions" and then try to sell them to three-letter-agencies.
  • coldtea 13y ago
    >Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? Why would they? They deal with ISPs, backbones and such.
  • aryastark 13y ago
    > I wonder how they store all that. Well, now they have a massive data center in Utah. That's most likely where it's all going today. Standard open source tools are all you really need. > Wouldn't they show up on, say, Sandvine's Internet traffic reports? No. If some script kiddie/hacker type installs a packet sniffer and logs all your traffic to your ISP, that won't show up anywhere. Traffic goes somewhere. You're sending packets out. Merely logging packets is entirely passive and undetectable.
• codex 13y ago
  The Guardian strongly implies this system is used to intentionally target US citizens in violation of the law, but then admits that would be "illegal." I wonder if the leaked presentation touches on this point.
  • jabbernotty 13y ago
    The Guardian doesn't 'admit' anything (it wasn't hiding anything in the first place), and legality doesn't predict whether actions are being taken or not. >I wonder if the leaked presentation touches on this point. That seems unlikely to me, as this is a technical presentation.
• mladenkovacevic 13y ago
  It seems this was meant to be declassified in 2032.. I guess by then they were hoping this would be so institutionalized and pervasive as to be the norm. Also I wonder to what extent this is really used to hunt terrorists down and how much of it is used to gain political or economic advantages over other countries.
  • madaxe 13y ago
    Secrecy is like copyright - keeps on getting extended, or the documents just get "mislaid" long before the declassification date.
• eksith 13y ago
  That UI looks awfully similar to a theme I've seen used in SharePoint Portal Server. I hope that's not what they use for the front end, but I wouldn't put it past them.
  • dualogy 13y ago
    Having worked with SharePoint extensively for years, I'm highly confident this isn't the case. It's just plain-old crappy custom-coded HTML4 forms from what I can tell and similar colors, so one could easily think so. But at closer look, it definitely not any of the SP versions 2003, 2007, 2010 or 2013. Maybe 2001 but I highly doubt even that.
  • duncans 13y ago
    Pretty sure it isn't - some of it looks like old ExtJS if anything. The more worrying thing is that they're still apparently using IE6. [EDIT: OK the presentation is from 2008 but still!]
    • VikingCoder 13y ago
      This, to me, is like showing the alleged UI on an Apple IIc: It just about proves that it's not true, to me.
• ck2 13y ago
  Imagine if storage limitations weren't holding back the NSA. Those 60TB density HAMR[1] drives that are due in 2016 are really going to take invasive to a whole new level. [1] http://storageeffect.media.seagate.com/files/2012/03/perphamr2.gif http://storageeffect.media.seagate.com/files/2012/03/perpham...
  • sixothree 13y ago
    20 TB per day is nothing these days.
  • javert 13y ago
    Imagine if the American people weren't holding back the NSA. Oh, wait...
  • TsiCClawOfLight 13y ago
    mmh... but maybe the military is also ahead in that area? Maybe DARPA has 1TB GB/s drives?
    • vidarh 13y ago
      I don't quite know what you intend "1 TB GB/s drives" to mean. But note that you can buy off the shelf PCIe cards with SSD's mounted that will give you 1TB storage and an aggregate read bandwidth of more than 1GB/sec today. I've got three sitting in various servers. They're expensive, and frankly for the future I'll rather get a couple of extra SATA III controllers and get multiple "regular" SSDs on separate controllers for that reason, but they're available. For NSA style data collection, though, the collection is trivially to do in parallel: Hash all keys to a "virtual bucket", and hold a map of virtual buckets to physical servers. Then when you want more capacity, you add some physical servers, reassigns some of the virtual buckets from other physical servers to the new ones, and synchronises any old data (given that NSA claims they could only hold the full data stream for three days, you don't even need the hassle of moving data, just make collection on different days map to different virtual buckets, so that on day one you "just" reassign virtual buckets the content of which is being expired on the old servers anyway, on day two, the next set etc. - you maintain full spread of read/write traffic by ensuring that in normal operation all servers have an even spread of "day 1", "day 2" and "day 3" buckets). It's amusing they see storage as an issue, but of course this was in 2008. Today I have 6TB in my home NAS, and my perfectly off the shelf tower case can easily fit 40TB+ with current size harddisks (though I doubt the noise would make me popular at home).
• mtgx 13y ago
  Nice pre-emptive "attack" by Greenwald today, just before the NSA hearings.
  • barylen 13y ago
    I was worried the leaks may have peaked a bit early, but this was very well timed.
    • griffordson 13y ago
      The House hearing was canceled several days ago to make time for the House Democrats to meet with Obama this morning. As far as I know they have not been rescheduled yet and I will be surprised if they happen before the August recess. Although Glenn Greenwald did say yesterday he hopes they get rescheduled in the next 24 - 48 hours.
      • griffordson 13y ago
        Here is the video from the Senate Judiciary Committee hearing on the subject today though: http://www.c-spanvideo.org/program/FISAS http://www.c-spanvideo.org/program/FISAS
    • w_t_payne 13y ago
      So it seems.
  • falk 13y ago
    Greenwald said they cancelled the hearing on Twitter yesterday. Do you know if they rescheduled it yet?
• hga 13y ago
  This bit both somewhat limits the impact and makes Greenwald et. al.'s claims that most everything is being Hoovered up a lot more credible: "The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours."" Of course, as the article goes on to detail, anything that's found to be of interest in that window can be saved permanently, and NSA analysis do that a lot.
  • sixothree 13y ago
    >"To solve this problem, the NSA has created a multi-tiered system that allows analysts to store "interesting" content in other databases, such as one named Pinwale which can store material for up to five years." It does not describe "interesting". Maybe metadata, encrypted sessions etc? Any conversations in threads linking to these articles?
  • spahl 13y ago
    But this was written in 2008. Storage capacity could/must have increased massively since then.
    • mpyne 13y ago
      But so has the volume of data that NSA would have to retain.
      • lukifer 13y ago
        ...and so have the compression and filtering tech, I'm sure. If you strip out most attachments, the average email message is incredibly tiny when gzipped.
  • haakon 13y ago
    Notice that the presentation is from 2008 - a lot changes in five years.
• mjfl 13y ago
  Well then they are going to need that big data center. That is an unimaginable amount of data...
  • deleted 13y ago
    [deleted]
  • cpursley 13y ago
    https://en.wikipedia.org/wiki/Utah_Data_Center https://en.wikipedia.org/wiki/Utah_Data_Center
  • sentinel 13y ago
    Yep. http://www.forbes.com/sites/kashmirhill/2013/07/24/blueprints-of-nsa-data-center-in-utah-suggest-its-storage-capacity-is-less-impressive-than-thought/ http://www.forbes.com/sites/kashmirhill/2013/07/24/blueprint...
• susi22 13y ago
  One of the screen shots: http://static.guim.co.uk/sys-images/Guardian/Pix/audio/video/2013/7/31/1375269238578/KS3-001.jpg http://static.guim.co.uk/sys-images/Guardian/Pix/audio/video... says: Top Secret Comm(?) REL() to USA, AUS, CAN, GBR, NZL confirming the previous suspicions that many other governments are on board. Der Spiegel actually has reported a few weeks back about XKeyscore [1] and that it is used by the BND (Germany's NSA). I.e. all this data is also available to the NSA equivalents of Australia, Candana, Great Britain and New Zealand. Many Americans trust their government (unfortunately), will they also trust the other governments? [1]: http://www.spiegel.de/international/world/german-intelligence-worked-closely-with-nsa-on-data-surveillance-a-912355.html http://www.spiegel.de/international/world/german-intelligenc... http://www.spiegel.de/international/germany/german-intelligence-agencies-used-nsa-spying-program-a-912173.html http://www.spiegel.de/international/germany/german-intellige...
  • packetlss 13y ago
    Not GER, it's GBR as in Great Britain http://en.wikipedia.org/wiki/UKUSA_Agreement http://en.wikipedia.org/wiki/UKUSA_Agreement
  • bnmathm 13y ago
    I'm pretty sure that's actually GBR - Great Britain.
    • colanderman 13y ago
      It definitely is; look at the actual (hi-res) presentation: http://www.theguardian.com/world/interactive/2013/jul/31/nsa-xkeyscore-program-full-presentation http://www.theguardian.com/world/interactive/2013/jul/31/nsa...
  • brown9-2 13y ago
    Those are the Five Eyes countries: http://en.wikipedia.org/wiki/Five_Eyes http://en.wikipedia.org/wiki/Five_Eyes
    • clicks 13y ago
• yread 13y ago
  This is brilliant, I love the screenshots: Foreignness factor: The person has stated that he is located outside the U.S. Human intelligence source indicates person is located outside the U.s. The person is a user of storage media seized outside the U.s. Foreign govt indicates that the person is located outside the U.s. Phone number country code indicates the person is located outside the U.s. Phone number is registered in a country other than the U.S. SIGINT reporting confirms person is located outside the U.S. Open source information indicates person is located outside the U.s. Network, machine or tech info indicates person is located outside the U.s. In direct contact w/ tgt overseas no info to show proposed tgt in U.S. It's quite easy to lose the protections of a U.S. citizen indeed!
  • sehugg 13y ago
    The person is a user of storage media seized outside the U.s. Interesting, so everyone who ever hit a MegaUpload link is potentially a foreign entity?
    • RyanZAG 13y ago
      Kind of puts into perspective why they would coordinate such a massive raid on Megaupload. The target may not have even been the data - merely seizing the data puts anybody who has accessed the megaupload website as an easy target.
      • netrus 13y ago
        You crossed the tinfoil line. Copyright infringement was sufficient motivation for the actions taken. The megaupload raid was not okay, but I am pretty sure Hollywood was behind it, not the NSA.
        • talmand 13y ago
          Just a few years ago this very article would cross the tinfoil line. Plus, don't be naive to think that the government wouldn't use an accusation of committing a crime to cover what they really want to do. For instance, need data from a server's hard drive? Accuse someone you know who has data on that server, not necessarily the data you want, to have an excuse to seize said hard drive and analyze it. Nope, turns out the accusation was incorrect, here's the hard drive back. Ah, is getting other data not covered by the warrant illegal? It just might be, but you can't complain if you don't know they did it and you probably don't have standing to sue over it to find out. Plus with authorities able to get double-secret warrants based on triple-super-secret laws issued by not-so-secret courts with "you can't even admit you were here" secret proceedings, how would anyone know in the first place? Remember, government agents have the authority to lie to you in an effort to complete their goals. Not that I'm saying the NSA was behind MegaUpload or anything, just saying it's feasible.
• martindale 13y ago
  Interesting; it appears someone failed to redact some data from the slides. In the Facebook chat example, the message is "to" 1536051595. Using the Facebook Graph API, we can gather information based on this ID: http://graph.facebook.com/1536051595 http://graph.facebook.com/1536051595 Which leads us to the Facebook profile (https://www.facebook.com/arash.gorjipour.5 https://www.facebook.com/arash.gorjipour.5) of an individual, real or contrived, named "Arash Gorjipour". His email address and phone number are all exposed in one of his uploaded photos: http://i.imgur.com/0UUk5cB.jpg http://i.imgur.com/0UUk5cB.jpg I wonder what the reason for this man being in these slides is.
  • junto 13y ago
    Probably picked at random. The screenshot 'test case' could have been you. We are all in the database after all.
    • junto 13y ago
      Of course, it would have been more amusing had the 'selector' been chosen as 'Barack Hussein Obama II'.
      • kazagistar 13y ago
        Forget amusing, that would have been a perfect action trigger: people are OK with privacy infringement on others, but when it happens to them, they are more likely to be upset. I suspect, or maybe just hope, that politicians are protected in some way from this. While it is unfair, at least it would mean less opportunities to extort or threaten lawmakers. Though, obviously, it would be best if we ALL were safe from that kind of crud.
    • iandanforth 13y ago
      I think it's a test account. The text string which reads something like 'does it still recognize me?' is very much like the kind of thing I'd type in my QA days when I was testing a new system. If I were putting together a deck on that system I'd also probably favor test data over live data, if for no other reason than it's easy to come by.
• api 13y ago
  At what point do the mathematical limits of data mining kick in here? How useful is all this information? I'm not an expert in this area of mathematics, so I could be wrong, but my impression is that as the haystack becomes larger the problem of false positives becomes more and more severe. As a data miner, what you want is the maximum number of "hits" (of whatever you're trying to hit) with the minimum number of misses and the minimum number of false positives. My impression is that this becomes progressively harder-- the golden region between too many false positives and too many false negatives becomes smaller and smaller and harder to hit. Eventually you either miss important hits, namely the next terrorist attack, or you get swamped with false positives that you have to manually investigate and rule out. I'd love someone who does know more here to chip in, but my personal suspicion is that this actually has a pretty huge pork angle to it. How much money are the contractors getting for building this stuff?
  • runn1ng 13y ago
    From the article The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours."
  • bane 13y ago
    so let's say there's a law that says "any American company doing business with a company that does business with a known terrorist organization will have a bad day" you don't need to use some kind of fancy data mining algorithm for this to work (generating false positives), you just need a ho-hum graph traversal algorithm and unbelievable amounts of graph data to generate "candidates for investigation". US Company A -> intermediate 1 -> known terrorist group B US Company A -> intermediate 2 -> known terrorist group B US Company A -> intermediate 3 -> known terrorist group B Each set of links is just one lead to investigate, but having a giant graph to work off of would make generating those leads simply. You might find out that intermediate 1 is a local falafel delivery place that "US Company A" uses for lunch catering. Can probably strike that one off the list. intermediate 2 is a utility (no choice but to use the local water monopoly), but intermediate 3 is a material supplier that employs several low level delivery guys from known terrorist group B, and the founder of the company is a cousin of the founder of known terrorist group B. So I'd wager it's not as simple as just running an algorithm and automatically sending out Skynet drones to blow things up. There's some kind of more subtle assessment being made, with the systems just providing help to the analysts.
• hannibal5 13y ago
  Just like suspected. If you use encryption like PGP, you become person of interest.
  • northwest 13y ago
    Re "Person of Interest":[0] If yesterday we were "conspiracy theorists" when we suspected things like XKeyscore, what are we today if we suspect things like "Person of Interest"-like programs? [0] http://www.imdb.com/title/tt1839578/ http://www.imdb.com/title/tt1839578/
    • TeMPOraL 13y ago
      "You are being watched. The government has a secret system: a machine that spies on you every hour of every day. I know, because I built it. I designed the machine to detect acts of terror, but it sees everything. Violent crimes involving ordinary people; people like you. Crimes the government considered 'irrelevant'. They wouldn't act, so I decided I would. But I needed a partner, someone with the skills to intervene. Hunted by the authorities, we work in secret. You'll never find us, but victim or perpetrator, if your number's up... we'll find you". (from season 1 opening) It's scary how this is actually not fiction.
  • tjr 13y ago
    Not surprised, but where is this statement in the article / screenshots? I am not seeing it.
    • LoganCale 13y ago
      It's in the full presentation linked at the top of the article. Under the heading "Finding Targets", "someone who is using encryption" is one of the listed means of identifying someone for targeting (along with "someone whose language is out of place for the region they are in"…).
  • MichaelGG 13y ago
    You've no information to back up that statement. Using PGP as part of a filter makes perfect sense. If you're looking for "bad guys" that do certain activities, as a starting filter, it doesn't hurt to say "OK, show me everyone in this region doing these activities. Now filter by language, etc. etc.". Just like if I was looking for gang members, I might start off a filter with "look for tattoos". It doesn't mean I'm saying everyone with a tattoo is gang member, it's just a way to start filtering. The NSA analysts are presumably actually trying to get something done (find people they think are bad). How stupid do you think they are? If you were an NSA analyst, would you tag "person of interest" on everyone using PGP? How would that help your goal of finding actual people of interest? They say they caught 300 "terrorists" with this program and other success stories. Presumably, they didn't achieve any success by wasting lots of time flagging random PGP users.
• jstalin 13y ago
  Snowden deserves the Nobel Peace Prize and the Vatican should consider canonizing him.
  • jbigelow76 13y ago
    To be canonized you have to have performed a miracle, but If he somehow get's pardoned by the DoJ or the Obama administration we could probably consider that requirement met.
    • skylan_q 13y ago
      You've performed a miracle, yourself. A good joke in the comments section of an HN post! :p
    • jaekwon 13y ago
      At this rate if the surveillance programs get shut down and most of the Patriot act etc get rolled back, it would be a miracle.
• nikcub 13y ago
  This is overwhelming. Even when you always hear the claims about we knew this was going on, somehow it is still shocking when you see it all laid out infront of you with screenshots and the capabilities described. I can see how they get HTTP information, since they would intercept at transit hubs - but how are they getting all Facebook private messages and Gmail? I was also looking for another unique ID that users are identified by - perhaps a machine or browser fingerprint or some form of intel that can 'glue' different browsers together and make a best guess if they are the same person (Facebook does this with device and user cookies) but couldn't find anything. It seems they rely solely on email addresses, IP addresses, cookies and HTTP headers. So if you are browsing via 16 tor circuits and a browser that defaults to incognito with session histories being wiped, they couldn't reconstruct your history. Users of PGP/encryption products being singled out is terrifying. The sooner we have the whole world using decent encryption tools, the better. Edit: Gmail messages must only be captured when they leave the Google network. They are the only provider to support server-to-server TLS: https://twitter.com/ashk4n/status/346807239002169344/photo/1 https://twitter.com/ashk4n/status/346807239002169344/photo/1 They must only be getting a slice of the Facebook chat data, since the transport there is also https. Facebook Messenger, on the other hand, uses MQTT, so it transmits and stores in plaintext. It has support for encrypted + signed messages with OTR if you are using an alternate client such as Adium or Pidgin. Really need to go out an audit all of these services and let users know which are better.
  • nwh 13y ago
    Assume that Tor is broken. With this level of deep network monitoring, low-latency onion routing is essentially useless.
    • peterwwillis 13y ago
      Hidden services are still secure, presumably, because there is no exposed section of the network to inspect. All they can do is monitor and do statistical analysis, and maybe mess with the traffic to try to get more ideas of flow.
      • 13y ago
• jstalin 13y ago
  Slide 6 of the presentation clearly shows that pretty much every government is in on the program, with heavy concentration in western Europe. One question, how did the dot in China get there? http://www.theguardian.com/world/interactive/2013/jul/31/nsa-xkeyscore-program-full-presentation http://www.theguardian.com/world/interactive/2013/jul/31/nsa...
  • jstalin 13y ago
    And look at slide 17: "Show me all VPN startups in country X, and give me data so I can decrypt and discover users." Holy crap. Is all encryption broken?
    • fabian2k 13y ago
      I find the "Show me all exploitable machines in country X" on page 24 pretty scary as well
    • MichaelGG 13y ago
      There's no evidence that any encryption is broken (other than people misusing it (edit: or broken protocols like PPTP)). Anyone could do this kinda thing given enough motivation and money. Determining a VPN's users? Just monitor all inbound connections to the VPN service. Now you have the IPs of the users. The IP alone might be enough to know the user or a search on that IP might show them logging into other services that reveal their ID. Pretty simple.
    • vabmit 13y ago
      I think so. Yes. IIRC, the first time I saw this leaked was a combo leak by a Navy Seal and a member of the Executive. The Seal leaked that they powered down Bin Laden's computers to take his hard drives after they shot him. The Executive member said that the drives were encrypted and it would take a few days to get the data. Jihadis are known to use a custom version of PGP with 2048bit RSA keys. They either used that, a COTS drive encryption program (unlikely), or reviewed and adapted an open source drive encryption program. In either of the likely scenarios they would have been using 2048bit RSA. Therefor, it is highly likely (due to the NSA having target motivation even if the drives weren't well encrypted) that the smooth barrier does not exist and the NSA can factor 2048bit RSA in a hours to days scale time frame. Also, it was leaked that NSA TAO had a 70%+ success rate compromising Chinese systems. Even with the tech companies giving them secret zero days for an extended period of time, anyone that has been a blackhat knows they're not getting to a 70% success rate through exploits. Therefor, it's highly likely they can decrypt VPN/SSH (TLS) traffic encrypted with AES256/RC4-128/3DES and/or the RSA/EC public cryptography used. As you noted the leaked slide seems to indicate that.
• runn1ng 13y ago
  From the slides http://www.theguardian.com/world/interactive/2013/jul/31/nsa-xkeyscore-program-full-presentation http://www.theguardian.com/world/interactive/2013/jul/31/nsa... "Show me all the VPN startups in country X, and give me the data so I can decrypt and discover the users" Does this mean using VPN is not very safe from dragnet?
  • znowi 13y ago
    If Google, Facebook, et al provide direct access to users' data, I'm fairly certain such critical infrastructure as VPN is also under NSA control.
    • patrickk 13y ago
      I thought Google, Facebook etc. only had to provide access as they are compelled to under US law? If the VPN provider was non-US (and did not/claimed they did not keep logs of user activity), would this help? Or do you reckon the NSA has the ability to get at the data without the cooperation of the VPN provider? It would be great to have a couple of slides revealed on this area.
  • kintamanimatt 13y ago
    You probably don't need to break the encryption because eventually all traffic has to exit the VPN's company's endpoint, and at that point it can be captured. Meta data such as the browser's fingerprint can be used to tie traffic to an individual, for example, if you see them log in to a regular HTTP site with an email or a username, this information could probably be used to figure out who they are. Armed with this information, all other traffic originating from that endpoint (or elsewhere) with the same browser fingerprint.[-1] can be monitored. Weak keys can also cause the encrypted tunnel to be compromised. Also, PPTP is considered a very insecure tunneling protocol[0] but still used. You could also break into the VPN company's servers and do interesting things too. There's also the possibility of timing attacks to determine the real IP address of the VPN user, although that's fairly a sophisticated method and quite difficult to do. Bear in mind that this presentation dates back to 2008, which is a long time in tech years. Who knows what they're capable of now. All that's known is that they're not capable of less. VPNs are useful for three things: protecting yourself against relatively unsophisticated bad guys sniffing traffic on a local network (for example, an unsecured wireless network), bypassing geographic content restrictions (e.g. using Pandora in Sweden), and circumventing ISP traffic shaping (often they'll not shape VPN traffic because it's used for businesses, and businesses can be whale customers). [-1] http://en.wikipedia.org/wiki/Device_fingerprint http://en.wikipedia.org/wiki/Device_fingerprint [0] http://blog.calyptix.com/2012/08/pptp-is-so-insecure-it-should-be.html http://blog.calyptix.com/2012/08/pptp-is-so-insecure-it-shou...
• znowi 13y ago
  Holy shit... Apparently, the only way to ensure privacy is to go Stallman. Funny how yesterday's "conspiracy crackpot" became today's visionary.
  • jacquesm 13y ago
    Stallman never was a conspiracy crackpot, he always was a visionary. The only thing that changed is some people's judgment of him.
    • griffordson 13y ago
      Exactly. It has been obvious and widely understood for years to anyone who has ever used a network analyzer that systems like this could be built. The question was always would they be built. Stallman, and others, bet correctly based on their better reading of history and human nature.
    • astrodust 13y ago
      You can be completely correct and still be a crackpot. What we need is strict limitations on what can and should be collected, and how it's used, plus better methods of securing what's being exchanged. For example, sending email as plain-text, leaving it on the server as plain-text, maybe that's a bad idea. The NSA isn't necessarily the only reason you'd do this. Foreign governments are going to take an interest in this, too, and it's only a matter of time before someone gets access the data the NSA is hoarding. No program of this scale is ever 100% secure.
      • blueprint 13y ago
        > You can be completely correct and still be a crackpot. This is very important. What do you mean by "crackpot"?
        • Karunamon 13y ago
          It's not feasible for the average person to restrict their lives to the point that RMS does and advocates for. * Reading the web via email only * Using completely free software and hardware (which as far as I can tell, limits you to a very small subset of Linux on a single Chinese-made netbook) * Not carrying a cellphone * Not using any social networks. Stallman's principled stand is admirable, but untenable for most. I need to violate every single one of these tenets in an average day at work. And that's before we even enter the realm of entertainment, which is even worse as far as the FSF's definition of freedom goes.
• shirro 13y ago
  You have to admit these guys are working on some cool problems. If you don't have a problem with the legality of it or potential for misuse it looks like a really interesting place to work.
  • stef25 13y ago
    Yeah and it looks like intelligence & skills doesn't always equate with having morals.
  • jacquesm 13y ago
    That's exactly how they get people to work on it in the first place. If you have no conscience there are lots of places where you can work on 'cool problems'.
    • astrodust 13y ago
      Weaponizing a nuclear reaction? Cool, right?
  • aroman 13y ago
    I'm going to give you the benefit of the doubt and assume you're not condoning their actions, but in case you need a more clear example of where this paradigm breaks down: IBM & Nazi Germany, Nazi Germany in general, etc. Ironically, sillicon valley came into existence building military SIGINT/ELINT systems for the cold war.[0] [0] www.youtube.com/watch?v=hFSPHfZQpIQ
    • shirro 13y ago
      I think there is a developing consensus that the emerging US police state (and that of its allies) aren't appropriate for constitutional democracies. Comparing them to Nazi experiments or nuclear weapons is still a bit premature. The potential is there for it to turn really ugly which is why the time for political action is now. But national security is important and worthwhile within constitutional limits and under democratic oversight and I suspect a scaled back NSA would be no more evil or less interesting than Bletchley Park or the development of Radar.
• impendia 13y ago
  Another data point on the relationship between government and terrorism: I live in Columbia, South Carolina. A mile from my house there is a prominent statue of Ben Tillman. Tillman was an explicit advocate of terrorism, and indeed personally engaged in it [1], which drove his popularity and ensured his election to the governorship and the United States Senate. Government programs such as the NSA's exist to protect the interests of the powerful. Same as it ever was. http://en.wikipedia.org/wiki/Benjamin_Tillman http://en.wikipedia.org/wiki/Benjamin_Tillman
• jneal 13y ago
  Noticed one of the screenshots have a URL. It's a little blurry, but I suppose it's an intranet URL since the TLD looks like .nsa URL looks like: https://gamut-wakefield.ein.nsa/utt/UTT/do/FRNewSelector#selector https://gamut-wakefield.ein.nsa/utt/UTT/do/FRNewSelector#sel...
  • jneal 13y ago
    Interesting addition. I wondered what some of these abbreviations in the URL stood for. Upon searching, I found this: http://www.techcareers.com/job.asp?id=64332188&aff=C014D02C-C8E2-4AF6-82A1-7B86393A9727 http://www.techcareers.com/job.asp?id=64332188&aff=C014D02C-... Job posting, requiring top-secret clearance, looking for people that have experience using certain tools including "GAMUT/UTT" - notice the URL from the NSA doc has "gamut" and "UTT". So i further looked into GAMUT/UTT and found this: http://williamaarkin.wordpress.com/2012/03/13/nsa-code-names-revealed/ http://williamaarkin.wordpress.com/2012/03/13/nsa-code-names...
• mortdeus 13y ago
  My real issue with this isn't so much the fact that the NSA monitors everything I say and do. Its the fact that there is nothing preventing the NSA from obtaining insider information that can be exploited for personal financial gain. I mean what stops an NSA analyst from being able to spy on an acquisition negotiation between corporation executives? What prevents that analyst from investing in the stock market using valuable info like this? There really needs to be more accountability and transparency within the NSA.
• giulianob 13y ago
  Holy shit.. they did really write a GUI in Visual Basic to track the killer's IP
  • griffordson 13y ago
    Thanks, there isn't much funny about any of this, but that got a legitimate laugh out of me. For anyone who doesn't know the reference, a clip from CSI: http://www.youtube.com/watch?v=hkDD03yeLnU http://www.youtube.com/watch?v=hkDD03yeLnU
  • mattbarrie 13y ago
    I think the guardian got trolled hard. This is too ridiculous to be true.
    • falk 13y ago
      I think you're in denial.
• peterwwillis 13y ago
  Here's an interesting legal question: If a non-US resident or NSA target posts a thread on HN, and a US person replies to the thread, is the US person now open to unlimited data collection? Alternately, if you Facebook-like the same thing an NSA target has, are you then subject to unlimited data collection?
  • gcv 13y ago
    I don't know about forums, but I suspect that being Facebook friends with a non-US citizen is enough to subject a US citizen to unlimited collection.
  • fchollet 13y ago
    The information we have already shows that the US person/non-US person distinction is purely cosmetic, meant to allow them to pretend that they're at least trying to respect US laws to some tiny extent. It's rhetorical. In reality you are always a valid target, US citizen or not.
• emhart 13y ago
  Fitting/sinister that the top of their stored data pyramid is titled "TrafficThief"
• forgotAgain 13y ago
  Aw, Fuck.
• coenhyde 13y ago
  This is the most terrifying thing i've ever seen. I'm not exaggerating in the slightest.
• arca_vorago 13y ago
  Lurkers first post here. I always said saying "I told you so." when stuff like this started getting revealed would feel like a hollow phrase. Some of us have spent quite a bit of time talking about these issues, and were mostly rejected as crackpot "conspiracy theorists". While there are plenty of those around, maybe I could use this slight moment of pseudo-clarity to propose something. I could tell you where this is going (removal of ex post facto, and eventually algorithmic based pre-crime), and who is largely behind it, but once again most of you would probably perform the standard knee-jerk reaction against "conspiracy theory", only to wait around and repeat the same kind of stuff you are saying now, whenever the next steps are put into action. We curious geeks have been too cocky, always thinking we could use our superior knowledge of technology to beat "the man". Well boys, the man is learning our tricks, and he's starting to get better at them than us... The NSA is but a cog in a greater machine, and until we all realize that and start conversing on what/who that machine is, we will continue to spin our wheels uselessly.
  • goombastic 13y ago
    Happened also because some of the geeks rented themselves out to the NSA.
    • arca_vorago 13y ago
      Just the MICC in general. I remember talking to Mudge on the phone about two years ago, asking him how Darpa was treating him. He said he loved it... guess where he is now? Google.
  • krelian 13y ago
    The problem with conspiracy theorists is that they believe their theories to be 99% certain instead of the most probable 1%. "I told you so" goes both ways except that conspiracy theorists never accept and admit it, there will always be another conspiracy to cover the current one. In the overall game it simply makes much more mathematical sense to treat conspiracy theorists as crackpots since they will so often get it wrong. Even a broken clock is correct twice a day.
• chewxy 13y ago
  I don't think I have been more conflicted about this. I've just been talking to my cofounders about the technical feasibilities of XKeyScore, and honestly, our back-of-napkin engineering configurations indicate this is really an awesome project to be working on. On the other hand, this is categorically 'evil' by my and my cofounders' ethical standards, and really, no one is safe. And that bugs the hell out of me. On the one hand: really fucking cool. On the other, I really do not like the idea that I am being spied on. I'm not sure how to process this information.
  • blackaspen 13y ago
    Welcome to working in the modern advertising industry.
    • chewxy 13y ago
      I AM currently in the advertising industry
      • LoganCale 13y ago
        No wonder you think it's cool.
      • typon 13y ago
        Sigh
  • gtaylor 13y ago
    > On the one hand: really fucking cool. I just don't see how this could be considered "cool". There are plenty of other marvels of modern computing that aren't so sinister. I think a better word for this is "scary", due to the level of cooperation from corporations and the level of secrecy it was running under for so long.
• hammerzeit 13y ago
  Reading these slides, I'm trying to parse what these slides do or do not say. I'd like to leave aside the speculation about what the NSA is probably doing. First of all, XKeyscore seems to be primarily about the frontend query interface rather than the backend data storage, at least as far as I can tell. It looks like you can basically query their database by email address and get a set of records (email, chat, http logs) back. It looks like there are separate tools for viewing specific records as well. I assume they're joining records on some combination of email address, IP address, timestamp, etc -- not unlike a modern ad server. A few practical thoughts: * It's worth noting what's not shown in these slides. Specifically, I don't see any ability to query the full text of emails. The more I see about this, the more I'm convinced the NSA is not collecting email body texts directly from corporate servers. Facebook messages I'm less sure of. * How are they collecting HTTP data? I assume intercepting at network hubs? * Given that it appears that individual records are HTTP requests, I'm shocked at how few requests are in the database. 41 billion seems an order of magnitude smaller than I'd expect. Could it be a record is something else? * Interesting to note the "Miranda number" and "Foreign Factor" fields that look like ways of saying "yes, I have permission to do this." Might explain why a sysadmin could bypass these things but your everyday NSA analyst could not.
  • cm2012 13y ago
    It doesn't show reading full emails in the screenshots, but the sentence right underneath reads: "The analyst then selects which of those returned emails they want to read by opening them in NSA reading software."
  • vidarh 13y ago
    Note regarding the amount of items, that the presentation is from 2008, and they claim to only be able to store 3 days worth of full data capture. Regarding ability to query the full text of emails, this program does not seem to indicate that it would collect the data directly from the services servers in anyway. But consider that they do indicate the ability to monitor web traffic at the protocol level. Capturing e-mail is no harder, so it'd be surprising if they're not.
  • stordoff 13y ago
• dictum 13y ago
  >Foreignness factor I know NSA's mandate is to spy on foreigners, but it's still very jingoistic and xenophobic that not being American makes it OK to spy on you.
  • kintamanimatt 13y ago
    One could assume that Americans are spied on by foreign governments and the data is just exchanged. The US spies on Brits, the UK spies on US persons, and the both compare notes.
    • lukifer 13y ago
      This is exactly why "every country spies on foreigners" is unacceptable. There's no reason to assume that this "note-sharing isn't already taking place.
    • foobarqux 13y ago
      It's less of an issue since many communication and internet services are located in the US.
    • s_q_b 13y ago
      Actually this is exactly what occurs. Intelligence exchange among America and its allies under Echelon, ANZUS, and UKUSA have been used in this exact way to end-run around anti-domestic surveillance laws.
• qwertzlcoatl 13y ago
  Livestream to senate hearings covering all this can be found here: http://www.judiciary.senate.gov/hearings/hearing.cfm?id=0d93f03188977d0d41065d3fa041decd[1] http://www.judiciary.senate.gov/hearings/hearing.cfm?id=0d93... As of this moment it's all about FISA. Wonder if this new allegation will be talked about.
  • BgSpnnrs 13y ago
    The senate hearings are with regard to phone metadata. This is a totally different show.
  • mrt0mat0 13y ago
    they did mention it offhandedly, the woman(i was only listening) started making a list of things that the nsa should release yearly including how many crimes 702 and whatever helped prevent, and the main guy asked her to add a view, in the light of the news posted today, but said they would investigate that further at a later time. sorry it's poorly detailed but just wanted you to know it was mentioned.
• naithemilkman 13y ago
  Surely the question on everyone's mind is: how good is incognito mode???
  • kintamanimatt 13y ago
    It's not. It's only just a way to visit sites without a record being stored on your computer. Read the warning. For example, Firefox's private browsing mode states: > While this computer won't have a record of your browsing history, your internet service provider or employer can still track the pages you visit. They should include government spooks in that warning!
    • shawabawa3 13y ago
      Chrome does! >Going Incognito doesn't affect the behaviour of other people, servers or software. Be wary of: > ... >Surveillance by secret agents
    • clarky07 13y ago
      Chromes warning does include govt iirc
  • LoganCale 13y ago
    The only thing incognito mode does is keep your activity and cookies while in that mode from being saved into your local browser history. It doesn't do anything at all externally.
• rosem 13y ago
  I think it's insane that so many people are pointing the finger at Snowden, yet no one is pointing the finger back at the NSA / US Government.
• venomsnake 13y ago
  Credit where credit is due - NSA made useful and usable email search. Please give it to gmail and outlook.com ... I want to be able to search trough my mail as good as you guys can do.
• kilian 13y ago
  A 'fun' bit of weasel-wording by the chairman of the House intelligence committee: "He's lying. It's impossible for him to do what he was saying he could do." They seem to be denying it, but all they're really denying is that Snowden had access to the system personally.
  • wittyphrasehere 13y ago
    It's impossible for him to do it, he's in Russia!
  • dlitz 13y ago
    They mean he didn't have the "capability" to "collect" that data, which in NSA newspeak means he had the technical means, but not the legal authorization to do so.
• deleted 13y ago
  [deleted]
• aspensmonster 13y ago
  >How do I find a strong-selector for a known target? >How do I find a cell of terrorists that has no known connection to strong-selectors? >Answer: Look for anomalous events >E.g. Someone whose language is out of place for the region they are in >Someone who is using encryption >Someone searching the web for suspicious stuff Lovely. Suspicious stuff and encryption. But wait! There's more! >Show me all the VPN startups in country X, and give me the data so I can decrypt and discover the users. Wait... what? I really hope that doesn't mean what it sounds like it means. >Slide 22 [regarding determining who authored a Jihadist document] redacted. Well that's interesting. >Show me all the exploitable machines in country X. That's cool. I'm guessing this is what Snowden meant by weak endpoint security. >Over 300 terrorists captured using intelligence generated from X-KEYSCORE >Slides 29 and 30 regarding this redacted. What a shame.
  • talmand 13y ago
    I'd like to know where they are keeping these "over 300 terrorists" that have been captured due to this program. How is one labeled a terrorist by this program I wonder?
    • o0-0o 13y ago
      By captured, I'm sure they mean captured via droning.
    • deleted 13y ago
      [deleted]
    • fnordfnordfnord 13y ago
      I'd like to know how they check for/track false positives.
    • bowlofpetunias 13y ago
      This. This is what worries me more than anything. Where have these 300 people gone? There should be records, trials, something. 300 suspected terrorists going on trial over a period of 5 years should have resulted in huge wave of almost 24/7 publicity. That the NSA is in the business of total surveillance is bad enough. But there is the faint hint that the NSA is in the business of making people disappear.
• yen223 13y ago
  Has anyone verified the accuracy of the slides? How do we know they even came from the NSA?
  • netrus 13y ago
    Let's see if NSA dares to dispute their authentity.
  • LoganCale 13y ago
    They came from Snowden. And the NSA is quoted in the article acknowledging the program by name.
• GI 13y ago
  I am going to be obliterated for this comment! Does the fact they've caught 300 terrorists in anyway justify what they're doing? I am not saying it does, I just wondered what people's thoughts were (although I can guess!). It's interesting that it was included in the article in an attempt to give it some 'balance'..
  • skore 13y ago
    Sir Thomas More decides that he would rather die than lie or betray his faith. And one moment he is arguing with the particularly vicious which hunting prosecutor. A servant of the king and a hungry and ambitious man. And More says: “You’d break the law to punish the devil, wouldn’t you?” The prosecutor says: “break it? I’d cut down every law in England if that would take it to catch him”. “Yes you would, wouldn’t you?” And then “When you would have cornered the devil and the devil would turn around to meet you, where would you run for protection, all the laws of England having been cut down and flattened? Who would protect you then?” Every time you violate – or propose the violate – the right to free speech of someone else, you in potentia you’re making a rod for your own back. Because (…), to who do you reward the right to decide which speech is harmful, or who is the harmful speaker? Or to determine in advance what are the harmful consequences going to be, that we know enough about in advance to prevent? To whom would you give this job? To whom you’re going to award the task of being the censor? http://howtoplayalone.wordpress.com/hitchens-on-free-speech/ http://howtoplayalone.wordpress.com/hitchens-on-free-speech/
    • sbi 13y ago
      These are quotes from the Bolt drama "A Man for All Seasons," not Sir Thomas More himself.
      • skore 13y ago
        Guess it was one of those days where Hitchens had a drink too many and got his quotes mixed up.
        • 13y ago
• mattbarrie 13y ago
  Um... surely this has to be a spoof. "Select foreignness factor"?? really? The user interface and way this is done just seems to amateur hour to believe this is actually true
  • mattbarrie 13y ago
    The numbers seem way off and too keystone cop to be true. 20 terabytes is not large for the NSA. It can search BCC?? Only the sender has them. so everything would have to be collected at each ISP (which isn't impossible).. but I think the guardian has been trolled.
    • mattbarrie 13y ago
      I'm calling this a troll.
      • mattbarrie 13y ago
        If you or I was in charge of the NSA there would not be a visual basic interface or a select foreignness dropdown. I'm calling bullshit on this.
        • jlgreco 13y ago
          Wait, are you talking and agreeing with yourself?
        • falk 13y ago
          In the PRISM related documents they used that exact term, "foreignness". http://www.theatlantic.com/technology/archive/2013/06/bombshell-report-nsa-and-fbi-tapping-directly-into-tech-companies-servers/276633/ http://www.theatlantic.com/technology/archive/2013/06/bombsh...
    • ihsw 13y ago
      If you're using Gmail then Google sends your email, so Google knows your BCC lines.
• vasilipupkin 13y ago
  I am curious: Suppose this is true and NSA analysts have the technical capability to access enormous amount of information with no authorization. But if they do do that, agains the law and the rules and their actions are recorded in the system, they could face penalties no? I mean I could kill someone with a hammer technically, that doesn't make hammer bad per se, does it ?
  • ryanmolden 13y ago
    It says can be audited. I suspect the number of times that has been done is vanishingly small, perhaps only on Snowden, after his revelations :) It smells to me like plausible deniability. Claim you have set up a system where things can be audited, don't ever answer how frequently it is done or what the penalty is for abuse of the system. Top secret and all. This kind of system really should freak people out. I imagine it would be a great blackmail info system if people start speaking against the govt or "causing problems". I can imagine public figures don't want to be publicly shamed for their online activities, gambling, porn, affairs, etc...
• antimatter15 13y ago
  I wonder how Sencha (http://www.sencha.com/ http://www.sencha.com/) feels about how the NSA is clearly using their ExtJS framework given the screenshots. I guess this kind of puts different perspective to the whole debate that came from JSMin's "The Software shall be used for Good, not Evil." clause (http://wonko.com/post/jsmin-isnt-welcome-on-google-code http://wonko.com/post/jsmin-isnt-welcome-on-google-code) given that conceivably your open source framework might be a significant part of something like this.
  • nemothekid 13y ago
    Its also very likely the machines are being run on Linux boxes. Should Linus being losing sleep knowing he aided the NSA in this? Even the very database this system runs on may be an offshoot of the Google BigTable paper. Should Google have never opens sourced the software that eventually became HBase/Hadoop/Cassandra because of the NSA? A tool is a tool. I don't think Henry Ford should feel guilty for enabling people to kidnap children with greater speed.
  • foobarqux 13y ago
    No it doesn't. This issue was raised and thought out long before JSMin, for example in the case of the license that forbade use by the South African military. It's a bad idea.
  • ssafejava 13y ago
    ExtJS is widely used on government systems and has been for years. If you want to deploy a rich web application that can handle large data tables with infinite scroll, filtering, sorting, etc., and run it all on IE >= 6 ExtJS is your only feasible choice.
• aspensmonster 13y ago
  Slide 13 just got slightly redacted. http://imgur.com/a/SVerP#0 http://imgur.com/a/SVerP#0
  • madaxe 13y ago
    Because it has someone's personal data in it, and the Guardian respects privacy - unlike the NSA, who use it in presentations. That's the only reason they've redacted anything, if you look at where the slides sit in context.
  • HNaTTY 13y ago
    The 64. IP address on that slide is to Google (obviously), here's the other end: GeoIP City Edition, Rev 1: PK, 08, Islamabad, N/A, 33.700001, 73.166702, 0, 0 5.157.65.58.in-addr.arpa domain name pointer mbl-65-157-5.dsl.net.pk. 23674 | 58.65.157.0/24 | MBL-AS | PK | NAYATEL.PK | MICRONET BROADBAND (PVT) LTD.
• akmiller 13y ago
  Honest, maybe naive question, but what types of programmers actively help build and maintain systems like this? I turned down a job for a company that is less than a mile from my house because I viewed their business as immoral. Hard for me to fathom anyone taking a job, helping to build systems like this. I get that many of the components of a system like this could be seen as harmless. However, a system of this complexity must have some talented engineers bringing it all together and making it work. How can they feel good about what they are doing?
  • skore 13y ago
    > How can they feel good about what they are doing? You'd be amazed. https://twitter.com/_nothingtohide https://twitter.com/_nothingtohide
    • rohit89 13y ago
      Seems to me that people aren't too concerned of nameless, faceless people looking through their stuff so long as people they know (parents, spouse, friends..) don't see it.
      • danenania 13y ago
        You know, probably one of the best things that could happen is for someone to pour out nsa data indiscriminately and en masse in order to just outright humiliate hundreds of millions of people. THAT certainly wouldn't risk an apathetic response.
      • kazagistar 13y ago
        Nameless, faceless people can show it to their parents, spouse, or friends, or threaten to.
        • katbyte 13y ago
          Unfortunately most people don't make that connection and assume that it "would never happen to them" and don't care if it happens to someone else.
• antitrust 13y ago
  I realize that it isn't morally right, but I think such data storage is inevitable. With the rise of instant communications, the amounts of data people generate are massive, and old school law enforcement can't keep up. Thus with the increase in technology, there's going to be an increase in counter-technology. I guess what we need to ask ourselves now is whether we want any secrets at all. A true Panopticon -- a society where everyone could see what everyone else was doing -- might bring a "freedom" from certain types of subterfuge, and attack. Then again, I don't want to live in it. That leads us to the question of how we handle the flood of data when looking for hostile activity, because governments are certain to use available technology to trap, parse and search that flood.
• kyro 13y ago
  Is this information the NSA could've attained left to their own devices, or is this sort of stuff only accessible with help from service providers, eg Facebook and Google? What I'm trying to ask is: with all the hullabaloo Google, Facebook, Yahoo, Microsoft, etc have made about individual, manual reviews of information requests, are we still being lied to? I suspect that we obviously haven't been told the whole story by these companies, and that they are a lot more implicit in this than they let on, but this article seems almost like definitive proof that they did indeed allow unlimited access to user information. If this is saying what I think it's saying, then I feel seriously back stabbed by the startup darlings -- Zuckerberg, Brin and Page, etc -- that so many people here love and idolize. They should absolutely be held accountable.
  • jgrahamc 13y ago
    If you can tap large Internet connections then you can siphon off all HTTP and SMTP traffic and from there it's trivial to reconstruct sessions and from there get application level stuff. I was doing similar things in the mid-1990s on shared Ethernet. It's really only a question of speed and scale and then of writing code that recognizes particular traffic (such as "this HTTP connection is a Facebook chat session").
    • kyro 13y ago
      Interesting. So then this can be done with zero assistance from service providers? Could providers have taken any steps to render that stream of information inaccessible? And if so, is it a costly effort?
      • jonknee 13y ago
        Yes, use HTTPS for everything. It's not a surprise that all the logos in this PowerPoint have since moved large portions of their traffic to SSL. SSL isn't perfect (you can still see what domain someone is requesting), but it does prevent a lot of the snooping outlined in the presentation (without vendor participation, it's always possible that Facebook is siphoning off their messages).
• northwest 13y ago
  "The NSA documents assert that by 2008, 300 terrorists had been captured using intelligence from XKeyscore." So, even IF this number is not just another lie, XKeyscore has been made worthless, with something ridiculously small as the 2 prison breaks of the recent days. That means: What remains is a police state that is not even "secure". Good job, governments/lobbyists/"defense" corporations.
• deleted 13y ago
  [deleted]
• jstalin 13y ago
  I don't think it's a leap to assume that ALL TLS certificates are compromised and the NSA can monitor ALL Internet traffic.
  • nathan_long 13y ago
    No, that's a huge leap, actually. You could go make a certificate right now for your web site and keep the private key private. I could visit your site and verify that the signature is the expected one. If you and I are both diligent, we can know that our TLS session is safe.
• rdl 13y ago
  At this point the only thing protecting the US (and the world) from the worst tyranny imaginable is that USG's essentially unlimited power is wielded by individuals (rank and file workers, career bureaucrats, political appointees, and politicians) with a fairly reasonable sense of morality and belief that they are constrained by both the constitution and morality. I'm don't think that is a stable long term system. Either some effective limitations (technical and political) are put in in the next several years, or a few decades of "us vs them" and self justifying security crises will produce a horrible result.
  • northwest 13y ago
    Absolutely. Now, the thing that prevents that from happening is money. So much money is concentrated on so few people that it protects itself and the owner becomes invulnerable. Add to that that too much money very often corrupts its owner. The predictable result for society seems pretty obvious.
    • jaekwon 13y ago
      You're spot on about money. I saw Keith Alexander's talk at the recent BlackHat conference, and was miffed at people applauding the guy, but then realized that it's a Black Hat conference, the epicenter of the monetization of exploits. The other factor is that the NSA's reputation is irreparably tarnished, and they will continue to attract the wrong kind of people. I think we need to prepare for some dark times ahead.
• totalforge 13y ago
  Just a friendly reminder that if you looked at the slides, you have read a classified document, and therefore are guilty of a Federal felony. Cheers!
  • w_t_payne 13y ago
    And the NSA is passing your details on to the FBI riiiight about now. Enjoy your federally-funded vacation!
    • northwest 13y ago
      At least you can be sure that dinner will always be served.
  • russellsprouts 13y ago
    I don't think it's a felony unless you have signed a government agreement, like working as an NSA contractor.
• rdouble 13y ago
  It seems like working for the NSA is more like working at Inintech than it is like Minority Report. I've been more embarrassed by how Office Space retarded this seems than I have about the privacy abuses. I'm skeptical any of these supposed systems work, or even exist. It reminds me of Iran's pretend fighter jet.
• w_t_payne 13y ago
  Imagine what Nixon would have got up to with this capability. He would probably still be in power!
• stef25 13y ago
  A couple years ago there was an AMA on Reddit from someone saying he was very deeply involved in spying on the general public's online lives, "at a level you can't imagine". Many technical questions were asked, all answered properly. I could never get it out of my head and now that Snowden has emerged I can't stop thinking he was the OP. Wish I could find this AMA again.
  • cracell 13y ago
    Do you happen to have a link to this AMA? Not sure what to search for to find it.
    • stef25 13y ago
      I can't find it, can't remember anything to search by. Very frustrating.
• dmix 13y ago
  Greenwald said in the comments, there is a lot more to come: > That House vote was about one specific topic - bulk collection of phone records - that this newest article has nothing to do with. That House vote isn't the be all and end all: it's just one small battle in what I can assure you will be a sustained and ongoing discussion/controversy. > There is a lot more to report still. Accuracy is the number one priority. That takes time.
• swalsh 13y ago
  The thing that blows my mind, is you hear over and over again about Billions of dollars being spent on large software projects for the government that seem fairly simplistic that ultimately fail. The NSA is accomplishing some pretty impressive things, what are they doing differently?
  • fnordfnordfnord 13y ago
    Spending tons of our money while being accountable only to themselves.
    • mpyne 13y ago
      Spending tons of money didn't help when DoD was trying to field DIHMRS. The military still runs on an ancient COBOL-era payroll system because they can't successfully develop and field a replacement, despite some billions in wasted $$$.
  • superuser2 13y ago
    Hiring mathematicians and computer scientists instead of MBAs and public policy people. Probably also not having to follow government contracting rules (lowest bidder, preferring minorities and veterans) because who would have the authority to review their purchases?
  • brewdad 13y ago
    Siphoning money away from "legitimate" projects leading to their ultimate "failure". It's just shell game accounting.
• rtf1 13y ago
  And it's not even limited by the internet. NSA collects every piece of information they can get their hands on, whether it's data on the internet or any other network, or spectrum signals, or simple imagery. And they do that by every means possible. James Bamford wrote years ago a number of books on the subject. And even before him, David Kahn painted a pretty clear picture. Why is everybody so excited so suddenly? Is there somebody on this forum who believes for a split second that Mr Obama and/or the US House/Senate are prepared to lay off way over 300,000 intelligence community workers, contractors and what have you??? -RTF
• deleted 13y ago
  [deleted]
• thejosh 13y ago
  The funniest part is the ancient version of IE they are using in the screenshots.
• MarcScott 13y ago
  The scale, depth and technical sophistication of everything I've heard and read so far has made me change my mind on whether or not there is a technical solution to NSA and GCHQ surveillance. I'm now convinced that the only way to solve this is through politics. We need representatives that will enforce our rights to privacy, not clever hacks.
  • kazagistar 13y ago
    That said, we need to do the technical solutions as well. Don't use the difficulty as an excuse to give up on one partial solution, when the other solution is partial as well.
• mrt0mat0 13y ago
  I asked this in a deeper thread, but i would like to reask anyone that can explain. If the NSA is tapping pipelines as it seems they are, wouldn't the sources such as facebook and google all come online at the same time? if they were in fact referring to the pipeline access as their way into facebook and company, why did they all have different onboarding times? wouldn't they have all come on at the same time: the time when they tapped the pipelines? Maybe i misunderstand the process. I get that maybe they had to write some interface that interpreted the packets and sorted them as such, but that wouldn't take years.
  • stordoff 13y ago
    My interpretation is that the NSA basically have two main forms of collection: data directly from fibre intercepts, and data obtained (via voluntary agreement, court order, or otherwise) from private companies. This slide [1] would certainly suggest such an arrangement. The fibre intercepts would fairly easily give access to HTTP traffic, and Facebook/Google/etc. would probably 'come online' at about the same time (there will likely be some differences as it appears there is a need to code a plug-in/processing engine for each major source to pull out usernames etc.[2]) What exactly the dates in the PRISM slide mean is somewhat unclear without more information. It could be, for example the date that the first court order is made, or the date when the company provides to the NSA a more automated way to query the data. I doubt that those dates are related to the fibre intercepts though. [1] https://image.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/6/8/1370711209084/b444b0a8-4436-4802-921e-5c3177bfc0eb-460x276.jpeg https://image.guim.co.uk/sys-images/Guardian/Pix/pictures/20... [2] https://image.guim.co.uk/sys-images/Guardian/Pix/audio/video/2013/7/31/1375269148029/KS1-001.jpg https://image.guim.co.uk/sys-images/Guardian/Pix/audio/video...
    • mrt0mat0 13y ago
      so if that's the case, then it would seem that facebook, google, etc. are still lying? or could this be more gag order stuff?
      • burntsushi 13y ago
• buggedplan 13y ago
  Germany used NSA's XKeyScore spying software: magazine report http://www.globalpost.com/dispatch/news/regions/europe/germany/130722/germany-used-nsa-xkeyscore-spying-software-magazine-repo http://www.globalpost.com/dispatch/news/regions/europe/germa...
• mikecarroll 13y ago
  I wrote an e-mail to Congressman Mike Rogers about his misleading quote in this article. I encourage others to reuse my template and also ask him to justify his misleading remarks about Snowden's statement: https://news.ycombinator.com/item?id=6134672 https://news.ycombinator.com/item?id=6134672 We should start hold our public servant to task for lying to the American people about these programs.
• adelpozo 13y ago
  What makes this even scarier is to see IE in the screenshots. :)
• VerilyForsooth 13y ago
  Why is the commentry on this topic always braindead? The article states that there is a query interface using the email address as the key. But Where does it say that every single email/webpage from every single person is being collected? Such a task would be technically impossible. It seems far more likely that it's querying a database of pretargeted people. There is so much hysterical nonsense regarding this topic. The cancer of conspiracy theory spreads.
  • computer 13y ago
    The raw slides are included in the article. You should consider looking at them.
    • obs4711 13y ago
      Yeah, I've read them. Where do they say that every single email/webpage from every single person is being collected?
      • deleted 13y ago
        [deleted]
      • acqq 13y ago
        It's nowhere in the slides.
  • pjmorris 13y ago
    > Such a task would be technically impossible. Arthur C. Clarke's first law of technology: 'When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.'
• mkhalil 13y ago
  Declassify on January 08, 2032. I wonder what kind of reaction the people of 2032 ( we who are still living that time ) would have had if they found out. Would they care? Worse reaction? Probably be used to gov. spying? It's a scary world.
• _sabe_ 13y ago
  xkeyscore.com leads to Google maps.......just saying o.0
  • diggan 13y ago
    No. It shows Google Maps in a iframe and was created 13 Jul 2013. https://gist.github.com/VictorBjelkholm/6127343 https://gist.github.com/VictorBjelkholm/6127343
• obelos 13y ago
  How weird is it that cnn.com is included as a reason for being interested in HTTP?
  • LoganCale 13y ago
    Another example is shown where they are searching for everyone who read a particular article on the BBC website.
  • chrisstanchak 13y ago
    https://news.ycombinator.com/item?id=6136016 https://news.ycombinator.com/item?id=6136016
• mkhalil 13y ago
  People aren't going to care about this until they understand what consequences this may have to THEIR personal life. Live with it. Not changing unless the knowledgeable/wise starts educating the general population on how they effects them. Most people might speak against it (include people here) but at the end, they have the "I'm not doing anything wrong, who cares, not worth the effort" mentality.
• fideloper 13y ago
  Are the major news networks ignoring this story? Briefly checking, I only see Fox News reporting related stories, naturally blaming the Obama Administration (perhaps fairly in this case).
• epoxyhockey 13y ago
  In the first slide, there are 25 red dots in a row on the bottom.. I wonder what those are supposed to represent?
  • junto 13y ago
    Digital finger-printing, so that you can tell who leaked what document. I dont want to detract from Snowden's very noble act, but I hazard a guess that Snowden knew that the documents he leaked could be traced back to him, or at a minimum a small team that he worked with.
• llamataboot 13y ago
  Can someone explain this to me like a 5 year old and how it interfaces with PRISM?
  • noir_lord 13y ago
    This is considerably more invasive than PRISM. This system logs HTTP metadata and data (think the address on the envelope and the contents of the envelope), the metadata for 30 days the contents for 3 days. This http data is essentially everything that goes over the wire all of which is then shovelled into a database with a fairly sophisticated (if not pretty) front-end that allows really invasive searches. You can search for stuff like "all emails that contain the words sex doll" or "nudes" and contain jpegs...of course the users would only use this system for legitimate operations covered by warrants.../s. This is the first of these releases that have really made me stop and go "whoa" mostly because this is "better" (bigger, more complex and capable) than anything I expected them to have now (and this was in 2008).
    • llamataboot 13y ago
      I guess I am mostly confused about whether they have any email content capabilities...and if so, how?
      • noir_lord 13y ago
        They are tapped into the hubs of internet communication as well as most (if not all) of the major webmail based systems (think Google, Hotmail, Yahoo). As to the how they are taking feeds directly off major internet routers (the vast majority of traffic will go through a major router at some point particularly if it is international though it's quite possible for a packet sent from one side of your country to another to go international as well). So yes they do have email content capabilities (if you look at the actual slides they also have a sophisticated filtering system, they can do stuff like "show me emails from iran with word documents attached containing IAEO"). This system is absolutely terrifying, it genuinely is the work of a dystopian sci-fi author from 30 years ago. ---- If you want to get right down in the trenches email is SMTP and POP over TCP/IP (normally), email is fundamentally a human readable text protocol which makes it trivially easy to parse (this was kind of the intention after all) so once they have the captured stream reconstructing the mail is not much harder (if any) than writing a mail client. You can see an example of SMTP if you open a console/shell and type "telnet smtp.gmail.com 25" and then when it has logged in type HELO the response is just plain text.
• cnlwsu 13y ago
  Ok, so ignoring all moral/ethical issues with this. Wouldn't it just be awesome to work on a project like this? Unheard of funds, tons of data, interesting CS problems all around. I am sure they did everything possible to make it miserable on the developers but nonetheless... sounds fun from a completely detached CS perspective.
  • ratsbane 13y ago
    Yes, I was thinking the same. Sort of the software engineer equivalent of flying the best fighter jets and recognizing that you're going to be using them to bomb people in mud huts instead of dogfighting equally-classed opponents.
• w_t_payne 13y ago
  I just read the actual XKeyscore slide deck. Unlike a lot of leaks, these slides are totally worth a look-see: http://www.theguardian.com/world/interactive/2013/jul/31/nsa-xkeyscore-program-full-presentation http://www.theguardian.com/world/interactive/2013/jul/31/nsa...
• imrank1 13y ago
  interesting that is not even on CNN or MSNBC yet. Has anyone else seen coverage on US news?
  • chrisstanchak 13y ago
    https://news.ycombinator.com/item?id=6136016 https://news.ycombinator.com/item?id=6136016
• api 13y ago
  I'm getting seriously irritated at the "I have nothing to hide" crowd. For starters, here are a few ways this can go horribly wrong: * Industrial espionage -- it's big business, and I'm sure it pays better than being an NSA analyst. * Foreign espionage -- since this gives unlimited querying power to every agent, a single "turned" agent could inflict massive damage on U.S. government and industry interests on behalf of a foreign power. The potential for double agents is huge. * False positives and guilt by association -- being flagged as a "person of interest" and then essentially persecuted because you have fringe ideological interests, are looking up a lot of info on terrorism for a book project, have a friend who knows radical Muslims, etc. * Corrupt use in political campaigns by incumbent politicians with access -- obvious. * Blackmail and other corruption. * Use by government agencies with access to spy on other agencies. ... I'm sure creative people can think of more.
  • antimatter 13y ago
    This. I can't seem to make people around me understand this point.
    • legutierr 13y ago
      Do you list these negatives when having that conversation? Do they not see the possibility of something like this being used against them or family members when you lay out the possibilities specifically (blackmail and industrial espionage could hit anybody)? The "nothing to hide" trope seems to me to be entirely based on a false dichotomy that contrasts "nothing to hide" with "unpatriotic/criminal". I think this is primarily because people lack the imagination to consider the other seedier and more lucrative uses of surveillance. If they were confronted with these other possibilities, would your acquaintances change their thinking? Or do these other risks--for example, the risk of having an employer targeted by competitors unfairly (potentially leading to layoffs), or the risk of having a representative vote against the interests of his or her district because of blackmail (potentially leading to a loss of government services and investment)--simply not resonate?
      • dwiel 13y ago
• scrrr 13y ago
  "Mr. <webservice-ceo> Does your company offer a backdoor for the government? It looks like it, even though you have declared that there was no backdoor just a few weeks ago!" - "Uhm. Not wittingly!"
  • MichaelGG 13y ago
    That's like accusing AT&T of having a backdoor because your apartment has thin walls and someone could overhear. Or that your ISP has a backdoor because your WiFi isn't encrypted. The FBI was doing this decades ago with Carnivore. Why is it at all surprising that such a program continues to collect unencrypted information you sent over the Internet?
    • bobbydavid 13y ago
      access. any citizen can go out and listen to public conversations, but we are not all invited to put our e-stethoscopes up to the internet backbone.
• deleted 13y ago
  [deleted]
• Buzaga 13y ago
  Yep I'm done, I'll just go about my life from now on... It's total power, I think it's unlikely that they'll want to give up on this kind of power, they'll probably keep signing governments and 'the tech' will eventually be exported and in the hands of governments everywhere, they'll keep building this and they'll create tons of algorythms of course because it's just too much data, any resistance can be crushed... and it's so much power eventually some dark times will begin... I'm done with the topic.
  • general_failure 13y ago
    Totally agree. They are not going to give this up... and worse, we are not going to fight this. We are speaking up only because this is the internet. We just want to get on with our lives. Kinda surprising why all the people who are 'overwhelmed' and 'terrified' in the parent thread don't come out and protest. Oh wait, there's kids to feed. My bad, sorry. (no snark)
    • overgryphon 13y ago
      For 99.9% percent of people, this is exactly what will happen. A few shocked moments, quickly followed by returning to all of the important things in their life. No one has time to protest all of the hundreds of wrong things we supposedly should be protesting and there aren't any clear or easy to implement steps to avoid NSA spying. There is simply no reasonable actionable item here for casual news readers. The real consequences of this news will be seen in the actions of companies. As 'cloud' (oh I hate buzzwords) technology becomes increasingly more efficient and cheaper, as Amazon, Microsoft, Openstack and VMware duke it out over cloud customers, will those customers trust them with their data? Will companies invest in private clouds for increased security, or will large public cloud service providers be able to win over and keep their trust? How much money have public cloud service providers lost since the leaks began? How many companies are now unwilling to use cloud services from US-based companies?
    • avdempsey 13y ago
      There may be a protest on Sunday near you: http://1984day.com http://1984day.com Many hands make light work.
• deleted 13y ago
  [deleted]
• andy_ppp 13y ago
  Judgement day is inevitable. G�#��$�5�%�����V��5�F�'98u�x�)�w���[_Fa��6�1f�!��['��"���VGu~w� ����|�U���Z�hep���G��^7{��K�wq��h|ڛ�m=�$L ��t� _��pM<�q��;����Y��C �M]!C�6ҝnV[�c�ϾWa�?C�M�"X*��b]0�Aļ��Li3`�P�#�j�f�u���!wb�]t_�q�&EԞw�����r�.<?K��{
• northwest 13y ago
  General reflection: As I recall it, our right to privacy is defined by the "reasonable expectation of privacy". Currently, I see any such "reasonable" expectation to be almost zero. Therefor, I have to conclude that we have already lost the right to privacy.
• kepano 13y ago
  Can someone with data center expertise extrapolate the physical scale of this operation? In terms of storage and computing power it must rival if not surpass what Google has built, no?
• heyitsnick 13y ago
  And, according to Greenwald, there's a lot more to come. From the comments: "There are thousands upon thousands of documents and they take time to read, process, vet, and report. These are very complex matters..... there is a lot more to report still. Accuracy is the number one priority. That takes time."
• badclient 13y ago
  As one slide indicates, the ability to search HTTP activity by keyword permits the analyst access to what the NSA calls "nearly everything a typical user does on the internet". It seems like everyone's been attacking the wrong folks. From this article it appears that bulk of the data is being tapped at the data center level and then parsed. This begs the question how it would be able to make sense of https traffic.
• curbrusiasm 13y ago
  This has been up here for 5 hours and on the Guardian's website for nearly 6 hours. How is it possible that not the NYTimes, FOX, NPR, the Washington Post, or CNN have picked this up? These organizations are an embarrassment to the profession of journalism.
  • chrisstanchak 13y ago
    https://news.ycombinator.com/item?id=6136016 https://news.ycombinator.com/item?id=6136016
  • 0003 13y ago
    Here is the newsdiff of today's NYT article on the disclosure of the declassified docs. [1] Talk about burying... http://newsdiffs.org/diff/290704/290768/www.nytimes.com/2013/08/01/us/nsa-surveillance.html http://newsdiffs.org/diff/290704/290768/www.nytimes.com/2013...
    • curbrusiasm 13y ago
      So rather than making a headline, update an article on another topic, specifically update an article about a PR move by the White House intended to stop talk on this issue. Assholes.
  • typon 13y ago
    I don't think its so much "an embarrassment to the profession of journalism" in terms of incompetency but rather willful ignorance.
• chrisstanchak 13y ago
  Summary of how this story is being covered around the world vs. in the US. http://imgur.com/a/tS7h4 http://imgur.com/a/tS7h4
  • mrt0mat0 13y ago
    i was shocked by this too. about an hour ago i went to CNN.com and searched the keyword xkeyscore and got 0 results. Not a single article?
• amckinlay 13y ago
  Wait, does the collection require FISA warrants? Is the collection still limited to foreign nationals?
• junto 13y ago
  The map on page 6 is interesting. Server locations of note: Moscow, Russia Caracas, Venezuela Tripoli, Libya Hubei Province, China Burma Lagos, Nigeria Saudi Arabia Iran (and geograhically surrounding Iran) Ukraine Based on page 13, I wonder if Google have any servers at these locations? Oh, what a surprise: http://royal.pingdom.com/2008/04/11/map-of-all-google-data-center-locations/ http://royal.pingdom.com/2008/04/11/map-of-all-google-data-c... Does that look familiar?
  • jnbiche 13y ago
    I'm very suspicious of Google's role in all this, but there's not much overlap in the list above and the map you refer to, outside of the normal population and business centers in Europe and the US. I didn't see any in Google data centers in Libya, Burma, Nigeria, Ukraine, Saudi Arabia, Iran, or even Venezuela (the only one in S. American looks like it's in Brazil). Based on a quick glance, the only overlap between the list above and Google's server looks like Moscow, a major global business center. Were we looking at the same map?
    • junto 13y ago
      Sorry, I wasn't suggesting Google were complicit, but the NSA need to suck up search queries, so it makes sense to locate these NSA servers right next to local Google installations. Again, page 13; a local Google Pakistan search query.
• andy_ppp 13y ago
  I just censored an email I was about to write in case it is used to discredit me in the future. Jesus. The thought police are here :-(
• sinak 13y ago
  Hey folks, Just wanted to add a note and say that if you're angry about this, the best thing that you can do is to get out into the streets and protest everything that's been going on. Check out the Restore the Fourth rallies happening this weekend, share them on social media, and sign up for your local event. http://1984Day.com http://1984Day.com Getting out into the streets is the single most significant thing you can do - even more effective than calling your legislators. The events on Sunday need to be bigger than the events July 4th for this to really be a success.
• LeeLorean 13y ago
  It is interesting on slide 17 that the NSA can decrypt all VPN traffic. Does this indicate that they have broken HTTPS, or simply that they own VPN companies like Private Internet Access?
• CPAhem 13y ago
  Slide 23: "Show me all the Microsoft Excel spreadsheets containing MAC addresses coming out of Iraq so I can perform network mapping" Does MS Excel store your MAC address in the xlsx file?
• budman 13y ago
  Wow. Just Wow. For years all of this was in the back of my mind as being capable but my not wanting to think like a conspiracy crackpot just dismissed the thought as it couldn't be possible. A conspiracy takes a lot of co-operation from within large corporations who must also remain it a secret. Surely someone would have a conscious and leak it? Or one of companies we all look up to as a modern example of do-good company would say "Hell NO" to the attempt and then let the world know what was attempted. Guess that was eventually proven true with Snowden (a real hero imo), just shocked they were able to operate to the scale they did for so long before a Snowden came along. In my mind, this is not so much a shock to me regarding the NSA as well as the current evil government we have had in place. Doesn't take a genius to realize the president lies to our face on TV about trivial issues/promises, so expected for top secret stuff. What is the BIG stomach churning shock to me is the very companies that we have come to know that are multi-billion dollar conglomerates providing service/products for millions for every day use has been a part of it. A part of this secret web while all the while proclaiming privacy for it users. I guess at end of day profits still rule the roost. "Just do this for us, turn a blind eye, and you get to go on making your billions". I wonder how many CEO's knew of all this. Gates? Zuckerburg? Etc etc. I feel like I have no outs now. There are no alternatives to current establishment of companies that make our lives easier. Should we all wipe our PC's and use Linux, sell our phones and use Ubuntu Phone, not pay for SSL certs anymore (another mafia), etc?
• nullc 13y ago
  It's been being weirdly suppressed on reddit: http://www.anonmgur.com/up/17832a6eafb09376d012090ff1b06dbe.png http://www.anonmgur.com/up/17832a6eafb09376d012090ff1b06dbe.... Every time a thread on this hits the top it gets mod-deleted.
• osth 13y ago
  http://s3.documentcloud.org/documents/743252/nsa-pdfs-redacted-ed.pdf http://s3.documentcloud.org/documents/743252/nsa-pdfs-redact... Missing: How much did this cost? Did the government (taxpayers) overpay?
• rehack 13y ago
  Makes me Wonder, if the Internet in this widespread form, was allowed so that they can snoop (so easily)? When I was a kid, my father, had told me a story that in Russia people are scared to speak their minds, for fear of being snooped via any hidden gadgets in the walls.
• w_t_payne 13y ago
  I wonder if we should try to put together a programme to try to drain the NSA of technical talent ... offering jobs or other incentives to try to persuade developers currently working for the agencies and their various contractors to resign?
• snambi 13y ago
  Wow... Govt is into big data. I wonder what they use for analyzing all this data