5 ms·

Documents shed light on U.S. surveillance programs

53 points by Suraj-Sun 13y ago

cgshaw 13y ago
This is a game of semantics. They only "monitor" communications that they are "lawfully authorized" to monitor. Every government official has been playing this game for 2 months now. The real questions are: 1. HOW MUCH DOMESTIC INTERNET TRAFFIC IS COLLECTED? 2. HOW IS THAT INFORMATION ORGANIZED? 3. HOW LONG DOES THE GOVERNMENT KEEP THE COLLECTED INFORMATION?
- hga 13y ago
  Don't use "collected"; as they use it "collection" only applies when captured and stored stuff is looked at.
- logn 13y ago
  The NSA are semantic sociopaths. You can never expect them to answer a question honestly because they'll redefine each term to some unexpected yet valid meaning.
  - dllthomas 13y ago
    ... or invalid. They're not that picky.
- unclebucknasty 13y ago
  Indeed. And I would add one more: WHY?
thezilch 13y ago
"monitor" ... "internet traffic" ... Great, you don't comb through every bit sent via YouTube, Netflix, etc. The smoke and mirrors are strong with this one. It was YouTube's Robert Kyncl that was quoted in 2012 that "90% of all web traffic will be video." Add in your image traffic, and there's a lot of room to still "monitor" EVERYTHING of real value.
- kawera 13y ago
  So true. I don't have the numbers but I guess 1.6% could be enough to monitor all email/messaging traffic.
  - cgshaw 13y ago
    Did some quick back of the napkin math. Daily internet traffic (from the NSA white paper) is 1,826 Petabytes. 1.6% (* .016) is 29 Petabytes that the NSA "touches." If the average email or chat is 75kb, then the NSA just admitted it touches 389 BILLION messages DAILY. Also, the NSA says it only "reviews" .025% (* .00025) of the 1.6% it "touches." Small right? Well that means that the NSA "reviews...." 97 MILLION messages DAILY. Assuming they probably don't check solicitations, I'm assuming they "touch" all emails, chats, etc. Right? And they "review" a non-trivial amount of them by quantity. Math check? Please someone tell me I'm wrong. Edited — I was off a decimal point, still a lot of messages daily.
    - function_seven 13y ago
      96 Million. Not that that changes your point at all. You're not wrong.
      - deleted 13y ago
        [deleted]
      - bwblabs 13y ago
        97M https://www.wolframalpha.com/input/?i=.025%25+of+1.6%25+of+1826+PB+%2F+75+kB https://www.wolframalpha.com/input/?i=.025%25+of+1.6%25+of+1... Of course 1.6% could be about everything in terms of information that is out there, excluding raw non-meta data like bit torrent, mpeg, etc. One way of 'hiding' data for the NSA would be to embed it in these raw data streams with incorrect metadata.. would definitely harder to find that.
        cgshaw 13y ago
        Edited. Nm, I missed a demical point in a different spot. #fml, you are right. 97M.
segacontroller 13y ago
Where is the white paper? edit: http://www.scribd.com/doc/159222203/NSA-White-Paper-on-PRISM http://www.scribd.com/doc/159222203/NSA-White-Paper-on-PRISM edit_2: http://www.scribd.com/doc/159218355/Obama-Administration-White-Paper-on-NSA-Phone-Metadata-Surveillance-Program http://www.scribd.com/doc/159218355/Obama-Administration-Whi...
- deleted 13y ago
  [deleted]
- mpyne 13y ago
  http://i2.cdn.turner.com/cnn/2013/images/08/09/2013_08_09_the_nsa_story1.pdf http://i2.cdn.turner.com/cnn/2013/images/08/09/2013_08_09_th... CNN's link for those who dislike Scribd because it hates us all. :)
- deleted 13y ago
  [deleted]
malandrew 13y ago
And what percent of those 1.6% may involve an American citizen? And of that percent, how many are being collected under a lawfully issued warrant?
- lostlogin 13y ago
  Yeah, it matters what they do to Americans, but what matters more is what is done to the rest of the world's stuff and people. We don't get to vote in the thing which gets a president in (and what an interesting process that is). We get no say, but its apparently ok to bomb us, drone us, spy on us, grope at us, render us, irradiate us to see us naked etc. I supposed it helps keep me from terrorising Americans though.
webwanderings 13y ago
So that place in Utah is gathering only 1.6% of ... whatever it is gathering. CNN and a government is a kool-aid for the masses.
gorbachev 13y ago
Wow, a 98.4% failure rate. /sarcasm
unclebucknasty 13y ago
Ah, yes. CNN. I noticed that when embassies in Yemen and elsewhere were recently shut down, CNN was careful to repeatedly note in its on-air and online coverage that it was thanks to the "intercepted" messages that we were aware of the threat. Ex: [1] In fact, in at least one story, they went so far as to even specifically mention that the "controversial NSA program" was the source--apparently for those too slow to get the hint. Such ridiculously transparent PR. And those brilliant and determined terrorists seem so easily thwarted, don't they? "Aww, rats. Foiled again! I knew I shouldn't have sent that email from my Hotmail account, with the subject 'Evil Terrorist Master Plan'. They closed the embassy. Now what do we do with these suicide belts?" Really, who believes this stuff? [1] http://www.cnn.com/2013/08/04/politics/us-embassies-close http://www.cnn.com/2013/08/04/politics/us-embassies-close
- nostromo 13y ago
  I believe it actually. Evil and effective are not mutually exclusive. The real question though is if prevented attacks would have been prevented anyway without this evidence, or in general if the price is worth paying.
  - scotth 13y ago
    > Evil and effective are not mutually exclusive. What is effective in this case?
  - gizmo686 13y ago
    I think effectiveness is open for debate as well. Consider the recent HN submission [1]. Without any information being released, we do not know if there was a credible threat, or if the NSA is simply delusional. [1] https://news.ycombinator.com/item?id=6184984 https://news.ycombinator.com/item?id=6184984
  - unclebucknasty 13y ago
    >Evil and effective are not mutually exclusive. So, we have instituted this massive security apparatus/surveillance state for groups of people who are largely incompetent, ineffective boobs? But, we don't know that they are incompetent boobs? Any way you slice it, it smells and begs the question: why are we doing this? And, in this particular case, it's a little too convenient that they are taking such public and hyper-publicized action on "threats" that do appear to be easily foiled. Because a.) the plotters allowed themselves to be intercepted even with so much recent publicity around our programs (btw, so much for the number one argument that whistleblowers like Snowden help the enemy) and b.) simply closing a few embassies nullified the threat. And, oh, BTW, let's make sure that our PR machine (read CNN and others) repeatedly mention that we all have the good ol' NSA to thank. Sorry, I just don't see how this goes down easily for anyone. >The real question though is if prevented attacks would have been prevented anyway without this evidence, or in general if the price is worth paying. In this case, I don't believe there was an imminent threat. I think it was propaganda. But, on a broader level, I think your questions are valid for those inclined to believe we should trade freedom/privacy/etc for security (i.e. most Americans). But, surely the answer is in large part a function of whether and how many "effective" threats are really out there vs. just evil people. So, your "real question" is unanswerable at this point on a general level. We don't know who the threats are and, in fact, they are intent on keeping this secret for "security reasons". It is the quintessential bogeyman. We must simply trust that these threats exist and are of a scale that warrant the massive security apparatus/surveillance state that we have erected.
dictum 13y ago
Can't wait to learn how, like "collect" now means inspect, "monitor" now means permanently store. Assuming storage, it's simple: metadata, deduplication, compression, selective monitoring, keeping only the logs but not the content (e.g. for YouTube videos), exclusion of certain bandwidth-intensive protocols, as 'thezilch mentioned...
- glitchdout 13y ago
  Aaaah, the confusing and deceitful language that is doublespeak.
mpyne 13y ago
Also of note: NSA claims that analysts review 0.00004% of global Internet traffic (0.0025% of the data they monitor, by my math).
jsmeaton 13y ago
> "In any case, the Justice memo says "any arguable privacy intrusion arising from the collection of telephony metadata would be outweighed by the public interest in identifying suspected terrorist operatives and thwarting terrorist plots." A little footnote at the bottom is all this is worth? How do they know this? How do they know that "any arguable privacy intrusion" is outweighed by the public interest in identifying "terrorist" plots, when no one was given the choice and no debate has occurred? That was (allegedly) Snowdens' entire point of releasing the information that he did! Not to mention that the premise isn't as simple as "privacy intrusion vs thousands of people getting blown up".
- AsymetricCom 13y ago
  because they debated this very thing before it was passed into law. it was public as well. perhpas you heard of the PATRIOT act?
- wpietri 13y ago
  For the record, if I ever get blown up, I don't want to be used as an excuse for stripping citizens of their liberties. My death is inevitable; freedom isn't. We thought it worth paying in blood to get that freedom, and I think it even more valuable today. And honestly, as a citizen, I see myself as partly responsible for a lot of the things that drive terrorist actions against the US. I know that if somebody blew up my family's wedding, I'd want revenge. That's not to say that I'd be happy to be killed in retaliation for some US government idiocy. But if the cycle of revenge violence has to stop somewhere, let it be with me.
rdl 13y ago
So, what's the interesting traffic left which can be passively monitored? We absolutely need OTR and then something like OTR for multiparty chat for chat. And, ideally, traffic analysis resistant. It needs to be the default, and painless. Email is mostly a lost cause for TA, but TLS covers a lot of transport. Sometime in 2014, TLS should be the default, and "required TLS" should be considered an acceptable way to configure your MTA. Standard web traffic is increasingly moving to HTTPS for a variety of reasons unrelated to NSA. The really hard part is protection from metadata collection, and making big targets resistant to demands for 1) disclosure and 2) modification to accommodate monitoring.
officemonkey 13y ago
In other news, 98.4% of the world's Internet traffic is Nigerian spam.
hosay123 13y ago
If they had any clue they'd just watch traffic to Google Analytics and instantly have something like 75% coverage. Or maybe they already do that.
hannibal5 13y ago
That's still huge amount, no matter what it actually means. If they ignore most of bittorrent and streaming services that dominate Internet traffic, the percentage of mail, skype and http traffic they monitor is incredibly high.
alternize 13y ago
so the NSA analysts actually review "only" 7.304 TB (1826 PB * 0.016 * 0.00025) of data daily...
- alternize 13y ago
  ... and to "monitor" the 1.6% traffic, they must filter the data at a (combined) rate of 2.705 Tb/s to keep up.
gmuslera 13y ago
Considering how much of the internet traffic is youtube and netflix (almost 50% according to http://www.reelseo.com/netflix-youtube-us-internet-traffic/ http://www.reelseo.com/netflix-youtube-us-internet-traffic/) and how much is just duplicated (specially in the big things like very shared p2p files) and things they already had, that number don't take out the possibility that they are taking all the interesting/private stuff that goes around. Also, that don't includes what is not internet traffic, like intrusion into private networks and servers.
samstave 13y ago
Reddit x-post from /u/Jou_ma_se_Poes in this thread: https://pay.reddit.com/r/worldnews/comments/1k0qwn/two_encrypted_email_services_have_closed_down_for/ https://pay.reddit.com/r/worldnews/comments/1k0qwn/two_encry... Everyone should read this, if you're interested in this story at all. Great succint capture of the currently known info: --- Knowledge is power. The NSA haven't been to careful with their information. With 15000 documents classified documents out there I am sure that eventually my "unknowns" for the programs below will be known. I hope someone at the NSA chokes on their coffee when they read what we know and what we will still know. I am sorry the list is so long, the NSA have been busy. If you know what any of the unknowns are do let us all know. Dial Number Recognition (DNR) Digital Network Intelligence (DNI) Geospatial Metadata Analysis (GMA) SIGINT Geospatial Analysis (SGA) SIGINT Terminal Guidance (STG) Each SIGAD is basically a collection site, physical or virtual; the SIGAD alphanumerics are used to indicate the source of intelligence FOR a particular report. The NSA listening post at Osan in Korea has the SIGAD USA-31. Clark Air Force Base is USA-57. US-987LA is with Bad Aibling Germany US-987LB are telecommunications surveillance in Afghanistan XKEYSCORE processes most of the SIGINT traffic that comes into the NSA by way of various SIGADs, and compartmentalizes it by selector. A selector might be "RUSFOR," which would stand for Russian foreign ministry intercepts. PRISM is US-984XN analyze information collected and deposited in a database - 117,675 active surveillance targets in PRISM's counterterrorism database Analysts working on a problem can request that a particular collection site be tasked, or used. The form they fill out is known as an SP0200. Additionally, when they wish to discontinue using a SIGAD for a mission, they send in another SP0200. TURBULANCE - Consists of nine sub programs including Traffic Thief, Turmoil and Tutelage. Other subprograms unknown. TRAFFIC THIEF - a database of "Meta-data from a subset of tasked strong-selectors" An example of a strong selector is an email address. In other words, it would be a database of the metadata associated with names, phone numbers, email addresses, that the intelligence services are specifically targeting AGILEVIEW: DNI tool AGILITY: DNI tool SKYWRITER: DNI reporting tool TREASUREMAP: DNI visualization tool DISHFIRE: DNI ETHEREAL: DNI TUNINGFORK/SEEKER: DNI OCTAVE: DNI/DNR tool for tasking UTT - DNR tool for tasking MAUI: a database for finished NSA intelligence products. PROTON: SIGINT database CULTWEAVE: SIGINT database HERCULES: CIA terrorism database MAIN CORE: database which contains personal and financial data of millions of U.S. citizens believed to be threats to national security PINWALE/UIS: archived foreign and domestic e-mails database (FBI access) MARINA - Internet/Telephone metadata database Unclear MAINWAY: Call records Database - contains metadata for hundreds of billions of telephone calls made through the four largest telephone carriers in the United States FALLOUT: Filter for MARINA and MAINWAY NUCLEON: Voice data Database (FBI access) CONVEYANCE: Filter for NUCLEON ARTEMIS: Geospatial analysis AIRHANDLER Geolocation analysis DRAGONFLY: Geolocation analysis GJALLER: Geospatial analysis GOSSAMER: Geospatial analysis GROWLER: Geospatial analysis KINGFISH: Geospatial analysis STINGRAY: Geospatial analysis TEMPTRESS: Geolocation analysis WITCHHUNT: Geolocation analysis CADENCE/GAMUT: Collection mission system for tasking UTT (Unified Targetting Tool?) COURIERSKILL: Collection mission system TURMOIL: Collection mission system METTLESOME: Collection mission system WEALTHYCLUSTER: Collection mission system Enhanced WEALTHYCLUSTER (EWC) MASTERLINK: Tasking source HIGHTIDE/SKYWRITER: Desktop dashboard RENOIR: Visualization tool SKOPE: SIGINT analytical toolkit DROPMIRE - NSA surveillance programme aimed at surveillance of foreign embassies and diplomatic staff, including those of NATO allies. BLARNEY - program to access internet data at key junctions and is facilitated by arrangements with commercial cable companies and internet servce providers. (part of Upstream) FAIRVIEW - NSA mass surveillance programme aimed at collecting phone, internet and e-mail data in bulk from the computers and mobile telephones of foreign countries' citizens. (part of Upstream) BOUNDLESS INFORMANT - is a big data analysis and data visualization system to give NSA managers summaries of NSA's world wide data collection activities - summarizes data records from 504 separate DNR and DNI collection sources (SIGADs). CPE (Content Preparation Environment) is a tool for writing report based on the analysis. ANCHORY: an intelligence community-wide database for intelligence reports. Mira4 and VERAS: German programs obtained by NSA (purpose unknown) Surveillance Detection Units (SDU) are organizations belonging to the US Government that have conducted secret surveillance that potentially broke national laws in various European countries. SIMAS - SDU's data is collected in a system called Security Incident Management Analysis System NSANET - network carrying information rated Top Secret or Sensitive Compartmented Information. (Snowden access) TERRORIST SURVEILLANCE PROGRAM - forerunner of PRISM STELLARWIND is the code name of a Sensitive Compartmented Information security compartment for information collected under the President's Surveillance Program TRAILBLAZER - earlier NSA effort. PIONEER GROUNDBREAKER - IBM, ATT and NSA... TEMPORA - British program including recordings of telephone calls, the content of email messages, Facebook entries and the personal internet history of users. Shares this with the NSA. CCDP - The Communications Capabilities Development Programme is a UK government initiative to create a ubiquitous mass surveillance scheme for the United Kingdom. It would involve the logging of every telephone call, email and text message between every inhabitant of the UK DHS FAST (Hostile Intent) - Future Attribute Screening Technology - precrime screening MATRIX - Multistate Anti-Terrorism Information Exchange Program closed down 2005 Trusted Traveller Program https://www.dhs.gov/trusted-traveler-programs https://www.dhs.gov/trusted-traveler-programs No Fly List - 22000 Terror Watch List - over a million Disposition Matrix - number unknown Programs unknown in the NSA: AIRGAP/COZEN - unknown AMHS - unknown ARCANAPUP - unknown ARTEMIS - unknown ASSOCIATION - unknown AUTOSOURCE - unknown BANYAN - unknown BEAMER - unknown BELLVIEW - unknown BLACKPEARL - unknown CADENCE - unknown CHALKFUN - unknown CINEPLEX - unknown CLOUD - unknown COASTLINE - unknown COMMONVIEW - unknown CONTRAOCTAVE - unknown CONVERGENCE - unknown CREEK - unknown CREST - unknown CROSSBONES - unknown CYBERTRANS - unknown DOUBLEARROW - unknown FASCIA - unknown FASTSCOPE - unknown FOREMAN - unknown GISTQUEUE - unknown GLAVE - unknown GLOBALREACH - unknown GOLDMINER - unknown GOLDPOINT - unknown GOSSAMER - unknown HIGHTIDE - unknown HOMEBASE - unknown INFOSHARE - unknown INTELINK - unknown JOLLYROGER - unknown LIQUIDFIRE - unknown MASTERSHAKE - unknown MESSIAH - unknown METRICS - unknown MICHIGAN - unknown NEWHORIZONS - unknown NIGHTSURF - unknown NORMALRUN/CHEWSTICK/FALLENORACLE - unknown OAKSTAR - unknown (part of Upstream) OCELOT - unknown OCTSKYWARD - unknown PATHMASTER/MAILORDER - unknown PANOPTICON - unknown PLUS - unknown PRESENTER - unknown PRINTAURA - unknown RAVENWING - unknown ROADBED - unknown SCORPIOFORE/CPE - unknown SIGNAV - unknown SHARKFINN - unknown SNAPE - unknown SPOTBEAM - unknown STORMBREW - unknown (part of Upstream) STINGRAY - unknown SURREY - unknown TAPERLAY - unknown TAROTCARD - unknown TRACFIN - unknown TRAILMAPPER - unknown TREASUREMAP - unknown TRICKLER - unknown TUNINGFORK - unknown TUSKATTIRE - unknown TWISTEDPATH - unknown WEBCANDID - unknown YELLOWSTONE/SPLITGLASS - unknown
Zigurd 13y ago
What percentage of the internet isn't passive content, like Netflix, other streaming media? Then subtract out garden variety Web content. Then subtract spam. I suspect 1.6% is pretty close to the amount of interpersonal e-mail and real-time voice and text that traverses the Internet.
dulob 13y ago
I would never trust NSA again EVER.

53 points by Suraj-Sun 13y ago

This is a game of semantics. They only "monitor" communications that they are "lawfully authorized" to monitor. Every government official has been playing this game for 2 months now. The real questions are: 1. HOW MUCH DOMESTIC INTERNET TRAFFIC IS COLLECTED? 2. HOW IS THAT INFORMATION ORGANIZED? 3. HOW LONG DOES THE GOVERNMENT KEEP THE COLLECTED INFORMATION?

hga 13y ago
Don't use "collected"; as they use it "collection" only applies when captured and stored stuff is looked at.
logn 13y ago
The NSA are semantic sociopaths. You can never expect them to answer a question honestly because they'll redefine each term to some unexpected yet valid meaning.
- dllthomas 13y ago
  ... or invalid. They're not that picky.
unclebucknasty 13y ago
Indeed. And I would add one more: WHY?

thezilch 13y ago

"monitor" ... "internet traffic" ... Great, you don't comb through every bit sent via YouTube, Netflix, etc. The smoke and mirrors are strong with this one. It was YouTube's Robert Kyncl that was quoted in 2012 that "90% of all web traffic will be video." Add in your image traffic, and there's a lot of room to still "monitor" EVERYTHING of real value.

kawera 13y ago
So true. I don't have the numbers but I guess 1.6% could be enough to monitor all email/messaging traffic.
- cgshaw 13y ago
  Did some quick back of the napkin math. Daily internet traffic (from the NSA white paper) is 1,826 Petabytes. 1.6% (* .016) is 29 Petabytes that the NSA "touches." If the average email or chat is 75kb, then the NSA just admitted it touches 389 BILLION messages DAILY. Also, the NSA says it only "reviews" .025% (* .00025) of the 1.6% it "touches." Small right? Well that means that the NSA "reviews...." 97 MILLION messages DAILY. Assuming they probably don't check solicitations, I'm assuming they "touch" all emails, chats, etc. Right? And they "review" a non-trivial amount of them by quantity. Math check? Please someone tell me I'm wrong. Edited — I was off a decimal point, still a lot of messages daily.
  - function_seven 13y ago
    96 Million. Not that that changes your point at all. You're not wrong.
    - deleted 13y ago
      [deleted]
    - bwblabs 13y ago
      97M https://www.wolframalpha.com/input/?i=.025%25+of+1.6%25+of+1826+PB+%2F+75+kB https://www.wolframalpha.com/input/?i=.025%25+of+1.6%25+of+1... Of course 1.6% could be about everything in terms of information that is out there, excluding raw non-meta data like bit torrent, mpeg, etc. One way of 'hiding' data for the NSA would be to embed it in these raw data streams with incorrect metadata.. would definitely harder to find that.
      - cgshaw 13y ago
        Edited. Nm, I missed a demical point in a different spot. #fml, you are right. 97M.

segacontroller 13y ago

Where is the white paper? edit: http://www.scribd.com/doc/159222203/NSA-White-Paper-on-PRISM http://www.scribd.com/doc/159222203/NSA-White-Paper-on-PRISM edit_2: http://www.scribd.com/doc/159218355/Obama-Administration-White-Paper-on-NSA-Phone-Metadata-Surveillance-Program http://www.scribd.com/doc/159218355/Obama-Administration-Whi...

deleted 13y ago
[deleted]
mpyne 13y ago
http://i2.cdn.turner.com/cnn/2013/images/08/09/2013_08_09_the_nsa_story1.pdf http://i2.cdn.turner.com/cnn/2013/images/08/09/2013_08_09_th... CNN's link for those who dislike Scribd because it hates us all. :)
deleted 13y ago
[deleted]

malandrew 13y ago

And what percent of those 1.6% may involve an American citizen? And of that percent, how many are being collected under a lawfully issued warrant?

lostlogin 13y ago
Yeah, it matters what they do to Americans, but what matters more is what is done to the rest of the world's stuff and people. We don't get to vote in the thing which gets a president in (and what an interesting process that is). We get no say, but its apparently ok to bomb us, drone us, spy on us, grope at us, render us, irradiate us to see us naked etc. I supposed it helps keep me from terrorising Americans though.

webwanderings 13y ago

So that place in Utah is gathering only 1.6% of ... whatever it is gathering. CNN and a government is a kool-aid for the masses.

gorbachev 13y ago

Wow, a 98.4% failure rate. /sarcasm

unclebucknasty 13y ago

Ah, yes. CNN. I noticed that when embassies in Yemen and elsewhere were recently shut down, CNN was careful to repeatedly note in its on-air and online coverage that it was thanks to the "intercepted" messages that we were aware of the threat. Ex: [1] In fact, in at least one story, they went so far as to even specifically mention that the "controversial NSA program" was the source--apparently for those too slow to get the hint. Such ridiculously transparent PR. And those brilliant and determined terrorists seem so easily thwarted, don't they? "Aww, rats. Foiled again! I knew I shouldn't have sent that email from my Hotmail account, with the subject 'Evil Terrorist Master Plan'. They closed the embassy. Now what do we do with these suicide belts?" Really, who believes this stuff? [1] http://www.cnn.com/2013/08/04/politics/us-embassies-close http://www.cnn.com/2013/08/04/politics/us-embassies-close

nostromo 13y ago
I believe it actually. Evil and effective are not mutually exclusive. The real question though is if prevented attacks would have been prevented anyway without this evidence, or in general if the price is worth paying.
- scotth 13y ago
  > Evil and effective are not mutually exclusive. What is effective in this case?
- gizmo686 13y ago
  I think effectiveness is open for debate as well. Consider the recent HN submission [1]. Without any information being released, we do not know if there was a credible threat, or if the NSA is simply delusional. [1] https://news.ycombinator.com/item?id=6184984 https://news.ycombinator.com/item?id=6184984
- unclebucknasty 13y ago
  >Evil and effective are not mutually exclusive. So, we have instituted this massive security apparatus/surveillance state for groups of people who are largely incompetent, ineffective boobs? But, we don't know that they are incompetent boobs? Any way you slice it, it smells and begs the question: why are we doing this? And, in this particular case, it's a little too convenient that they are taking such public and hyper-publicized action on "threats" that do appear to be easily foiled. Because a.) the plotters allowed themselves to be intercepted even with so much recent publicity around our programs (btw, so much for the number one argument that whistleblowers like Snowden help the enemy) and b.) simply closing a few embassies nullified the threat. And, oh, BTW, let's make sure that our PR machine (read CNN and others) repeatedly mention that we all have the good ol' NSA to thank. Sorry, I just don't see how this goes down easily for anyone. >The real question though is if prevented attacks would have been prevented anyway without this evidence, or in general if the price is worth paying. In this case, I don't believe there was an imminent threat. I think it was propaganda. But, on a broader level, I think your questions are valid for those inclined to believe we should trade freedom/privacy/etc for security (i.e. most Americans). But, surely the answer is in large part a function of whether and how many "effective" threats are really out there vs. just evil people. So, your "real question" is unanswerable at this point on a general level. We don't know who the threats are and, in fact, they are intent on keeping this secret for "security reasons". It is the quintessential bogeyman. We must simply trust that these threats exist and are of a scale that warrant the massive security apparatus/surveillance state that we have erected.

dictum 13y ago

Can't wait to learn how, like "collect" now means inspect, "monitor" now means permanently store. Assuming storage, it's simple: metadata, deduplication, compression, selective monitoring, keeping only the logs but not the content (e.g. for YouTube videos), exclusion of certain bandwidth-intensive protocols, as 'thezilch mentioned...

glitchdout 13y ago
Aaaah, the confusing and deceitful language that is doublespeak.

mpyne 13y ago

Also of note: NSA claims that analysts review 0.00004% of global Internet traffic (0.0025% of the data they monitor, by my math).

jsmeaton 13y ago

> "In any case, the Justice memo says "any arguable privacy intrusion arising from the collection of telephony metadata would be outweighed by the public interest in identifying suspected terrorist operatives and thwarting terrorist plots." A little footnote at the bottom is all this is worth? How do they know this? How do they know that "any arguable privacy intrusion" is outweighed by the public interest in identifying "terrorist" plots, when no one was given the choice and no debate has occurred? That was (allegedly) Snowdens' entire point of releasing the information that he did! Not to mention that the premise isn't as simple as "privacy intrusion vs thousands of people getting blown up".

AsymetricCom 13y ago
because they debated this very thing before it was passed into law. it was public as well. perhpas you heard of the PATRIOT act?
wpietri 13y ago
For the record, if I ever get blown up, I don't want to be used as an excuse for stripping citizens of their liberties. My death is inevitable; freedom isn't. We thought it worth paying in blood to get that freedom, and I think it even more valuable today. And honestly, as a citizen, I see myself as partly responsible for a lot of the things that drive terrorist actions against the US. I know that if somebody blew up my family's wedding, I'd want revenge. That's not to say that I'd be happy to be killed in retaliation for some US government idiocy. But if the cycle of revenge violence has to stop somewhere, let it be with me.

rdl 13y ago

So, what's the interesting traffic left which can be passively monitored? We absolutely need OTR and then something like OTR for multiparty chat for chat. And, ideally, traffic analysis resistant. It needs to be the default, and painless. Email is mostly a lost cause for TA, but TLS covers a lot of transport. Sometime in 2014, TLS should be the default, and "required TLS" should be considered an acceptable way to configure your MTA. Standard web traffic is increasingly moving to HTTPS for a variety of reasons unrelated to NSA. The really hard part is protection from metadata collection, and making big targets resistant to demands for 1) disclosure and 2) modification to accommodate monitoring.

officemonkey 13y ago

In other news, 98.4% of the world's Internet traffic is Nigerian spam.

hosay123 13y ago

If they had any clue they'd just watch traffic to Google Analytics and instantly have something like 75% coverage. Or maybe they already do that.

hannibal5 13y ago

That's still huge amount, no matter what it actually means. If they ignore most of bittorrent and streaming services that dominate Internet traffic, the percentage of mail, skype and http traffic they monitor is incredibly high.

alternize 13y ago

so the NSA analysts actually review "only" 7.304 TB (1826 PB * 0.016 * 0.00025) of data daily...

alternize 13y ago
... and to "monitor" the 1.6% traffic, they must filter the data at a (combined) rate of 2.705 Tb/s to keep up.

gmuslera 13y ago

Considering how much of the internet traffic is youtube and netflix (almost 50% according to http://www.reelseo.com/netflix-youtube-us-internet-traffic/ http://www.reelseo.com/netflix-youtube-us-internet-traffic/) and how much is just duplicated (specially in the big things like very shared p2p files) and things they already had, that number don't take out the possibility that they are taking all the interesting/private stuff that goes around. Also, that don't includes what is not internet traffic, like intrusion into private networks and servers.

samstave 13y ago

Reddit x-post from /u/Jou_ma_se_Poes in this thread: https://pay.reddit.com/r/worldnews/comments/1k0qwn/two_encrypted_email_services_have_closed_down_for/ https://pay.reddit.com/r/worldnews/comments/1k0qwn/two_encry... Everyone should read this, if you're interested in this story at all. Great succint capture of the currently known info: --- Knowledge is power. The NSA haven't been to careful with their information. With 15000 documents classified documents out there I am sure that eventually my "unknowns" for the programs below will be known. I hope someone at the NSA chokes on their coffee when they read what we know and what we will still know. I am sorry the list is so long, the NSA have been busy. If you know what any of the unknowns are do let us all know. Dial Number Recognition (DNR) Digital Network Intelligence (DNI) Geospatial Metadata Analysis (GMA) SIGINT Geospatial Analysis (SGA) SIGINT Terminal Guidance (STG) Each SIGAD is basically a collection site, physical or virtual; the SIGAD alphanumerics are used to indicate the source of intelligence FOR a particular report. The NSA listening post at Osan in Korea has the SIGAD USA-31. Clark Air Force Base is USA-57. US-987LA is with Bad Aibling Germany US-987LB are telecommunications surveillance in Afghanistan XKEYSCORE processes most of the SIGINT traffic that comes into the NSA by way of various SIGADs, and compartmentalizes it by selector. A selector might be "RUSFOR," which would stand for Russian foreign ministry intercepts. PRISM is US-984XN analyze information collected and deposited in a database - 117,675 active surveillance targets in PRISM's counterterrorism database Analysts working on a problem can request that a particular collection site be tasked, or used. The form they fill out is known as an SP0200. Additionally, when they wish to discontinue using a SIGAD for a mission, they send in another SP0200. TURBULANCE - Consists of nine sub programs including Traffic Thief, Turmoil and Tutelage. Other subprograms unknown. TRAFFIC THIEF - a database of "Meta-data from a subset of tasked strong-selectors" An example of a strong selector is an email address. In other words, it would be a database of the metadata associated with names, phone numbers, email addresses, that the intelligence services are specifically targeting AGILEVIEW: DNI tool AGILITY: DNI tool SKYWRITER: DNI reporting tool TREASUREMAP: DNI visualization tool DISHFIRE: DNI ETHEREAL: DNI TUNINGFORK/SEEKER: DNI OCTAVE: DNI/DNR tool for tasking UTT - DNR tool for tasking MAUI: a database for finished NSA intelligence products. PROTON: SIGINT database CULTWEAVE: SIGINT database HERCULES: CIA terrorism database MAIN CORE: database which contains personal and financial data of millions of U.S. citizens believed to be threats to national security PINWALE/UIS: archived foreign and domestic e-mails database (FBI access) MARINA - Internet/Telephone metadata database Unclear MAINWAY: Call records Database - contains metadata for hundreds of billions of telephone calls made through the four largest telephone carriers in the United States FALLOUT: Filter for MARINA and MAINWAY NUCLEON: Voice data Database (FBI access) CONVEYANCE: Filter for NUCLEON ARTEMIS: Geospatial analysis AIRHANDLER Geolocation analysis DRAGONFLY: Geolocation analysis GJALLER: Geospatial analysis GOSSAMER: Geospatial analysis GROWLER: Geospatial analysis KINGFISH: Geospatial analysis STINGRAY: Geospatial analysis TEMPTRESS: Geolocation analysis WITCHHUNT: Geolocation analysis CADENCE/GAMUT: Collection mission system for tasking UTT (Unified Targetting Tool?) COURIERSKILL: Collection mission system TURMOIL: Collection mission system METTLESOME: Collection mission system WEALTHYCLUSTER: Collection mission system Enhanced WEALTHYCLUSTER (EWC) MASTERLINK: Tasking source HIGHTIDE/SKYWRITER: Desktop dashboard RENOIR: Visualization tool SKOPE: SIGINT analytical toolkit DROPMIRE - NSA surveillance programme aimed at surveillance of foreign embassies and diplomatic staff, including those of NATO allies. BLARNEY - program to access internet data at key junctions and is facilitated by arrangements with commercial cable companies and internet servce providers. (part of Upstream) FAIRVIEW - NSA mass surveillance programme aimed at collecting phone, internet and e-mail data in bulk from the computers and mobile telephones of foreign countries' citizens. (part of Upstream) BOUNDLESS INFORMANT - is a big data analysis and data visualization system to give NSA managers summaries of NSA's world wide data collection activities - summarizes data records from 504 separate DNR and DNI collection sources (SIGADs). CPE (Content Preparation Environment) is a tool for writing report based on the analysis. ANCHORY: an intelligence community-wide database for intelligence reports. Mira4 and VERAS: German programs obtained by NSA (purpose unknown) Surveillance Detection Units (SDU) are organizations belonging to the US Government that have conducted secret surveillance that potentially broke national laws in various European countries. SIMAS - SDU's data is collected in a system called Security Incident Management Analysis System NSANET - network carrying information rated Top Secret or Sensitive Compartmented Information. (Snowden access) TERRORIST SURVEILLANCE PROGRAM - forerunner of PRISM STELLARWIND is the code name of a Sensitive Compartmented Information security compartment for information collected under the President's Surveillance Program TRAILBLAZER - earlier NSA effort. PIONEER GROUNDBREAKER - IBM, ATT and NSA... TEMPORA - British program including recordings of telephone calls, the content of email messages, Facebook entries and the personal internet history of users. Shares this with the NSA. CCDP - The Communications Capabilities Development Programme is a UK government initiative to create a ubiquitous mass surveillance scheme for the United Kingdom. It would involve the logging of every telephone call, email and text message between every inhabitant of the UK DHS FAST (Hostile Intent) - Future Attribute Screening Technology - precrime screening MATRIX - Multistate Anti-Terrorism Information Exchange Program closed down 2005 Trusted Traveller Program https://www.dhs.gov/trusted-traveler-programs https://www.dhs.gov/trusted-traveler-programs No Fly List - 22000 Terror Watch List - over a million Disposition Matrix - number unknown Programs unknown in the NSA: AIRGAP/COZEN - unknown AMHS - unknown ARCANAPUP - unknown ARTEMIS - unknown ASSOCIATION - unknown AUTOSOURCE - unknown BANYAN - unknown BEAMER - unknown BELLVIEW - unknown BLACKPEARL - unknown CADENCE - unknown CHALKFUN - unknown CINEPLEX - unknown CLOUD - unknown COASTLINE - unknown COMMONVIEW - unknown CONTRAOCTAVE - unknown CONVERGENCE - unknown CREEK - unknown CREST - unknown CROSSBONES - unknown CYBERTRANS - unknown DOUBLEARROW - unknown FASCIA - unknown FASTSCOPE - unknown FOREMAN - unknown GISTQUEUE - unknown GLAVE - unknown GLOBALREACH - unknown GOLDMINER - unknown GOLDPOINT - unknown GOSSAMER - unknown HIGHTIDE - unknown HOMEBASE - unknown INFOSHARE - unknown INTELINK - unknown JOLLYROGER - unknown LIQUIDFIRE - unknown MASTERSHAKE - unknown MESSIAH - unknown METRICS - unknown MICHIGAN - unknown NEWHORIZONS - unknown NIGHTSURF - unknown NORMALRUN/CHEWSTICK/FALLENORACLE - unknown OAKSTAR - unknown (part of Upstream) OCELOT - unknown OCTSKYWARD - unknown PATHMASTER/MAILORDER - unknown PANOPTICON - unknown PLUS - unknown PRESENTER - unknown PRINTAURA - unknown RAVENWING - unknown ROADBED - unknown SCORPIOFORE/CPE - unknown SIGNAV - unknown SHARKFINN - unknown SNAPE - unknown SPOTBEAM - unknown STORMBREW - unknown (part of Upstream) STINGRAY - unknown SURREY - unknown TAPERLAY - unknown TAROTCARD - unknown TRACFIN - unknown TRAILMAPPER - unknown TREASUREMAP - unknown TRICKLER - unknown TUNINGFORK - unknown TUSKATTIRE - unknown TWISTEDPATH - unknown WEBCANDID - unknown YELLOWSTONE/SPLITGLASS - unknown

Zigurd 13y ago

What percentage of the internet isn't passive content, like Netflix, other streaming media? Then subtract out garden variety Web content. Then subtract spam. I suspect 1.6% is pretty close to the amount of interpersonal e-mail and real-time voice and text that traverses the Internet.

dulob 13y ago

I would never trust NSA again EVER.