9 ms·

To me the warrantless wiretapping scandal IS the proof. I mean if we know that there are Room 614As [1] all over the country, and that they are diverting nearl

To me the warrantless wiretapping scandal IS the proof. I mean if we know that there are Room 614As [1] all over the country, and that they are diverting nearly ALL communications to their facilities, in addition to them building new facilities to house data all the time [2]. All of this culminating with their shiny new data center [3], which cost 2 billion dollars to build, covers 900,000 sqft, and has a 40 million dollar annual utility bill, it becomes less about proving that they're doing this, but proving that they aren't doing this. [1] http://en.wikipedia.org/wiki/Room_641A http://en.wikipedia.org/wiki/Room_641A [2] http://www2.sacurrent.com/news/story.asp?id=69607 http://www2.sacurrent.com/news/story.asp?id=69607 [3] http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/all/ http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/al...

tripzilch 14y ago
Exactly. If they're not tapping all communications on the net, then why are they building a data centre to store and process all of that data? Because I personally can't imagine another datastream 1) of such magnitude and 2) worth billions of dollars to build a data centre for.
- lallysingh 14y ago
  Even at 100k square feet, I don't think they have the capacity to record and store all data on the Internet backbone (they're recording a lot of data outside the US, too). First, they'd need a shadow Internet backbone capable of transporting all that data to the DC. Second, they'd have to build a system with the same write-rate as all of the internet backbone. This stuff isn't cheap, and the NSA's budget is big, but not that big. As 1/3rd of the US's internet traffic is porn, which you can really just de-dup by URL, it's a safe bet to say that they're filtering before they transmit data back home. The same goes for streaming video traffic. The remainder is mostly web with a small fraction being actual communication between people. For the NSA's mission, even implemented in the most evil way, they just don't have the money, means, or motivation to record everything. Instead, filtering on *hotmail.com connections, anything over SMTP ports, etc. makes substantially more sense.
  - deleted 14y ago
    [deleted]
  - jacquesm 14y ago
    They don't need to store the traffic. All they need is to store the endpoints and timestamps and you could compress that quite well to have usable information, such as who talked to who and when. Your actual data is just a subpoena away anyway in most cases.
    - sp332 14y ago
      Exactly, why store all the data in your super-expensive top-secret warehouse, when Facebook and Google will keep it for you for free? Free == didn't come out of the NSA's budget
      - Strshps1MoreTim 14y ago
        Money are of no objection to that industry. The combined budget of NSA, CIA, FBI, Pentagon, foreign bases, Afghanistan and Iraq occupation, etc. is in the trillions. Why trust FB or Google if all it costs to capture and store the information are money.
    - philwelch 14y ago
      NSA is an intelligence agency, not a law enforcement agency. They don't operate within the court process. Which means they don't really do subpoenas or warrants. Furthermore, any data covertly collected by the NSA would be inadmissible in court anyway, which would make it impossible to get a subpoena or warrant based on it anyway.
  - belorn 14y ago
    Doing some digging, I find it rather hard to find data on the amount of traffic being sent in and out, and internally of the US. The best numbers I could find was from almost 10 years ago, which said around 970Gbit/s. Say it has gone up 100x, and we get a nice round number of 10 TB disk space needed per second. We know, thanks to XCD, that 7523 hard drives per second is created by the storage industry (globally, a total of 650 million drives per year). Say that the average storage space is over the last 5 years, around 500GB. That mean, so long NSA buy's 0.265% of all hard drives produced each year, they will have enough hard drives to record all data transmitted inside the US borders. They would still need to write the data. One obvious way would be to store it on site, and transport the drives to a central place. Drives are not big, but it is a noticeable work, so if people were doing this, there should be more verifiable proof of it. If we include post-storage compression, finding duplicates and any other tricks, the numbers should be able to be lowered by 50-75% or so, and might be enough to send some through the wire and only the overflow through drives loaded onto trucks. One could also ask what 0.265% of the storage industry output is in raw cash. To answer that, my answer is, I dont know :).
indiecore 14y ago
Next time I can't find an IRC log I'll just call the NSA.
- nitrogen 14y ago
  An interesting take on your comment: would people be more open to the NSA snarfing everything if they also acted as a personal Google, giving you access to all your own "lost" data?
  - lsc 14y ago
    I've been thinking about this lately, with regard to my own privacy policy. I was thinking something to the effect "I will make all data I save that references your account or IP available to you through the following portal" - I of course, can't do it yet, as I haven't built the portal or figured out how to separate the data, but I think it's a neat solution to some of the problems I have (e.g. I really want to start doing console logging on VPSs 'cause it gives me a wealth of data when someone says "something was wrong with my VPS last night" - but so far I haven't (I do log physical consoles, just 'cause it's impossible to deal with hardware otherwise) due to privacy concerns.)
    - indiecore 14y ago
      at /domain/portal just put a contact page and ask them to specify what they are looking for. You'll have to do it manually until you build the automated system but maybe the inconvenience would be motivation?
      - lsc 14y ago
        >but maybe the inconvenience would be motivation? I need to limit the number of times I give myself extra work as 'motivation' - you get to the point where you spend all your time shooting alligators, and don't have any time at all to drain the swamp. I mean, sometimes this is better than just not doing it? but not in this case, if you ask me.
  - 14y ago

tripzilch 14y ago

Exactly. If they're not tapping all communications on the net, then why are they building a data centre to store and process all of that data? Because I personally can't imagine another datastream 1) of such magnitude and 2) worth billions of dollars to build a data centre for.

lallysingh 14y ago
Even at 100k square feet, I don't think they have the capacity to record and store all data on the Internet backbone (they're recording a lot of data outside the US, too). First, they'd need a shadow Internet backbone capable of transporting all that data to the DC. Second, they'd have to build a system with the same write-rate as all of the internet backbone. This stuff isn't cheap, and the NSA's budget is big, but not that big. As 1/3rd of the US's internet traffic is porn, which you can really just de-dup by URL, it's a safe bet to say that they're filtering before they transmit data back home. The same goes for streaming video traffic. The remainder is mostly web with a small fraction being actual communication between people. For the NSA's mission, even implemented in the most evil way, they just don't have the money, means, or motivation to record everything. Instead, filtering on *hotmail.com connections, anything over SMTP ports, etc. makes substantially more sense.
- deleted 14y ago
  [deleted]
- jacquesm 14y ago
  They don't need to store the traffic. All they need is to store the endpoints and timestamps and you could compress that quite well to have usable information, such as who talked to who and when. Your actual data is just a subpoena away anyway in most cases.
  - sp332 14y ago
    Exactly, why store all the data in your super-expensive top-secret warehouse, when Facebook and Google will keep it for you for free? Free == didn't come out of the NSA's budget
    - Strshps1MoreTim 14y ago
      Money are of no objection to that industry. The combined budget of NSA, CIA, FBI, Pentagon, foreign bases, Afghanistan and Iraq occupation, etc. is in the trillions. Why trust FB or Google if all it costs to capture and store the information are money.
  - philwelch 14y ago
    NSA is an intelligence agency, not a law enforcement agency. They don't operate within the court process. Which means they don't really do subpoenas or warrants. Furthermore, any data covertly collected by the NSA would be inadmissible in court anyway, which would make it impossible to get a subpoena or warrant based on it anyway.
- belorn 14y ago
  Doing some digging, I find it rather hard to find data on the amount of traffic being sent in and out, and internally of the US. The best numbers I could find was from almost 10 years ago, which said around 970Gbit/s. Say it has gone up 100x, and we get a nice round number of 10 TB disk space needed per second. We know, thanks to XCD, that 7523 hard drives per second is created by the storage industry (globally, a total of 650 million drives per year). Say that the average storage space is over the last 5 years, around 500GB. That mean, so long NSA buy's 0.265% of all hard drives produced each year, they will have enough hard drives to record all data transmitted inside the US borders. They would still need to write the data. One obvious way would be to store it on site, and transport the drives to a central place. Drives are not big, but it is a noticeable work, so if people were doing this, there should be more verifiable proof of it. If we include post-storage compression, finding duplicates and any other tricks, the numbers should be able to be lowered by 50-75% or so, and might be enough to send some through the wire and only the overflow through drives loaded onto trucks. One could also ask what 0.265% of the storage industry output is in raw cash. To answer that, my answer is, I dont know :).

indiecore 14y ago

Next time I can't find an IRC log I'll just call the NSA.

nitrogen 14y ago
An interesting take on your comment: would people be more open to the NSA snarfing everything if they also acted as a personal Google, giving you access to all your own "lost" data?
- lsc 14y ago
  I've been thinking about this lately, with regard to my own privacy policy. I was thinking something to the effect "I will make all data I save that references your account or IP available to you through the following portal" - I of course, can't do it yet, as I haven't built the portal or figured out how to separate the data, but I think it's a neat solution to some of the problems I have (e.g. I really want to start doing console logging on VPSs 'cause it gives me a wealth of data when someone says "something was wrong with my VPS last night" - but so far I haven't (I do log physical consoles, just 'cause it's impossible to deal with hardware otherwise) due to privacy concerns.)
  - indiecore 14y ago
    at /domain/portal just put a contact page and ask them to specify what they are looking for. You'll have to do it manually until you build the automated system but maybe the inconvenience would be motivation?
    - lsc 14y ago
      >but maybe the inconvenience would be motivation? I need to limit the number of times I give myself extra work as 'motivation' - you get to the point where you spend all your time shooting alligators, and don't have any time at all to drain the swamp. I mean, sometimes this is better than just not doing it? but not in this case, if you ask me.
- 14y ago