5 ms·

That may be true. However, the giant elephant in the room is the Utah data center. This is a data center designed to, supposedly, store data on the scale of a

That may be true. However, the giant elephant in the room is the Utah data center. This is a data center designed to, supposedly, store data on the scale of a yottabyte. I only say "a" yottabyte, because to assume even slightly greater than that is sheer lunacy. That is freaking massive. If you took all terrorist cells and all terrorist activity for the history of terrorism and terrorist activity, you would not even touch a fraction of a percent utilization. We're talking rain drops in the ocean. There is no way the NSA is merely watching the bad guys here. The data center is a few magnitudes too large for such a task. I would assume right now they are merely recording all data, in hopes that one day they will have technology to quickly crack encryption. However, even without knowing what is said (the content), the metadata of connections gives plenty of information on what people are doing.

tptacek 13y ago
That Utah data center is literally the Internet's Area 51.
- jongraehl 13y ago
  "literally" :) and yes, agreed - analogous on so many levels - a publicly admitted places where secret government things happen, which can be invoked to give an aura of reality to conspiracy theories true and false alike.
tome 13y ago
Maybe they haven't filled it up yet. A quick search suggests the whole internet is on the scale of exabytes, so a yottabyte could store a million internets.
eco 13y ago
A yottabyte would take 3.333×10^11 3 terabyte hard disks. A 3.5 inch hard disk has a volume of 376.77344 cm³. The NSA buildings in Bluffdale take up around 1.5 million square feet. That means just the hard discs for a yottabyte would be just a bit less than a kilometer high (901 meters). That's about a 100 meters taller than Burj Khalifa. I think it's unlikely it's designed for anything near a yottabyte.
- Amadou 13y ago
  Clearly they do not yet have a yottabyte worth of data. It would be silly to build a warehousing site like that and have it be full-up with data the day it first opens for business. I expect these sizing claims (which presumably come from some sort of government statements about the facility) are based on a timeline on the order of ten years or more. A YB in 2024 is going to take a lot less physical space than a YB in 2014.
  - eco 13y ago
    You'd need a 2-3 order of magnitude increase in storage density to get anywhere near reasonable. I suspect that's not going to happen in 10 years but I don't know the current and projected rates of storage density increases.
    - Amadou 13y ago
      Last I read, storage capacity per dollar has been increasing faster than Moore's Law over the last decade or so. I tracked down what appears to be the origin of these yottabyte claims: http://www.nybooks.com/articles/archives/2009/nov/05/whos-in-big-brothers-database/ http://www.nybooks.com/articles/archives/2009/nov/05/whos-in... Based on that, it sounds like the yottabyte claims refer to raw, unprocessed data collected. Off the bat, I'm willing to believe in a 2 orders of magnitude decrease after data reduction techniques are applied to the raw data. My experience with data collected from telescopes was that we got about 100:1 reduction on the raw data versus what went into permanent storage. The San Antonio site was news to me, though obviously no big secret since that book was published years ago.
lawnchair_larry 13y ago
It's already been established that they redefine "collect" as a human actually viewing the data. The admit that they store roughly everything they can on everyone.
asperous 13y ago
Well even ignoring the PRISM part and Google and all that, look on the slide again: "Collection of communications on fiber cables and infrastructure as data flows past" We already know about Room_641A[1] [1]: http://en.wikipedia.org/wiki/Room_641A http://en.wikipedia.org/wiki/Room_641A
croikle 13y ago
The yottabyte stuff is basically BS. HN user Jabbles found a possible source for that, which just says '''The target GIG supports capacities exceeding exabytes (10^18 bytes) and possibly yottabytes (10^24 bytes) of data.'''[0] Exabytes are eminently reasonable; yottabytes are not. [0]: https://news.ycombinator.com/item?id=5854587 https://news.ycombinator.com/item?id=5854587

tptacek 13y ago

That Utah data center is literally the Internet's Area 51.

jongraehl 13y ago
"literally" :) and yes, agreed - analogous on so many levels - a publicly admitted places where secret government things happen, which can be invoked to give an aura of reality to conspiracy theories true and false alike.

tome 13y ago

Maybe they haven't filled it up yet. A quick search suggests the whole internet is on the scale of exabytes, so a yottabyte could store a million internets.

eco 13y ago

A yottabyte would take 3.333×10^11 3 terabyte hard disks. A 3.5 inch hard disk has a volume of 376.77344 cm³. The NSA buildings in Bluffdale take up around 1.5 million square feet. That means just the hard discs for a yottabyte would be just a bit less than a kilometer high (901 meters). That's about a 100 meters taller than Burj Khalifa. I think it's unlikely it's designed for anything near a yottabyte.

Amadou 13y ago
Clearly they do not yet have a yottabyte worth of data. It would be silly to build a warehousing site like that and have it be full-up with data the day it first opens for business. I expect these sizing claims (which presumably come from some sort of government statements about the facility) are based on a timeline on the order of ten years or more. A YB in 2024 is going to take a lot less physical space than a YB in 2014.
- eco 13y ago
  You'd need a 2-3 order of magnitude increase in storage density to get anywhere near reasonable. I suspect that's not going to happen in 10 years but I don't know the current and projected rates of storage density increases.
  - Amadou 13y ago
    Last I read, storage capacity per dollar has been increasing faster than Moore's Law over the last decade or so. I tracked down what appears to be the origin of these yottabyte claims: http://www.nybooks.com/articles/archives/2009/nov/05/whos-in-big-brothers-database/ http://www.nybooks.com/articles/archives/2009/nov/05/whos-in... Based on that, it sounds like the yottabyte claims refer to raw, unprocessed data collected. Off the bat, I'm willing to believe in a 2 orders of magnitude decrease after data reduction techniques are applied to the raw data. My experience with data collected from telescopes was that we got about 100:1 reduction on the raw data versus what went into permanent storage. The San Antonio site was news to me, though obviously no big secret since that book was published years ago.

lawnchair_larry 13y ago

It's already been established that they redefine "collect" as a human actually viewing the data. The admit that they store roughly everything they can on everyone.

asperous 13y ago

Well even ignoring the PRISM part and Google and all that, look on the slide again: "Collection of communications on fiber cables and infrastructure as data flows past" We already know about Room_641A[1] [1]: http://en.wikipedia.org/wiki/Room_641A http://en.wikipedia.org/wiki/Room_641A

croikle 13y ago

The yottabyte stuff is basically BS. HN user Jabbles found a possible source for that, which just says '''The target GIG supports capacities exceeding exabytes (10^18 bytes) and possibly yottabytes (10^24 bytes) of data.'''[0] Exabytes are eminently reasonable; yottabytes are not. [0]: https://news.ycombinator.com/item?id=5854587 https://news.ycombinator.com/item?id=5854587