12 ms·

The NSA's Utah data center was designed to host more storage than existed on the entire planet at the time. Sheer volume of data is among the least of their con

by rosser 13y ago

The NSA's Utah data center was designed to host more storage than existed on the entire planet at the time. Sheer volume of data is among the least of their concerns.

thrownaway2424 13y ago
OK, so you can read wikipedia, but does it make any sense? A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon. Remember when IBM announced they'd built the largest filesystem in history? That was only 120PB. A YB is 10 million times bigger than that. Consider also that a YB is 1000x the entire storage industry output from 2012. So under your theory there is a parallel hard disk drive industry consisting of at least 99% of all hard disk production on Earth.
- jonknee 13y ago
  Facebook's servers spend most of their time finding, sorting and then transmitting bits to the billion people who use Facebook. Storing a bunch of data and storing a bunch of data that needs to be dynamically assembled in real time for a billion users are two very different things. Regardless, no one has theorized that the NSA storing yottabytes of data. The (still probably crazy) rumors are zettabytes.
  - thrownaway2424 13y ago
    "no one has theorized" that except for Wikipedia, Wired magazine, C-Net, and most of the people commenting on this thread. "NSA to store yottabytes in Utah data centre" -- http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-utah-data-centre-49304118&#x2F http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-u...;
- mullingitover 13y ago
  >" A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon." You're off by a factor of 10-15 on the square footage. Per the wiki page, "The planned structure is 1 million or 1.5 million square feet."
  - thrownaway2424 13y ago
    "According to USACE, the center will have 100,000 square feet of raised-floor data center space and more than 900,000 square feet of technical support and administrative space." Typical government installation. 90% overhead. http://defensesystems.com/Articles/2011/01/07/NSA-spy-cyber-intelligence-data-center-Utah.aspx http://defensesystems.com/Articles/2011/01/07/NSA-spy-cyber-...
- adventured 13y ago
  The NSA wouldn't be storing all the photos or videos from Facebook. Storing text is trivial compared to that. If you reduced Facebook's storage demands down to always static text, I think there's little doubt you could eliminate over 90% of their storage requirements (I'd say closer to 97%). And that's without any kind of compression.
magicalist 13y ago
According to the Wired story on it, it's four 25,000 square foot data center buildings[1]. That's smaller than one of Apple's smaller data centers. Apple's Oregon one is going to be 500,000 square feet. It's not going to be able to hold more than an average size datacenter, and there are a lot of data centers out there, each holding more data that would have to be copied. The NSA has huge capabilities, but they're not nearly there yet. Most of what we should be worried about is their legal capabilities. edit: it's also worth noting that the wikipedia article's source is the wired article, and all the wired article says is that a DoD report says that they need to start designing their network to handle yottabytes, which isn't even related to the NSA's data center. Wired then does some interpolation by using "yottabyte" to make the requirements of the data center sound big, but they never say that it was designed to handle that level of data or that it will ever be able to handle that level of data (in fact, they never even cite any level of data that it was designed to handle). But the Wikipedia article cites the Wired article, and suddenly it's become "a data storage facility for the United States Intelligence Community that is designed to store data on the scale of yottabytes". [1] http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/all/1 http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/al... [2] http://appleinsider.com/articles/12/08/16/apples_oregon_data_center_to_rival_size_of_nc_icloud_installation http://appleinsider.com/articles/12/08/16/apples_oregon_data...
- jonknee 13y ago
  > It's not going to be able to hold more than an average size datacenter, and there are a lot of data centers out there. T Why not? Most data centers aren't designed to store a bunch of stuff, but instead are designed to serve a lot of users. Facebook has at least one storage forward data center and it packs an exabyte into a 60k sqft footprint: http://www.datacenterknowledge.com/archives/2013/01/18/facebook-builds-new-data-centers-for-cold-storage&#x2F http://www.datacenterknowledge.com/archives/2013/01/18/faceb...; I don't think the NSA has zettabytes at hand, but you can store a massive amount of stuff in 100k square feet.
  - magicalist 13y ago
    I was replying to the fantastical claim that the datacenter "was designed to host more storage than existed on the entire planet at the time". Of course 100,000 square feet is a medium sized data center and can hold quite a bit of data. If you're happy to go even slower, you could fit even more in with a tape archive that big. You're still so very very far away from a data center sucking up all human-produced data (even accounting for duplication) and storing it for later retrieval and analysis as many here apparently believe. So yes, the NSA does have a new datacenter. But no, it's not the new Area 51 nor is it host to magic computers.

by rosser 13y ago

The NSA's Utah data center was designed to host more storage than existed on the entire planet at the time. Sheer volume of data is among the least of their concerns.

thrownaway2424 13y ago

OK, so you can read wikipedia, but does it make any sense? A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon. Remember when IBM announced they'd built the largest filesystem in history? That was only 120PB. A YB is 10 million times bigger than that. Consider also that a YB is 1000x the entire storage industry output from 2012. So under your theory there is a parallel hard disk drive industry consisting of at least 99% of all hard disk production on Earth.

jonknee 13y ago
Facebook's servers spend most of their time finding, sorting and then transmitting bits to the billion people who use Facebook. Storing a bunch of data and storing a bunch of data that needs to be dynamically assembled in real time for a billion users are two very different things. Regardless, no one has theorized that the NSA storing yottabytes of data. The (still probably crazy) rumors are zettabytes.
- thrownaway2424 13y ago
  "no one has theorized" that except for Wikipedia, Wired magazine, C-Net, and most of the people commenting on this thread. "NSA to store yottabytes in Utah data centre" -- http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-utah-data-centre-49304118&#x2F http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-u...;
mullingitover 13y ago
>" A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon." You're off by a factor of 10-15 on the square footage. Per the wiki page, "The planned structure is 1 million or 1.5 million square feet."
- thrownaway2424 13y ago
  "According to USACE, the center will have 100,000 square feet of raised-floor data center space and more than 900,000 square feet of technical support and administrative space." Typical government installation. 90% overhead. http://defensesystems.com/Articles/2011/01/07/NSA-spy-cyber-intelligence-data-center-Utah.aspx http://defensesystems.com/Articles/2011/01/07/NSA-spy-cyber-...
adventured 13y ago
The NSA wouldn't be storing all the photos or videos from Facebook. Storing text is trivial compared to that. If you reduced Facebook's storage demands down to always static text, I think there's little doubt you could eliminate over 90% of their storage requirements (I'd say closer to 97%). And that's without any kind of compression.

magicalist 13y ago

According to the Wired story on it, it's four 25,000 square foot data center buildings[1]. That's smaller than one of Apple's smaller data centers. Apple's Oregon one is going to be 500,000 square feet. It's not going to be able to hold more than an average size datacenter, and there are a lot of data centers out there, each holding more data that would have to be copied. The NSA has huge capabilities, but they're not nearly there yet. Most of what we should be worried about is their legal capabilities. edit: it's also worth noting that the wikipedia article's source is the wired article, and all the wired article says is that a DoD report says that they need to start designing their network to handle yottabytes, which isn't even related to the NSA's data center. Wired then does some interpolation by using "yottabyte" to make the requirements of the data center sound big, but they never say that it was designed to handle that level of data or that it will ever be able to handle that level of data (in fact, they never even cite any level of data that it was designed to handle). But the Wikipedia article cites the Wired article, and suddenly it's become "a data storage facility for the United States Intelligence Community that is designed to store data on the scale of yottabytes". [1] http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/all/1 http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/al... [2] http://appleinsider.com/articles/12/08/16/apples_oregon_data_center_to_rival_size_of_nc_icloud_installation http://appleinsider.com/articles/12/08/16/apples_oregon_data...

jonknee 13y ago
> It's not going to be able to hold more than an average size datacenter, and there are a lot of data centers out there. T Why not? Most data centers aren't designed to store a bunch of stuff, but instead are designed to serve a lot of users. Facebook has at least one storage forward data center and it packs an exabyte into a 60k sqft footprint: http://www.datacenterknowledge.com/archives/2013/01/18/facebook-builds-new-data-centers-for-cold-storage&#x2F http://www.datacenterknowledge.com/archives/2013/01/18/faceb...; I don't think the NSA has zettabytes at hand, but you can store a massive amount of stuff in 100k square feet.
- magicalist 13y ago
  I was replying to the fantastical claim that the datacenter "was designed to host more storage than existed on the entire planet at the time". Of course 100,000 square feet is a medium sized data center and can hold quite a bit of data. If you're happy to go even slower, you could fit even more in with a tape archive that big. You're still so very very far away from a data center sucking up all human-produced data (even accounting for duplication) and storing it for later retrieval and analysis as many here apparently believe. So yes, the NSA does have a new datacenter. But no, it's not the new Area 51 nor is it host to magic computers.