4 ms·

According to the NSA[1], their Utah center is "measured in zettabytes" Let's break this down: >1,000 gigabytes is a terabyte. >1,000 terabytes is a petabyte.

According to the NSA[1], their Utah center is "measured in zettabytes" Let's break this down: >1,000 gigabytes is a terabyte. >1,000 terabytes is a petabyte. >1,000 petabytes is an exabyte. >1,000 exabytes is a zettabyte. >1,000 zettabytes is a yottabyte. Assuming they use the Microsoft standard to count and not 1024 bytes to a kilobyte. Let's assume they have 1 yottabyte of data storage. That's 1000 zettabytes. That's 1,000,000 exabytes. And, finally, that's 1,000,000,000 petabytes. At a rate of 22 petabytes per day it will take approximately 45454545.45 days or 124,533 years to fill only the Utah data center. (Thank you Wolfram alpha [2]) So the scary answer to your rhetorical question is that "Yes, the NSA has the storage capacity to easily, if not comfortably store 22 petabytes of data per day, or even 500 times that without breaking a sweat." ---------- [1]http://nsa.gov1.info/utah-data-center/ http://nsa.gov1.info/utah-data-center/ [2]http://www.wolframalpha.com/input/?i=%281%C3%9710%5E9%2F22%29+days+to+years http://www.wolframalpha.com/input/?i=%281%C3%9710%5E9%2F22%2...

barrkel 13y ago
Read this: http://nsa.gov1.info/about/about.html http://nsa.gov1.info/about/about.html Total HDD shipments are ~135 million per quarter[1]. If every one of those drives was 4TB, entire world production would be about 2 zettabytes yearly. 1 yottabyte would take 500 years of current HDD production. http://www.storagenewsletter.com/news/marketreport/trendfocus-hdd-2q13 http://www.storagenewsletter.com/news/marketreport/trendfocu...
- JonSkeptic 13y ago
  lol. They got me there. I missed that it was a parody site XD. The analysis is still good though, assuming that the data center does have a one yottabyte capacity. However, let's redo the analysis with a 2 zetabyte assumption, keeping in mind that the full data capacity of the center is probably not purchased and set up in a one year period: We have 2,000 exabytes. That is 2,000,000 petabytes. That is 90909.1 days or 249.1 years. There we go, that sounds better! They can now only store a little over 10x the 22 petabytes per day comfortably. http://www.wolframalpha.com/input/?i=2000000%2F22+days+to+years http://www.wolframalpha.com/input/?i=2000000%2F22+days+to+ye...
  - segacontroller 13y ago
    In most interviews, William Binney estimates Utah will have one zettabyte of storage. I may come back with a source later. He worked very closely with these systems in their infancy, and estimates they can store 500 years of current data at maximum... but will most likely not use one large volume, and in actuality they can store 100 years of data written numerous times so they can parallel process for cryptocracking.
    - nl 13y ago
      Close. The NSA's Utah Data Center will be able to handle and process five zettabytes of data, according to William Binney, a former NSA technical director turned whistleblower. http://www.npr.org/2013/06/10/190160772/amid-data-controversy-nsa-builds-its-biggest-data-farm http://www.npr.org/2013/06/10/190160772/amid-data-controvers... However, from the same article: Despite its capacity, the Utah center does not satisfy NSA's data demands. Last month, the agency broke ground on its next data farm at its headquarters at Ft. Meade, Md. But that facility will be only two-thirds the size of the mega-complex in Utah.
chiph 13y ago
How many 4TB drives to store 1 YB? Assuming 80% fill factor for error correction & redundancy, it's 3.1 x 10^11 http://www.wolframalpha.com/input/?i=%281+yottabyte+%2F+4+terabyte%29+%2F.8 http://www.wolframalpha.com/input/?i=%281+yottabyte+%2F+4+te...
- iamthepieman 13y ago
  so how big a data center would it take to store, cool and operate more than 300 billion 4TB hard drives.
  - chiph 13y ago
    Looking around on the EMC website, they have a product that can put 1.9 PB in a 40U rack. So that's 5.26 x 10^8 racks. Each rack consumes about 18 sq ft. of floor space (includes room for aisles, servicing, etc.), so doing the math, it comes out to 339 square miles. Assuming just one floor. http://www.wolframalpha.com/input/?i=%285.26+x+10%5E8%29+*+18+sq+ft http://www.wolframalpha.com/input/?i=%285.26+x+10%5E8%29+*+1... What does this tell me? Either the NSA has storage that is 2 or more orders of magnitude denser than current industry parts, or that a lot of their data is not in online storage - it's been moved to tape for warehouse storage (but LTO v6 is not as dense as hard drive - only 2.5TB in a larger device, so the warehouse would still be huge...) Or most likely, the NSA just doesn't have 1 YB in their data center. Think about the disruption there would be in the hard drive market by someone buying 3.1 billion hard drives, even if they spaced out their purchases. And coming after the factories in Thailand were flooded a few years ago, with the shortage that happened. So what if they used data compression on disk? Assume they got 90% compression (english text, voice, etc are pretty compressible), that means they only need 1/10th the number of drives. Wolfram Alpha says their data center space requirements go down to 34 square miles. Ok, a big savings. But it's still an area the size of Manhattan.
ars 13y ago
> is "measured in zettabytes" Yah right. That would require the ENTIRE worlds production of hard disks for several years. At 3TB per hard disk you need 350 million of them. Total world production is about 600 million - but the vast majority are much smaller than 3TB. And that would require a building 1/5 of a mile on each side - just to hold the hard disks, never mind power, cooling, computers or network. It would require 2.5GW to power just the hard disks - and never mind cool them, or power the computers and routers. So: Yah Right.
- nl 13y ago
  It's not as simple as that. Large scale commercially available SANs advertise de-duplication in the order of 50%[1]. Commercially available backup software advertises de-duplication rates that reduce storage requirements by up to 99% [2] I don't know how they measure "storage", but the The Utah data center requires 65 MW of power[3] which is a non-trivial amount. [1] http://www.netapp.com/us/products/platform-os/dedupe.aspx http://www.netapp.com/us/products/platform-os/dedupe.aspx [2] http://thebackupblog.typepad.com/thebackupblog/2011/07/inside-avamar-global-client-deduplication.html http://thebackupblog.typepad.com/thebackupblog/2011/07/insid... [3] http://www.npr.org/2013/06/10/190160772/amid-data-controversy-nsa-builds-its-biggest-data-farm http://www.npr.org/2013/06/10/190160772/amid-data-controvers...
- deleted 13y ago
  [deleted]
m8urn 13y ago
Relevant: "Dear NSA, I Don’t Think You Meant Yottabytes" http://xato.net/privacy/dear-nsa-meant-yottabytes/ http://xato.net/privacy/dear-nsa-meant-yottabytes/

barrkel 13y ago

Read this: http://nsa.gov1.info/about/about.html http://nsa.gov1.info/about/about.html Total HDD shipments are ~135 million per quarter[1]. If every one of those drives was 4TB, entire world production would be about 2 zettabytes yearly. 1 yottabyte would take 500 years of current HDD production. http://www.storagenewsletter.com/news/marketreport/trendfocus-hdd-2q13 http://www.storagenewsletter.com/news/marketreport/trendfocu...

JonSkeptic 13y ago
lol. They got me there. I missed that it was a parody site XD. The analysis is still good though, assuming that the data center does have a one yottabyte capacity. However, let's redo the analysis with a 2 zetabyte assumption, keeping in mind that the full data capacity of the center is probably not purchased and set up in a one year period: We have 2,000 exabytes. That is 2,000,000 petabytes. That is 90909.1 days or 249.1 years. There we go, that sounds better! They can now only store a little over 10x the 22 petabytes per day comfortably. http://www.wolframalpha.com/input/?i=2000000%2F22+days+to+years http://www.wolframalpha.com/input/?i=2000000%2F22+days+to+ye...
- segacontroller 13y ago
  In most interviews, William Binney estimates Utah will have one zettabyte of storage. I may come back with a source later. He worked very closely with these systems in their infancy, and estimates they can store 500 years of current data at maximum... but will most likely not use one large volume, and in actuality they can store 100 years of data written numerous times so they can parallel process for cryptocracking.
  - nl 13y ago
    Close. The NSA's Utah Data Center will be able to handle and process five zettabytes of data, according to William Binney, a former NSA technical director turned whistleblower. http://www.npr.org/2013/06/10/190160772/amid-data-controversy-nsa-builds-its-biggest-data-farm http://www.npr.org/2013/06/10/190160772/amid-data-controvers... However, from the same article: Despite its capacity, the Utah center does not satisfy NSA's data demands. Last month, the agency broke ground on its next data farm at its headquarters at Ft. Meade, Md. But that facility will be only two-thirds the size of the mega-complex in Utah.

chiph 13y ago

How many 4TB drives to store 1 YB? Assuming 80% fill factor for error correction & redundancy, it's 3.1 x 10^11 http://www.wolframalpha.com/input/?i=%281+yottabyte+%2F+4+terabyte%29+%2F.8 http://www.wolframalpha.com/input/?i=%281+yottabyte+%2F+4+te...

iamthepieman 13y ago
so how big a data center would it take to store, cool and operate more than 300 billion 4TB hard drives.
- chiph 13y ago
  Looking around on the EMC website, they have a product that can put 1.9 PB in a 40U rack. So that's 5.26 x 10^8 racks. Each rack consumes about 18 sq ft. of floor space (includes room for aisles, servicing, etc.), so doing the math, it comes out to 339 square miles. Assuming just one floor. http://www.wolframalpha.com/input/?i=%285.26+x+10%5E8%29+*+18+sq+ft http://www.wolframalpha.com/input/?i=%285.26+x+10%5E8%29+*+1... What does this tell me? Either the NSA has storage that is 2 or more orders of magnitude denser than current industry parts, or that a lot of their data is not in online storage - it's been moved to tape for warehouse storage (but LTO v6 is not as dense as hard drive - only 2.5TB in a larger device, so the warehouse would still be huge...) Or most likely, the NSA just doesn't have 1 YB in their data center. Think about the disruption there would be in the hard drive market by someone buying 3.1 billion hard drives, even if they spaced out their purchases. And coming after the factories in Thailand were flooded a few years ago, with the shortage that happened. So what if they used data compression on disk? Assume they got 90% compression (english text, voice, etc are pretty compressible), that means they only need 1/10th the number of drives. Wolfram Alpha says their data center space requirements go down to 34 square miles. Ok, a big savings. But it's still an area the size of Manhattan.

ars 13y ago

> is "measured in zettabytes" Yah right. That would require the ENTIRE worlds production of hard disks for several years. At 3TB per hard disk you need 350 million of them. Total world production is about 600 million - but the vast majority are much smaller than 3TB. And that would require a building 1/5 of a mile on each side - just to hold the hard disks, never mind power, cooling, computers or network. It would require 2.5GW to power just the hard disks - and never mind cool them, or power the computers and routers. So: Yah Right.

nl 13y ago
It's not as simple as that. Large scale commercially available SANs advertise de-duplication in the order of 50%[1]. Commercially available backup software advertises de-duplication rates that reduce storage requirements by up to 99% [2] I don't know how they measure "storage", but the The Utah data center requires 65 MW of power[3] which is a non-trivial amount. [1] http://www.netapp.com/us/products/platform-os/dedupe.aspx http://www.netapp.com/us/products/platform-os/dedupe.aspx [2] http://thebackupblog.typepad.com/thebackupblog/2011/07/inside-avamar-global-client-deduplication.html http://thebackupblog.typepad.com/thebackupblog/2011/07/insid... [3] http://www.npr.org/2013/06/10/190160772/amid-data-controversy-nsa-builds-its-biggest-data-farm http://www.npr.org/2013/06/10/190160772/amid-data-controvers...
deleted 13y ago
[deleted]

m8urn 13y ago

Relevant: "Dear NSA, I Don’t Think You Meant Yottabytes" http://xato.net/privacy/dear-nsa-meant-yottabytes/ http://xato.net/privacy/dear-nsa-meant-yottabytes/