4 ms·

Please throw out an estimate of how big you think a mirror of all Google's network traffic might be.

by thrownaway2424 13y ago

adamdavis 13y ago
And this is relevant to what?
- thrownaway2424 13y ago
  It's relevant because some of the things that are being suggested here on HN are clearly infeasible, from an infrastructure standpoint.
  - adamdavis 13y ago
    Word. I get that. I just thought the question sort of steered away from the main point of the parent comment, which was that if Google did have some sort of nefarious partnership that they would be legally obliged to deny it no matter what. My apologies, I'm probably just being a bit nitpicky.
  - Finster 13y ago
    O RLY? http://en.wikipedia.org/wiki/Utah_Data_Center http://en.wikipedia.org/wiki/Utah_Data_Center
- minwcnt5 13y ago
  I think his point is it would be extremely expensive. The leaked presentation claimed the cost of PRISM was only $20M.
  - akiselev 13y ago
    PRISM is clearly not the whole picture. The Utah Data Center alone is much more than $20M a year and PRISM's specified purpose was using US IT infrastructure to intercept foreign communications, which would presumably be filtered from a large data set. The bulk of the data transmission and storage costs are probably under another budget
rosser 13y ago
The NSA's Utah data center was designed to host more storage than existed on the entire planet at the time. Sheer volume of data is among the least of their concerns.
- thrownaway2424 13y ago
  OK, so you can read wikipedia, but does it make any sense? A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon. Remember when IBM announced they'd built the largest filesystem in history? That was only 120PB. A YB is 10 million times bigger than that. Consider also that a YB is 1000x the entire storage industry output from 2012. So under your theory there is a parallel hard disk drive industry consisting of at least 99% of all hard disk production on Earth.
  - jonknee 13y ago
    Facebook's servers spend most of their time finding, sorting and then transmitting bits to the billion people who use Facebook. Storing a bunch of data and storing a bunch of data that needs to be dynamically assembled in real time for a billion users are two very different things. Regardless, no one has theorized that the NSA storing yottabytes of data. The (still probably crazy) rumors are zettabytes.
    - thrownaway2424 13y ago
      "no one has theorized" that except for Wikipedia, Wired magazine, C-Net, and most of the people commenting on this thread. "NSA to store yottabytes in Utah data centre" -- http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-utah-data-centre-49304118&#x2F http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-u...;
  - mullingitover 13y ago
    >" A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon." You're off by a factor of 10-15 on the square footage. Per the wiki page, "The planned structure is 1 million or 1.5 million square feet."
ImprovedSilence 13y ago
the social "who's talking to who, and what sites did they visit" information... oh, probably this big -> http://en.wikipedia.org/wiki/Utah_Data_Center http://en.wikipedia.org/wiki/Utah_Data_Center
- saalweachter 13y ago
  I'd like to point out that their upper capacity limit, "a trillion terabytes", would require approximately 1 trillion hard drives, probably at a cost of $10-$100 trillion. I feel comfortable saying that they have not built out to that scale yet.
_stephan 13y ago
They don't need to access to all network traffic, just the interesting bits, like e.g. the internal traffic for the storage/backup systems used for GMail or Google Accounts .
jonknee 13y ago
Big. But the NSA has been sucking in ridiculous amounts of data for many many years. http://en.wikipedia.org/wiki/Room_641A http://en.wikipedia.org/wiki/Room_641A http://en.wikipedia.org/wiki/Carnivore_(software) http://en.wikipedia.org/wiki/Carnivore_(software)

Please throw out an estimate of how big you think a mirror of all Google's network traffic might be.

adamdavis 13y ago

And this is relevant to what?

thrownaway2424 13y ago
It's relevant because some of the things that are being suggested here on HN are clearly infeasible, from an infrastructure standpoint.
- adamdavis 13y ago
  Word. I get that. I just thought the question sort of steered away from the main point of the parent comment, which was that if Google did have some sort of nefarious partnership that they would be legally obliged to deny it no matter what. My apologies, I'm probably just being a bit nitpicky.
- Finster 13y ago
  O RLY? http://en.wikipedia.org/wiki/Utah_Data_Center http://en.wikipedia.org/wiki/Utah_Data_Center
minwcnt5 13y ago
I think his point is it would be extremely expensive. The leaked presentation claimed the cost of PRISM was only $20M.
- akiselev 13y ago
  PRISM is clearly not the whole picture. The Utah Data Center alone is much more than $20M a year and PRISM's specified purpose was using US IT infrastructure to intercept foreign communications, which would presumably be filtered from a large data set. The bulk of the data transmission and storage costs are probably under another budget

rosser 13y ago

The NSA's Utah data center was designed to host more storage than existed on the entire planet at the time. Sheer volume of data is among the least of their concerns.

thrownaway2424 13y ago
OK, so you can read wikipedia, but does it make any sense? A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon. Remember when IBM announced they'd built the largest filesystem in history? That was only 120PB. A YB is 10 million times bigger than that. Consider also that a YB is 1000x the entire storage industry output from 2012. So under your theory there is a parallel hard disk drive industry consisting of at least 99% of all hard disk production on Earth.
- jonknee 13y ago
  Facebook's servers spend most of their time finding, sorting and then transmitting bits to the billion people who use Facebook. Storing a bunch of data and storing a bunch of data that needs to be dynamically assembled in real time for a billion users are two very different things. Regardless, no one has theorized that the NSA storing yottabytes of data. The (still probably crazy) rumors are zettabytes.
  - thrownaway2424 13y ago
    "no one has theorized" that except for Wikipedia, Wired magazine, C-Net, and most of the people commenting on this thread. "NSA to store yottabytes in Utah data centre" -- http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-utah-data-centre-49304118&#x2F http://crave.cnet.co.uk/gadgets/nsa-to-store-yottabytes-in-u...;
- mullingitover 13y ago
  >" A single 65MW datacenter with only 100ksqft of machine room space can hold all of the data in the world? That's not even as much power as one of Facebook's facilities in Oregon." You're off by a factor of 10-15 on the square footage. Per the wiki page, "The planned structure is 1 million or 1.5 million square feet."

ImprovedSilence 13y ago

the social "who's talking to who, and what sites did they visit" information... oh, probably this big -> http://en.wikipedia.org/wiki/Utah_Data_Center http://en.wikipedia.org/wiki/Utah_Data_Center

saalweachter 13y ago
I'd like to point out that their upper capacity limit, "a trillion terabytes", would require approximately 1 trillion hard drives, probably at a cost of $10-$100 trillion. I feel comfortable saying that they have not built out to that scale yet.

_stephan 13y ago

They don't need to access to all network traffic, just the interesting bits, like e.g. the internal traffic for the storage/backup systems used for GMail or Google Accounts .

jonknee 13y ago

Big. But the NSA has been sucking in ridiculous amounts of data for many many years. http://en.wikipedia.org/wiki/Room_641A http://en.wikipedia.org/wiki/Room_641A http://en.wikipedia.org/wiki/Carnivore_(software) http://en.wikipedia.org/wiki/Carnivore_(software)