4 ms·

Anyone care to speculate on the technology that allows them to offer the fast retrieval times and low cost per GB?

• puzzle 8y ago
  "Fast" is relative here. It's fast compared to Glacier and others, but it's going to be slower than the more expensive tiers. You might not need to speculate much about how it works, it's probably implemented as described by Google themselves in slides 22ff. here: http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Keynote.pdf http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke... (In a nutshell, pack some hot data with a lot of cold data on many large drives, then put a Flash-based cache in front of them to get long tail performance predictability back.)
  • remus 8y ago
    Thanks for the link! Interesting stuff.
    • puzzle 8y ago
      There was also a talk about the low level storage service and the performance isolation work that allows it to mix batch and latency-sensitive traffic on the same drive, but it doesn't seem to have been recorded: http://www.pdl.cmu.edu/SDI/2012/101112.html http://www.pdl.cmu.edu/SDI/2012/101112.html Gory details are in the patents, 9781054, 9262093 and 8612990, which I'm not linking directly, because your lawyers might not approve. There's even a follow up, 10257111. It's so new, from two days ago, that Google Patents can't find it, while Justia can.
• whoisjuan 8y ago
  You have to keep the data in that class for a certain period of time. That's the drawback. You can access the data at that price as long as you keep it there for a long time. So I suspect this is not a fully cold storage. That's why they can retrieve the data faster. Seems more like an economics hack (Longer commitment to keep the data, allows them to buy and operate the storage hardware/software at a cost that can be amortized against those commitments)