6 ms·

In a side-note in the article, the author mentions that Amazon Glacier runs on tape. Although I'm not sure Amazon officially explained the technology, I heard (

by hbbio 11y ago

In a side-note in the article, the author mentions that Amazon Glacier runs on tape. Although I'm not sure Amazon officially explained the technology, I heard (and read on Wikipedia) that they store data on conventional HDDs, that are however kept "offline". Has anyone here some more knowledge about that?

_dark_matter_ 11y ago
Yes please. I was also intrigued by this statement. I was under the impression that Amazon never told what the "secret sauce" was behind Glacier. Looking at Wikipedia now, it seems there are still multiple contending theories. Also, is it true that the service is not very popular? Are people here using it or similar?
- takasc2 11y ago
  Everyone knows it is not very popular - the limitations are extreme so you have to have a very specific use case for it.
- JimmaDaRustla 11y ago
  I'm using it for incremental backups of my photo collection. If I ever need to restore my entire collection, its available on S3/Glacier.
- deleted 11y ago
  [deleted]
- kstrauser 11y ago
  My home Synology unit does nightly backups to Glacier. It'd be terrible to lose our house to natural disaster, but losing all our digital data (photos, records, etc.) would make it just that much worse. For about $10 a month, I know I can recover all that stuff even if we lose everything else.
- teraflop 11y ago
  The big sticking point with Glacier is that the cost model for retrievals is complicated and unintuitive. There's a two-step process where you have to first retrieve data and then download it, and you're billed based on the peak retrieval rate in a month, not the total amount downloaded. It's not so bad now that they let you set a retrieval policy to limit your throughput, but before that, you had to carefully schedule your retrievals in order to avoid unexpected costs. As far as I can tell, Google's new nearline storage offering is superior to Glacier in nearly every way. The retrieval is much faster, and the retrieval costs are a flat rate per gigabyte that's equal to what Glacier gives you in the best case. Only drawback is the lack of a free tier.
  - discodave 11y ago
    You're not wrong, but if you look at the problem that Glacier is trying to solve then it doesn't look so bad: 1. Glacier is competing with tape. So for potential customers, if it's easier to acquire and maintain than your own tape library then that's a win. 2. In the words of James Hamilton the use case for Glacier is referred "jokingly to as Write Only Storage". If you're using Glacier for disaster recovery then paying a one time cost to retrieve a bunch of data pales in in comparison to the potential costs of not being able to recover your data. 3. Is nearline actually superior for the case of "oh shit I need to retrieve everything right now"? 4. AWS already has this cloud storage thingy called S3 and it has reduced redundancy storage if you're looking for something slightly cheaper. http://perspectives.mvdirona.com/2012/08/glacier-engineering-for-cold-data-storage-in-the-cloud/ http://perspectives.mvdirona.com/2012/08/glacier-engineering...
    - teraflop 11y ago
      I guess you've got a good point as far as #3 goes; the analysis is actually more interesting than I thought it was. Both Google's nearline storage and Glacier scale your retrieval capabilities proportionally to how much data you have stored with them. You can express it as a tradeoff between cost/GB and the time taken to retrieve your entire dataset Google throttles your downloads to a fixed rate, so they're effectively forcing you to one point in that space: (3 days, $0.01/GB). With Glacier, you get a curve depending on how fast you want it. It's much more expensive at the same speed (3 days, $0.10/GB) but you can go to one extreme of getting all your data for free over the course of 20 months, or getting it all within 4 hours for about $1.80/GB. So I guess it's fair to say that Glacier gives you more flexibility, but Google's offering is a much better value at the particular point that it optimizes for.
bglusman 11y ago
It was never confirmed, but I recall this[1] article making a pretty persuasive case Glacier was using optical disks. [1]https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...
JimmaDaRustla 11y ago
I'm probably explaining something most everyone here already knows, but: I don't know what Glacier uses, but the "offline" storage is much the same how mainframes use tape - disk based files (datasets in mainframe speak) are migrated to tape after a given amount of time to decrease storage costs. These datasets needs to be recalled back to disk to be used again. This is nearly identical to how Glacier operates with S3 as explained by Amazon. The reason tape isn't available immediately is because the tape drive, unlike a traditional harddrive, doesn't contain a reader. So multiple tapes typically sit in machines with only only a few readers that move back and forth based on requests made to read the tapes. If there are many requests, the amount of time to recall a dataset increases. Glacier may or may not be using tape, but the behavior is nearly identical to how tape has been used in past systems.
- Someone 11y ago
  You're referring to https://en.m.wikipedia.org/wiki/Hierarchical_storage_management https://en.m.wikipedia.org/wiki/Hierarchical_storage_managem.... AFAIK, most systems work at the file level, but if you address space is large enough, you could implement this as a virtual memory system that swaps RAM to disk, disk to slower disk, slower disk to tape, tapes to tapes stored offline, etc. I don't know what Amazon uses, either, but I expect Glacier data to be pushed out to some layer that doesn't need power while idle at all, heavily checksummed and also replicated across the globe. The relatively heavy prices tag for accessing it probably is there to keep contention on some relatively thin pipe low. For example, if they use blue ray, they may have only a few readers online for every 100 or 1000 customers, both to keep prices low and to differentiate Glacier more from their other products.
  - icebraining 11y ago
    if you address space is large enough, you could implement this as a virtual memory system that swaps RAM to disk, disk to slower disk, slower disk to tape, tapes to tapes stored offline, etc. Funny to imagine, - My program has been stuck copying a variable for over half an hour! - Yeah, there was a page fault and the tape loading guy still hasn't come from lunch
stephenjudkins 11y ago
Here's one theory: https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd... and here's the HN thread: https://news.ycombinator.com/item?id=7647571 https://news.ycombinator.com/item?id=7647571 Facebook uses Blu-ray discs with a robotic storage system for its cold storage: http://www.datacenterdynamics.com/it-networks/robot-digs-through-blu-ray-disks-to-find-your-old-facebook-photos/84832.fullarticle http://www.datacenterdynamics.com/it-networks/robot-digs-thr...
- agumonkey 11y ago
  It would be nice if Facebook released some stats, kinda like backblaze, about blu-ray durability.
  - sliverstorm 11y ago
    I thought degradation of optical medium was already pretty well understood? (And happened on timescales generally longer than FB has been in operation)
    - agumonkey 11y ago
      I was too young at the time optical medium was everywhere to search for actual data, and public claims were contradictory (5 years, 10 years .. more .. ). And that was CD-ROM density. Interestingly, http://blog.digistor.com/the-unparalleled-durability-of-blu-ray-burner-discs/ http://blog.digistor.com/the-unparalleled-durability-of-blu-... says that there were new design constraint in blurays making them much more durable.
    - deftnerd 11y ago
      Perhaps Amazon's Glacier center operates in a lights-out environment with a decreased oxygen environment to increase the life span of the discs.
    - stephenjudkins 11y ago
      There is Blu-ray compatible archival disk format that the manufacturers claim will last 1000 years: https://en.wikipedia.org/wiki/M-DISC https://en.wikipedia.org/wiki/M-DISC I don't know how this claim could be verified in our lifetimes.
discodave 11y ago
Pretty sure Amazon said publicly that they are not using tape for Glacier, I can't find the source except this second hand one: https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...
SixSigma 11y ago
Tape is guaranteed for 10 years without data loss
- wmf 11y ago
  Yet Google is using RAID-4 for their tape backups.
arca_vorago 11y ago
I think a lot of people may have confused Google's tape storage systems for Amazon's. I don't think Amazon is using tape, but the big G most definitely is: https://www.google.com/about/datacenters/gallery/images/_3000/MNK_005.jpg https://www.google.com/about/datacenters/gallery/images/_300...
ghaff 11y ago
Nothing authoritative. The links down-thread are almost all Robin Harris speculating about what it might be based on using various back of the envelope calculations. I'm actually pretty amazed that Amazon has been able to keep even minimal details off the web as far as I can tell. The closest I've seen to a direct statement from Amazon is: " An Amazon statement sent to Ars says only that "Glacier is built from inexpensive commodity hardware components," and is "designed to be hardware-agnostic, so that savings can be captured as Amazon continues to drive down infrastructure costs."" http://arstechnica.com/information-technology/2012/08/for-one-cent-a-month-amazon-glacier-stores-your-data-for-centuries/ http://arstechnica.com/information-technology/2012/08/for-on... FWIW, Google's competitive Nearline service almost certainly uses disks: The difference between Google’s Standard Storage and Nearline Storage slices on its public cloud comes down to a software layer, Tom Kershaw, director of product management for Google Cloud Platform, tells The Platform. We also think there might possibly be extra physical distance between the devices hosting the Nearline Storage and servers residing in the Google datacenters and network links that expose Cloud Platform services to the outside Internet. The Nearline Storage could also be running on slightly older equipment that is heavier on the disk drives but that is not beyond its technical or economic life. http://www.theplatform.net/2015/03/16/why-the-coldest-storage-at-google-is-nearline/ http://www.theplatform.net/2015/03/16/why-the-coldest-storag...

by hbbio 11y ago

_dark_matter_ 11y ago

Yes please. I was also intrigued by this statement. I was under the impression that Amazon never told what the "secret sauce" was behind Glacier. Looking at Wikipedia now, it seems there are still multiple contending theories. Also, is it true that the service is not very popular? Are people here using it or similar?

takasc2 11y ago
Everyone knows it is not very popular - the limitations are extreme so you have to have a very specific use case for it.
JimmaDaRustla 11y ago
I'm using it for incremental backups of my photo collection. If I ever need to restore my entire collection, its available on S3/Glacier.
deleted 11y ago
[deleted]
kstrauser 11y ago
My home Synology unit does nightly backups to Glacier. It'd be terrible to lose our house to natural disaster, but losing all our digital data (photos, records, etc.) would make it just that much worse. For about $10 a month, I know I can recover all that stuff even if we lose everything else.
teraflop 11y ago
The big sticking point with Glacier is that the cost model for retrievals is complicated and unintuitive. There's a two-step process where you have to first retrieve data and then download it, and you're billed based on the peak retrieval rate in a month, not the total amount downloaded. It's not so bad now that they let you set a retrieval policy to limit your throughput, but before that, you had to carefully schedule your retrievals in order to avoid unexpected costs. As far as I can tell, Google's new nearline storage offering is superior to Glacier in nearly every way. The retrieval is much faster, and the retrieval costs are a flat rate per gigabyte that's equal to what Glacier gives you in the best case. Only drawback is the lack of a free tier.
- discodave 11y ago
  You're not wrong, but if you look at the problem that Glacier is trying to solve then it doesn't look so bad: 1. Glacier is competing with tape. So for potential customers, if it's easier to acquire and maintain than your own tape library then that's a win. 2. In the words of James Hamilton the use case for Glacier is referred "jokingly to as Write Only Storage". If you're using Glacier for disaster recovery then paying a one time cost to retrieve a bunch of data pales in in comparison to the potential costs of not being able to recover your data. 3. Is nearline actually superior for the case of "oh shit I need to retrieve everything right now"? 4. AWS already has this cloud storage thingy called S3 and it has reduced redundancy storage if you're looking for something slightly cheaper. http://perspectives.mvdirona.com/2012/08/glacier-engineering-for-cold-data-storage-in-the-cloud/ http://perspectives.mvdirona.com/2012/08/glacier-engineering...
  - teraflop 11y ago
    I guess you've got a good point as far as #3 goes; the analysis is actually more interesting than I thought it was. Both Google's nearline storage and Glacier scale your retrieval capabilities proportionally to how much data you have stored with them. You can express it as a tradeoff between cost/GB and the time taken to retrieve your entire dataset Google throttles your downloads to a fixed rate, so they're effectively forcing you to one point in that space: (3 days, $0.01/GB). With Glacier, you get a curve depending on how fast you want it. It's much more expensive at the same speed (3 days, $0.10/GB) but you can go to one extreme of getting all your data for free over the course of 20 months, or getting it all within 4 hours for about $1.80/GB. So I guess it's fair to say that Glacier gives you more flexibility, but Google's offering is a much better value at the particular point that it optimizes for.

bglusman 11y ago

It was never confirmed, but I recall this[1] article making a pretty persuasive case Glacier was using optical disks. [1]https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...

JimmaDaRustla 11y ago

I'm probably explaining something most everyone here already knows, but: I don't know what Glacier uses, but the "offline" storage is much the same how mainframes use tape - disk based files (datasets in mainframe speak) are migrated to tape after a given amount of time to decrease storage costs. These datasets needs to be recalled back to disk to be used again. This is nearly identical to how Glacier operates with S3 as explained by Amazon. The reason tape isn't available immediately is because the tape drive, unlike a traditional harddrive, doesn't contain a reader. So multiple tapes typically sit in machines with only only a few readers that move back and forth based on requests made to read the tapes. If there are many requests, the amount of time to recall a dataset increases. Glacier may or may not be using tape, but the behavior is nearly identical to how tape has been used in past systems.

Someone 11y ago
You're referring to https://en.m.wikipedia.org/wiki/Hierarchical_storage_management https://en.m.wikipedia.org/wiki/Hierarchical_storage_managem.... AFAIK, most systems work at the file level, but if you address space is large enough, you could implement this as a virtual memory system that swaps RAM to disk, disk to slower disk, slower disk to tape, tapes to tapes stored offline, etc. I don't know what Amazon uses, either, but I expect Glacier data to be pushed out to some layer that doesn't need power while idle at all, heavily checksummed and also replicated across the globe. The relatively heavy prices tag for accessing it probably is there to keep contention on some relatively thin pipe low. For example, if they use blue ray, they may have only a few readers online for every 100 or 1000 customers, both to keep prices low and to differentiate Glacier more from their other products.
- icebraining 11y ago
  if you address space is large enough, you could implement this as a virtual memory system that swaps RAM to disk, disk to slower disk, slower disk to tape, tapes to tapes stored offline, etc. Funny to imagine, - My program has been stuck copying a variable for over half an hour! - Yeah, there was a page fault and the tape loading guy still hasn't come from lunch

stephenjudkins 11y ago

Here's one theory: https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd... and here's the HN thread: https://news.ycombinator.com/item?id=7647571 https://news.ycombinator.com/item?id=7647571 Facebook uses Blu-ray discs with a robotic storage system for its cold storage: http://www.datacenterdynamics.com/it-networks/robot-digs-through-blu-ray-disks-to-find-your-old-facebook-photos/84832.fullarticle http://www.datacenterdynamics.com/it-networks/robot-digs-thr...

agumonkey 11y ago
It would be nice if Facebook released some stats, kinda like backblaze, about blu-ray durability.
- sliverstorm 11y ago
  I thought degradation of optical medium was already pretty well understood? (And happened on timescales generally longer than FB has been in operation)
  - agumonkey 11y ago
    I was too young at the time optical medium was everywhere to search for actual data, and public claims were contradictory (5 years, 10 years .. more .. ). And that was CD-ROM density. Interestingly, http://blog.digistor.com/the-unparalleled-durability-of-blu-ray-burner-discs/ http://blog.digistor.com/the-unparalleled-durability-of-blu-... says that there were new design constraint in blurays making them much more durable.
  - deftnerd 11y ago
    Perhaps Amazon's Glacier center operates in a lights-out environment with a decreased oxygen environment to increase the life span of the discs.
  - stephenjudkins 11y ago
    There is Blu-ray compatible archival disk format that the manufacturers claim will last 1000 years: https://en.wikipedia.org/wiki/M-DISC https://en.wikipedia.org/wiki/M-DISC I don't know how this claim could be verified in our lifetimes.

discodave 11y ago

Pretty sure Amazon said publicly that they are not using tape for Glacier, I can't find the source except this second hand one: https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...

SixSigma 11y ago

Tape is guaranteed for 10 years without data loss

wmf 11y ago
Yet Google is using RAID-4 for their tape backups.

arca_vorago 11y ago

I think a lot of people may have confused Google's tape storage systems for Amazon's. I don't think Amazon is using tape, but the big G most definitely is: https://www.google.com/about/datacenters/gallery/images/_3000/MNK_005.jpg https://www.google.com/about/datacenters/gallery/images/_300...

ghaff 11y ago

Nothing authoritative. The links down-thread are almost all Robin Harris speculating about what it might be based on using various back of the envelope calculations. I'm actually pretty amazed that Amazon has been able to keep even minimal details off the web as far as I can tell. The closest I've seen to a direct statement from Amazon is: " An Amazon statement sent to Ars says only that "Glacier is built from inexpensive commodity hardware components," and is "designed to be hardware-agnostic, so that savings can be captured as Amazon continues to drive down infrastructure costs."" http://arstechnica.com/information-technology/2012/08/for-one-cent-a-month-amazon-glacier-stores-your-data-for-centuries/ http://arstechnica.com/information-technology/2012/08/for-on... FWIW, Google's competitive Nearline service almost certainly uses disks: The difference between Google’s Standard Storage and Nearline Storage slices on its public cloud comes down to a software layer, Tom Kershaw, director of product management for Google Cloud Platform, tells The Platform. We also think there might possibly be extra physical distance between the devices hosting the Nearline Storage and servers residing in the Google datacenters and network links that expose Cloud Platform services to the outside Internet. The Nearline Storage could also be running on slightly older equipment that is heavier on the disk drives but that is not beyond its technical or economic life. http://www.theplatform.net/2015/03/16/why-the-coldest-storage-at-google-is-nearline/ http://www.theplatform.net/2015/03/16/why-the-coldest-storag...