4 ms·

I've been looking into Glacier, but the pricing in the case where I might want to retrieve a substantial portion of my data is quite complex. As far as I can te

I've been looking into Glacier, but the pricing in the case where I might want to retrieve a substantial portion of my data is quite complex. As far as I can tell, the headline $0.01/GB rate for overages they quote is misleading. They don't, as you might expect from that language, calculate ([GB retrieved] - [5% of GB stored]) x $0.01 and bill you that. Rather, they charge you on a monthly basis, according to your highest hourly retrieval rate from any single hour applied to the whole month (standardized at 720 hours). So if your peak hourly retrieval in a month was 10 GB/hr above your pro-rated free quota for that hour, you're charged for 7200 GB of retrieval overage, or $72, even if you retrieved nowhere near 7 TB of data. For companies with relatively constant retrieval this doesn't matter, but for small users, someone who has a large overage one day will be charged as if they were incurring the same overage continually all month. To put it concretely, if you store 200GB of photos and then do a bulk retrieval, you aren't charged $1.90 ((200-10) x 0.01). Rather, the xfer is counted as 4 hours, so you have a peak transfer of 50 GB/hr. Your free daily retrieval is allocated to the hours in the day you retrieved data, so in this case 10/4 = 2.5 GB/hr free. So your overage is 47.5 GB/hr. Multiply by 720 and $0.01, and the fee for retrieving 200GB of photos is $342. Plus $22.88 in outgoing AWS bandwidth. Or at least, I think that's what their pricing page says. Maybe someone else can better interpret this explanation: http://aws.amazon.com/glacier/faqs/#How_will_I_be_charged_when_retrieving_large_amounts_of_data_from_Amazon_Glacier http://aws.amazon.com/glacier/faqs/#How_will_I_be_charged_wh...

jychang 13y ago
This should be mitigated if you're running a service, though. It averages out.
- wmf 13y ago
  Glacier retrieval pricing has a strange and un-Amazony economy of scale. If you're just backing up your own data you'll be charged a ZOMG huge amount to ever download it. But if you're backing up 100 people you pay nothing to get data back.
  - _delirium 13y ago
    Yeah, I wasn't expecting to find it here. This style of pricing isn't entirely unreasonable, and you do find charges based on peak monthly usage elsewhere (it's reminiscent of 95th-percentile bandwidth pricing, for example). But AWS otherwise has pretty straightforward pricing based on micro-accounting of usage, so this seems atypical for them.
tnorthcutt 13y ago
Holy crap. I started mulling over the idea of using Glacier as a cloud backup to my Synology NAS, but this makes me extremely hesitant.
csears 13y ago
I seem to recall a Glacier discussion from a while back suggesting that the recovery fees might be covered by insurance (homeowners, loss of business, etc). If you're primarily concerned about doing a restore after a fire, theft, or natural disaster, it might be worth looking into.
sreitshamer 13y ago
Right, you definitely shouldn't request all 200GB at the same time, because it'd be expensive and also because you probably can't download all 200GB immediately either. Arq (a backup app I wrote) takes a transfer rate from you (guesses your max transfer rate as a default) and then requests enough to be downloaded in 4 hours at that rate. 4 hours later, it requests another 4 hours' worth while it downloads the first 4 hours' worth, and so on. This mitigates the peak-transfer fee.
- aarondf 13y ago
  just downloaded arq. looks amazing, can't believe I didn't know about this before.
- jasondenizac 13y ago
  Does Arq fit the OP's use case of backing up files to AWS without keeping them on your local machine?

jychang 13y ago

This should be mitigated if you're running a service, though. It averages out.

wmf 13y ago
Glacier retrieval pricing has a strange and un-Amazony economy of scale. If you're just backing up your own data you'll be charged a ZOMG huge amount to ever download it. But if you're backing up 100 people you pay nothing to get data back.
- _delirium 13y ago
  Yeah, I wasn't expecting to find it here. This style of pricing isn't entirely unreasonable, and you do find charges based on peak monthly usage elsewhere (it's reminiscent of 95th-percentile bandwidth pricing, for example). But AWS otherwise has pretty straightforward pricing based on micro-accounting of usage, so this seems atypical for them.

tnorthcutt 13y ago

Holy crap. I started mulling over the idea of using Glacier as a cloud backup to my Synology NAS, but this makes me extremely hesitant.

csears 13y ago

I seem to recall a Glacier discussion from a while back suggesting that the recovery fees might be covered by insurance (homeowners, loss of business, etc). If you're primarily concerned about doing a restore after a fire, theft, or natural disaster, it might be worth looking into.

sreitshamer 13y ago

Right, you definitely shouldn't request all 200GB at the same time, because it'd be expensive and also because you probably can't download all 200GB immediately either. Arq (a backup app I wrote) takes a transfer rate from you (guesses your max transfer rate as a default) and then requests enough to be downloaded in 4 hours at that rate. 4 hours later, it requests another 4 hours' worth while it downloads the first 4 hours' worth, and so on. This mitigates the peak-transfer fee.

aarondf 13y ago
just downloaded arq. looks amazing, can't believe I didn't know about this before.
jasondenizac 13y ago
Does Arq fit the OP's use case of backing up files to AWS without keeping them on your local machine?