4 ms·

So here’s a question: let’s say I store a backup in AWS Glacier. Do they just store it and forget it till I try to access, or copy it to new media every month/

by yumraj 2y ago

So here’s a question: let’s say I store a backup in AWS Glacier. Do they just store it and forget it till I try to access, or copy it to new media every month/year to make sure it’s still accessible? Same question for Backblaze B2 and such. Does anyone know?

thrtythreeforty 2y ago
"Durability" is the term you're looking for. S3 and Glacier famously have 99.999999999% durability. The quote goes "if you store 10,000,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000 years." Backblaze makes similar claims.
- gavindean90 2y ago
  I think the commenter was asking if they refresh the storage enough for it to not die if natural causes. If data is left around on most storage media it will be gone within 75 years if not refreshed somehow.
  - yumraj 2y ago
    Exactly..
- saagarjha 2y ago
  This is only relevant if you have billions of objects in S3, when your failure rate can be amortized. Otherwise either Amazon is going to go bankrupt or you’ll be the one person who has total data loss by chance.
  - hinkley 2y ago
    Already happened didn’t it?
trevyn 2y ago
It’s M-of-N erasure coded and periodically verified. So yes, they copy it to new media, after it has failed. Math is crazy. Glacier might also have a heuristic for media lifetime, but ehhh.
krater23 2y ago
Counterquestion: How much money do they lose when you notice in 50 years that your data are gone? And how much money you payd until then? So, it's just simple math that you can use to calculate your answer.
- yumraj 2y ago
  You completely missed the intent behind my question. I’m trying to figure out an appropriate strategy for backing up things that are important to me. If they move things around, great, else I’ll need to do it myself.
telgareith 2y ago
AWS does regular integrity checks on all media, and repairs any errors it finds. This includes Glacier and Deep Glacier. (It's in the first few bits of the S3 docs) Backblaze B2 is untrustworthy. I specified a bad encryption key- and it accepted the data, but it wasn't retreiveable. For the extra $1/TB/mo Wasabi is the best For hot backups, IMHO. Also, I theorize that you can avoid the major cost of AWS Glacier deep archive by downloading it through CloudFront- which has 1TB/no free. So: 1. Make your restore request near the end of the month. 2. Glacier restore ~2TB of days (bulk restore, slow.) to S3 hot tier. 3. Download 1TB while on the tail end of the first month, and another 1TB when CloudFront resets. The huge egress costs are minimized this way, as well as minimizing the GB/s charges of regular S3.
- e40 2y ago
  > Backblaze B2 is untrustworthy. I specified a bad encryption key- and it accepted the data, but it wasn't retreiveable. That’s an odd dig at B2. If you had done the bare minimum to test your backup it wouldn’t have been an issue.

by yumraj 2y ago

thrtythreeforty 2y ago

"Durability" is the term you're looking for. S3 and Glacier famously have 99.999999999% durability. The quote goes "if you store 10,000,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000 years." Backblaze makes similar claims.

gavindean90 2y ago
I think the commenter was asking if they refresh the storage enough for it to not die if natural causes. If data is left around on most storage media it will be gone within 75 years if not refreshed somehow.
- yumraj 2y ago
  Exactly..
saagarjha 2y ago
This is only relevant if you have billions of objects in S3, when your failure rate can be amortized. Otherwise either Amazon is going to go bankrupt or you’ll be the one person who has total data loss by chance.
- hinkley 2y ago
  Already happened didn’t it?

trevyn 2y ago

It’s M-of-N erasure coded and periodically verified. So yes, they copy it to new media, after it has failed. Math is crazy. Glacier might also have a heuristic for media lifetime, but ehhh.

krater23 2y ago

Counterquestion: How much money do they lose when you notice in 50 years that your data are gone? And how much money you payd until then? So, it's just simple math that you can use to calculate your answer.

yumraj 2y ago
You completely missed the intent behind my question. I’m trying to figure out an appropriate strategy for backing up things that are important to me. If they move things around, great, else I’ll need to do it myself.

telgareith 2y ago

AWS does regular integrity checks on all media, and repairs any errors it finds. This includes Glacier and Deep Glacier. (It's in the first few bits of the S3 docs) Backblaze B2 is untrustworthy. I specified a bad encryption key- and it accepted the data, but it wasn't retreiveable. For the extra $1/TB/mo Wasabi is the best For hot backups, IMHO. Also, I theorize that you can avoid the major cost of AWS Glacier deep archive by downloading it through CloudFront- which has 1TB/no free. So: 1. Make your restore request near the end of the month. 2. Glacier restore ~2TB of days (bulk restore, slow.) to S3 hot tier. 3. Download 1TB while on the tail end of the first month, and another 1TB when CloudFront resets. The huge egress costs are minimized this way, as well as minimizing the GB/s charges of regular S3.

e40 2y ago
> Backblaze B2 is untrustworthy. I specified a bad encryption key- and it accepted the data, but it wasn't retreiveable. That’s an odd dig at B2. If you had done the bare minimum to test your backup it wouldn’t have been an issue.