3 ms·

Yev from Backblaze -> That meteor question comes up a lot (https://www.backblaze.com/blog/vault-cloud-storage-architecture/#comment-1901229080 https://www.backb

by atYevP 12y ago

Yev from Backblaze -> That meteor question comes up a lot (https://www.backblaze.com/blog/vault-cloud-storage-architecture/#comment-1901229080 https://www.backblaze.com/blog/vault-cloud-storage-architect...). We currently do have one data center, but this design allows us to bring others online. If the datacenter was hit by a meteor all our customers would get an email blast urging them to create a local backup immediately. The chances that both the DC and the user would get hit by the same natural disaster are relatively small. Still, it's not a storage service like S3 so geo-redundancy plays a smaller role. We do plan on building out other datacenters in the future, but since we're bootstrapped, we have to do that when the time is right, otherwise it would be very easy to over-extend ourselves and start losing money. edit -> I ignored your Backblaze dev question, sorry. We have multiple processes running at all times on the pods, and they go shard-by-shard. We're always optimizing, but the short answer is, we're always looking for errors.

jacquesm 12y ago
It doesn't take a meteor. I got hit when EV1 had a fire in one of their DCs, fortunately we had a local backup from which we restored and kept on running but a lot of companies were not in that position and had a hard time surviving. EV1 was hurt badly by this, it's not just meteors. What happened was that a transformer on the floor exploded, took a dividing wall with it and cause a (surprisingly!) relatively minor fire. What took down the DC for several weeks was the firedepartements investigation. They took their time to figure out the root cause of the fire which is their right but the collateral damage of that was substantial. So don't just plan for meteors.
- jewel 12y ago
  The nice thing from backblaze's perspective and their customers is that downtime is far more tolerable than it would be for most businesses. Most disasters that are going to impact a data center aren't going to destroy the physical hard drives, assuming the data center has the usual safeguards in place.
  - brianwski 12y ago
    Brian from Backblaze here-> this is definitely true. If you ask to have a 5 TByte restore prepared, it will take us a full 22 hours to get that all assembled for you. If you want us to FedEx the prepared restore on a USB hard drive, it will take ANOTHER 24 hours, and if you are in Europe it's more like 48 hours. And what's luxurious about "backup" as a business is this doesn't bother many customers. As long as we keep communicating to them on the progress, and we assure them they are going to get every solitary bit/byte/jpeg/mp3/movie back - they often tell us to take our time and do it right. For "backup" accurate and durable is about a thousand times more important than "instant gratification".

by atYevP 12y ago

jacquesm 12y ago

It doesn't take a meteor. I got hit when EV1 had a fire in one of their DCs, fortunately we had a local backup from which we restored and kept on running but a lot of companies were not in that position and had a hard time surviving. EV1 was hurt badly by this, it's not just meteors. What happened was that a transformer on the floor exploded, took a dividing wall with it and cause a (surprisingly!) relatively minor fire. What took down the DC for several weeks was the firedepartements investigation. They took their time to figure out the root cause of the fire which is their right but the collateral damage of that was substantial. So don't just plan for meteors.

jewel 12y ago
The nice thing from backblaze's perspective and their customers is that downtime is far more tolerable than it would be for most businesses. Most disasters that are going to impact a data center aren't going to destroy the physical hard drives, assuming the data center has the usual safeguards in place.
- brianwski 12y ago
  Brian from Backblaze here-> this is definitely true. If you ask to have a 5 TByte restore prepared, it will take us a full 22 hours to get that all assembled for you. If you want us to FedEx the prepared restore on a USB hard drive, it will take ANOTHER 24 hours, and if you are in Europe it's more like 48 hours. And what's luxurious about "backup" as a business is this doesn't bother many customers. As long as we keep communicating to them on the progress, and we assure them they are going to get every solitary bit/byte/jpeg/mp3/movie back - they often tell us to take our time and do it right. For "backup" accurate and durable is about a thousand times more important than "instant gratification".