4 ms·

Additionally, it implies that data needs to be spread across 10,000 drives, which is unrealistic anyway.

• brianwski 8y ago
  Disclaimer: I wrote the blog post. :-) > spread across 10,000 drives, which is unrealistic I claim it is also undesirable. Backblaze specifically made the conscious decision that the parts of any one single "large file" (these can be up to 10 TBytes each) are all stored within the same "vault". A vault is 20 computers in 20 separate racks. This allows a single vault to check the consistency and integrity of a large file periodically without communicating to other vaults in the datacenter. The vaults have been a really good unit of scaling for Backblaze. If the vaults can maintain their performance, then we know we can just stamp out more vaults because there is almost no communication between vaults.
• Dylan16807 8y ago
  As to the one second claim, I think that's a math error, because even 10k * 10MB is only 100GB. But spreading the data over 10k drives isn't unrealistic, it's a different architecture. Pick a different 20 drives for each file. Working it through: Assume 200 machines with 50 drives each. Each machine has to read 1TB, transmit it over the network, do a parity calculation, and write out ~50GB. With dual 10gbps ports the bottleneck is the network, and if we dedicate one on each machine to the rebuild we get a 15 minute clock. Not that having such a monolithic architecture is worth the complication and extra bugs.