4 ms·
Yes. The combination of erasure coding plus regular scrubbing is already the standard in the largest proprietary storage systems. As it happens, I worked on exa
by notacoward 6y ago
Yes. The combination of erasure coding plus regular scrubbing is already the standard in the largest proprietary storage systems. As it happens, I worked on exactly the scrubbing piece ("anti entropy") for such a system at Facebook. There was a lot of analysis of data-loss probabilities based on encodings, placement across power/network domains, scrub rates, repair rates, etc. Since there are also performance and resource-use implications behind many of these, it's actually a very complex balancing act. That's why the knowledge of these second- or third-order problems and their solutions is slowly filtering down to storage systems you can deploy yourself.