3 ms·
Your proposed scheme might work, but the odds of a system failure are extremely high (like, what if 2 servers go out instead of one?). How hard is your constra
by chris_va 13y ago
Your proposed scheme might work, but the odds of a system failure are extremely high (like, what if 2 servers go out instead of one?).
How hard is your constraint? Can you just use an aggressive Reed Solomon codec with a ~1.5x blowup factor (so, 100TB of extra space instead of 50TB), and spread the chunks around to each of the machines?
- nanch 13y agofailure of 2 servers is a risk for the proposed design. for a v2, the system could use Reed-Solomon on the backup-server to add entropy and support the simultaneous failure of N servers. Yes, a distributed design with the backup blocks distributed across servers would be adequate as well, even without the RS blowup factor.
- chris_va 13y agoThis is your best bet: Distributing blocks to all servers evenly. Using RS to encode your blocks to limit storage space requirements, and make recovery obvious. Also, make sure you don't buy all of your hard drives from the same end manufacturer. Most of these types of systems assume random hard drive failure... turns out drives can have highly correlated failures.