Here's a crazy but interesting suggestion: > Backup > ...Even with RAID overhead it should be possible to have 480TB of usable storage (66%). Quoting https:/

Here's a crazy but interesting suggestion: > Backup > ...Even with RAID overhead it should be possible to have 480TB of usable storage (66%). Quoting https://code.facebook.com/posts/1433093613662262/-under-the-hood-facebook-s-cold-storage-system-/ https://code.facebook.com/posts/1433093613662262/-under-the-...: > Fortunately, with a technique called erasure coding, we can. Reed-Solomon error correction codes are a popular and highly effective method of breaking up data into small pieces and being able to easily detect and correct errors. As an example, if we take a 1 GB file and break it up into 10 chunks of 100 MB each, through Reed-Solomon coding, we can generate an additional set of blocks, say four, that function similar to parity bits. As a result, you can reconstruct the original file using any 10 of those final 14 blocks. So, as long as you store those 14 chunks on different failure domains, you have a statistically high chance of recovering your original data if one of those domains fails. Facebook didn't release the system they used to do this. I can see two reasons why not to: desire for competitive edge; or the implementation not being a general-purpose solution. Considering Facebook's general openness, I say get in touch, just in case! It's quite possible that you might be able to figure out something interesting. I suspect the reason the system wasn't released was due to the latter case - it seems to be technically quite simple and easily achievable for a[ny] company full of algorithms Ph.Ds.