3 ms·
For these large scale infrastructures, they typically use JBOD, i.e. no RAID whatsoever and they achieve redundancy by maintaining multiple copies of the data o
by cm2187 7y ago
For these large scale infrastructures, they typically use JBOD, i.e. no RAID whatsoever and they achieve redundancy by maintaining multiple copies of the data over multiple disks spread around the datacentre. So it's not hardware RAID that requires a certain response time (I suspect the drives mentioned here drop because they timeout as it take a long time to random write on an SMR disk), more like a distributed software RAID. I think they likely also have their own file system so they are not dependent on a fixed block size, which allows them to save space if they have loads of small files.
- smueller1234 7y agoI think that's about half right: for large storage infrastructure, you would indeed eschew local RAID, but instead of storing multiple copies, you'd use Reed Solomon encoding to stripe a single copy across many disks/servers/failure domains with a configurable number of added parity stripes. Full copies are really expensive!
- cm2187 7y agoI believe azure uses full copies [1]. I assume the other cloud providers must do the same. [1] https://docs.microsoft.com/en-us/azure/storage/common/storage-redundancy https://docs.microsoft.com/en-us/azure/storage/common/storag...
- namibj 7y agoGoogle seems to use erasure coding for some (iirc. multi-AZ nearline, or so) storage classes, last I checked.
- Serow225 7y agohttps://www.backblaze.com/blog/reed-solomon/ https://www.backblaze.com/blog/reed-solomon/