14 ms·
I do not follow. How is it possible that the latency is lower in a 4-of-5 read of a coded stripe compared to a 1-of-4 replicated stripe?
by jeffbee 3y ago
I do not follow. How is it possible that the latency is lower in a 4-of-5 read of a coded stripe compared to a 1-of-4 replicated stripe?
- ec109685 3y agoAren’t they comparing 4 of 5 with 4 of 4?
- jeffbee 3y agoIf so that doesn't make a great deal of sense because the alternative to a coded stripe isn't a unique unreplicated stripe, it's a replicated one. So the alternative would be to race multiple reads of all known replicas, or to hedge them.
- ec109685 3y agoGood point. I guess it’s really 2 out of 2 versus 4 out of 6.
- benlivengood 3y agoFull replication is probably always lower latency but costs N times as many copies that you want to store whereas erasure coding costs N/M. The cost of replicating write traffic is similar.
- jeffbee 3y agoCertainly, but there are other tradeoffs. You can write a byte to a replicated stripe, but encoded stripes either demand full-stripe writes or writing a byte is hilariously expensive. It's not clear in the blog if we are talking about sealed read-only data or what.
- benlivengood 3y agoPractically all modern storage has a minimum sector write size; now commonly 4096 bytes for HDD and megabytes for NAND. Most systems with fancy erasure codes address this by gathering multiple writes into single stripes and are effectively append-only filesystems with garbage collection to support snapshots. A large number of RS erasure codes are over GF(2^8) which could allow individual byte updates to a stripe for e.g. Octane as storage. Generally, of course, most filesystems checksum at a block level and so byte-writes are rarely going to be implemented as such.
- vlovich123 3y agoSmall correction I think. The 4KiB for HDDs is fairly new and afaik most NAND is going to also be 4KiB with the exception of QLC which last I saw had a 64KiB minimum sector write size. If you have a source that NAND flash is using MiB minimum sector size, can you point me to that link?
- toast0 3y ago4k sectors for hard drives has been around since roughly 2010, depending on your specific drive. 512e drives are very common; if you do a write that's not 4k aligned, the drive will turn it into a read / modify / write cycle.