3 ms·
I'm kind of glad Redis did the fork approach first. It's the reason I went with a userspace COW implementation in my work instead of forking and that paid huge
by arielweisberg 12y ago
I'm kind of glad Redis did the fork approach first. It's the reason I went with a userspace COW implementation in my work instead of forking and that paid huge dividends. It's the difference between starting COW in 10-20 milliseconds versus seconds and most of that time is distributed coordination not flipping the COW boolean.
When you crank up the density per node to 256 or 512 gigabytes even bare metal is problematic and in some domains like telecommunications they don't care that the spikes are concentrated because they cause cascading failures.
I think a userspace COW implementation in Redis would be a big project because you would need a different strategy for every data structure. Being single threaded also makes it challenging to get other/more cores to do the serialization and IO. It's very doable just not within the current philosophy of thou shalt have one thread per process.
- eldavido 12y agoI think "userspace COW" is the wrong approach here. The entire idea of getting a point-in-time snapshot of something subject to rapid change is problematic. Your options boil down to (1) make a "snapshot" and save that (Redis's current approach) or (2) accept that point-in-time consistency might be impossible, and work around it. I wonder whether it'd be possible to have two persistence strategies in Redis: "consistent" and "low-latency". "Consistent" would use the current fork(2) COW behavior, "low-latency" would do some kind of one-chunk-at-a-time block copy and amortize the latency spike over the entire operation, while having the overall effect of less of a "cliff" to latency.
- deleted 12y ago[deleted]