3 ms·

https://redis.io/topics/latency#latency-induced-by-transparent-huge-pages https://redis.io/topics/latency#latency-induced-by-transpare...

anarazel 8y ago
IOW, a bad persistence design in redis currently interacts with a suboptimal COW-after-fork implementation in linux.
- antirez 8y ago
  There is nothing wrong in the Linux COW implementation. The problem is that 2MB pages are too much to duplicate for a single byte change. The 4kb page works very well to do that instead, and actually is very hard to do better in user-space with a different persistence model. AFAIK only thing that could be improved in Linux is that when THP are enabled, it should split the 2MB page into 4kg pages and copy just that.
  - anarazel 8y ago
    > The problem is that 2MB pages are too much to duplicate for a single byte change. Splitting up such pages into their 4kb counterparts when dirtying a COWed huge page isn't particularly crazy. I don't think there's much reason to do that when huge pages are used, but given THP it seems pretty obvious that there's potential for substantial regressions without that feature. > and actually is very hard to do better in user-space with a different persistence model. Howso? It's not like management of a buffering layer over disks is a particularly new thing. There's plenty of different data stores that have persistence models that don't have the problems (uneven write rate, explosion in memory usage after fork, latency increases due to the increase in page misses, ...) of fork() and then writing out the data in the fork.
    - antirez 8y ago
      Here the problem is different: write on disk a point in time snapshot of what is in memory. Redis is not a common data store.
  - the8472 8y ago
    Maybe they could avoid this issue by cloning in smaller increments via memfd + mmap(..., MAP_PRIVATE)? That's assuming persistence doesn't fundamentally require atomic clone of the whole address space.
  - trasz 8y ago
    Which is, I believe, exactly what FreeBSD does.

anarazel 8y ago

IOW, a bad persistence design in redis currently interacts with a suboptimal COW-after-fork implementation in linux.

antirez 8y ago
There is nothing wrong in the Linux COW implementation. The problem is that 2MB pages are too much to duplicate for a single byte change. The 4kb page works very well to do that instead, and actually is very hard to do better in user-space with a different persistence model. AFAIK only thing that could be improved in Linux is that when THP are enabled, it should split the 2MB page into 4kg pages and copy just that.
- anarazel 8y ago
  > The problem is that 2MB pages are too much to duplicate for a single byte change. Splitting up such pages into their 4kb counterparts when dirtying a COWed huge page isn't particularly crazy. I don't think there's much reason to do that when huge pages are used, but given THP it seems pretty obvious that there's potential for substantial regressions without that feature. > and actually is very hard to do better in user-space with a different persistence model. Howso? It's not like management of a buffering layer over disks is a particularly new thing. There's plenty of different data stores that have persistence models that don't have the problems (uneven write rate, explosion in memory usage after fork, latency increases due to the increase in page misses, ...) of fork() and then writing out the data in the fork.
  - antirez 8y ago
    Here the problem is different: write on disk a point in time snapshot of what is in memory. Redis is not a common data store.
- the8472 8y ago
  Maybe they could avoid this issue by cloning in smaller increments via memfd + mmap(..., MAP_PRIVATE)? That's assuming persistence doesn't fundamentally require atomic clone of the whole address space.
- trasz 8y ago
  Which is, I believe, exactly what FreeBSD does.