4 ms·
> Rather than allowing multiple gigabytes of outstanding buffered writes and deferring writeback until a gigabyte or more has accumulated, you'd set things to t
by toast0 3y ago
> Rather than allowing multiple gigabytes of outstanding buffered writes and deferring writeback until a gigabyte or more has accumulated, you'd set things to trigger writebacks almost immediately and then force processes doing write IO to wait for disk writes to complete once you have more than a relatively small volume of outstanding writes.
I think having the trigger be size based rather than timebased is the real problem. Or bounds on both...
I probably don't want to buffer writes for more than X seconds, or let the buffer grow beyond Y% of ram. At least for the time based limit, you'd really want to be able to say start writing to disk when the buffer has data over 10 seconds old, but still accept writes into a new buffer, only blocking writes when there's a buffer being written out and the current buffer is too old or too big.
- loeg 3y agoIdeally the system starts flushing buffered writes basically immediately, but with low device-level queue depth so that subsequently issued higher priority IOs do not suffer from a ton of additional latency.
- magicalhippo 3y agoZFS has both a time-based and size-based limit. IIRC the default is 15 seconds and lower of 2GB and some % of system memory. Though in ZFS' case it's not really a regular write cache as such, as it's used to minimize updates to its on-disk copy-on-write structure.
- vlovich123 3y agoYou're mostly right in that it is similar to the problem of buffer bloat in networking. However, it's not quite the same thing because for example you can do things like write a file to disk & unlink it or overwrite some portion of contents, meaning that by buffering for longer you can avoid the writeback in the first place. By buffering for longer, the kernel is trying to balance things landing on disk and avoiding touching the disk if the dirtied data will be dirtied again. Granted not a common use-case these days, but consider the case where you have object files being created during a build over & over again. It's easy to construct scenarios where you indeed wouldn't want to write to disk so quickly. It's not out of hand a terrible idea to avoid flushing data to disk and there's no free lunch here as any workload you optimize for will have a different workload that suffers. People try to come up with general heuristics that work in most situations on consumer machines, but there's no one size fits all for all HW + use-case combos. That's why hyperscalars tune the kernel beyond that / have kernel developers writing code to optimize for their use-case. It's telling that the performance analysis in the article is pretty hand-wavy without any clear demonstration of a concrete problem. As for sync, I believe the author is mistaken. You can do fsync instead which is more efficient as it only creates a barrier for writeback of the file descriptor rather than a system-wide sync. And invoking fsync I believe is more common than sync. You should be able to have multiple parallel fsync happening concurrently for unrelated files that don't block on each other so much (ideally the kernel would prioritize those writebacks and interleave for fairness, but I doubt it does).
- charleshn 3y agoIt is the case, see the vm.dirty_expire_centisecs sysctl [0] at the VM layer, and the commit interval for e.g. ext4 [1] at the filesystem layer, and [2] for their interaction. [0] https://docs.kernel.org/admin-guide/sysctl/vm.html#dirty-expire-centisec https://docs.kernel.org/admin-guide/sysctl/vm.html#dirty-exp... [1] https://www.kernel.org/doc/html/v4.19/filesystems/ext4/ext4.html#options https://www.kernel.org/doc/html/v4.19/filesystems/ext4/ext4.... [2] https://serverfault.com/a/995887 https://serverfault.com/a/995887