4 ms·

The problem is every filesystem would implement a different version of fsync_that_works or rename_for_safety and then applications will have to call all of them

The problem is every filesystem would implement a different version of fsync_that_works or rename_for_safety and then applications will have to call all of them, and then people will start building a compat framework that tries to assemble the optimal sequence of calls for each filesystem except it won't work in every case and so a new fsync_but_no_cheating function will be proposed.

appplication 3y ago
Does this suggest that the idea of an overarching fsync/rename is perhaps being proposed at the wrong level of abstraction? If it can’t be reliably implemented at every subsequent level, what’s the point in having it at this level?
- tedunangst 3y ago
  Different filesystems have different on disk structures, with varying degrees of complexity and cost in getting those things on disk. Everything can reliably get everything written eventually, but if you want to fully utilize your disk bandwidth, you only want to flush the data that can get there fast, which is going to vary by filesystem. And different applications have different durability requirements.
  - layer8 3y ago
    This is still no argument to not define syscalls for the semantics that applications typically care about. If it is slow on a particular file system, then so be it, it’s still the semantics the application requires.
    - tedunangst 3y ago
      So call fsync.
dale_glass 3y ago
The trick is not allowing such a thing. Don't have something vague like "fsync_that_works" that could be implemented arbitrarily. Have a function that is documented to implement a highly specific contract, like "writes all pending blocks of this file to disk, and ensures they'll be there if there's a crash after the function returns".
- fl0ki 3y ago
  That's exactly what people expect from fsync(). The problem explained in the article is that, due to the difficulties of tracing every possible related write, it is (was?) often implemented conservatively and waited for more writes to land than were strictly necessary. I don't know how much this tracing has improved in various kernels since 2009, but I do know that I would want them to benefit fsync(), not to make a new function with a subtly different contract.

appplication 3y ago

Does this suggest that the idea of an overarching fsync/rename is perhaps being proposed at the wrong level of abstraction? If it can’t be reliably implemented at every subsequent level, what’s the point in having it at this level?

tedunangst 3y ago
Different filesystems have different on disk structures, with varying degrees of complexity and cost in getting those things on disk. Everything can reliably get everything written eventually, but if you want to fully utilize your disk bandwidth, you only want to flush the data that can get there fast, which is going to vary by filesystem. And different applications have different durability requirements.
- layer8 3y ago
  This is still no argument to not define syscalls for the semantics that applications typically care about. If it is slow on a particular file system, then so be it, it’s still the semantics the application requires.
  - tedunangst 3y ago
    So call fsync.

dale_glass 3y ago

The trick is not allowing such a thing. Don't have something vague like "fsync_that_works" that could be implemented arbitrarily. Have a function that is documented to implement a highly specific contract, like "writes all pending blocks of this file to disk, and ensures they'll be there if there's a crash after the function returns".

fl0ki 3y ago
That's exactly what people expect from fsync(). The problem explained in the article is that, due to the difficulties of tracing every possible related write, it is (was?) often implemented conservatively and waited for more writes to land than were strictly necessary. I don't know how much this tracing has improved in various kernels since 2009, but I do know that I would want them to benefit fsync(), not to make a new function with a subtly different contract.