2 ms·
> The fact that lightning might have struck the drive just exactly then is not your problem [...] If you think you might need fsync(), you don't. [...] Those th
by lxgr 2y ago
> The fact that lightning might have struck the drive just exactly then is not your problem [...] If you think you might need fsync(), you don't. [...] Those things all already have their own layers with their own responsibilities to be doing all the necessary testing [...]
You're mixing up the concepts of durability and consistency in pretty significant ways here, and are implying that everybody that is fine with a lack of the former will also be fine with a lack of the latter.
This is absolutely not true, and can cause extremely painful and hard-to-track-down bugs.
For better or worse, there is no way to directly tell the OS "whatever you do, make sure you don't reorder these writes I just did with those other writes I'm about to do".
The next best (portable) thing we have to achieve that outcome is fsync. It's a bit heavy handed, in that it gives you durability even if you only want consistency. That absolutely doesn't mean it's redundant, though.
> Basically if you aren't writing the filesystem itself, then you shouldn't be calling fsync().
Given that fsync is a syscall, but file systems are generally implemented in the kernel, this is a pretty nonsensical statement by itself.
File systems usually have (and need, for performance) a much lower level view of the underlying block storage, and fine-grained control over it.
Just as one example, Linux has the concept of write barriers. Not using these correctly (in the filesystem driver) can cause data leaks across files owned by different users and processes.
- Brian_K_White 2y ago"there is no way to directly tell the OS "whatever you do, make sure you don't reorder these writes I just did with those other writes I'm about to do" Perhaps because there is no reason for such a thing to exist. Tell me an example.
- lxgr 2y agoText editors: I want my file content to either be what it was before I pressed "save", or what it was afterwards; I don't want my file to be mangled halfway through. (In other words, in this case I need consistency, but not necessarily durability.) How would you do that without fsync? Databases: People generally don't like unrecoverable consistency errors just because their computer crashed during a write. Not generally possible with reordered writes. Sometimes people also need durability on top of consistency, e.g. for everything where you want to make at most one request to some server; you can do that by e.g. writing "I did the thing" to a log file, fsync'ing it, and then making your request.