3 ms·
Imagine a juggler, juggling 5 balls at a time. At a set period of time, he will drop one ball and accept another ball thrown at him. He handles this very well b
by pknopf 7y ago
Imagine a juggler, juggling 5 balls at a time. At a set period of time, he will drop one ball and accept another ball thrown at him. He handles this very well because there is order/cadence.
Now imagine asking him to support being thrown balls randomly, at any interval. He may make it work, but I'd imagine he will stutter a bit.
In my experience, anytime you interrupt the page cache's normal routine, it stutters everything. I've seen the "sync" command freeze my Ubuntu machine (music player, GUI, etc).
I work on embedded devices, and my employer wanted to reduce the window at which data-loss would happen for 30mb+ files (video capture). It wasn't a supported use case, "But why not! It makes our product theoretically better!" I put my foot down. We aren't touching page-cache until there is a clear benefit to the user. It almost got me fired, but good riddance if so.
- mjw1007 7y agoThe annoying thing is that the Postgres people don't even really need "sync this file right now" for the main data files. AIUI what they'd ideally like is "sync this file reasonably soon, and let me know when it's finished".
- the8472 7y agoio_uring supports issuing async sync_file_range requests, which can be used to do most of the heavy lifting. You will still need a final fsync to also get the metadata written, but that should faster at that point.
- cyphar 7y agoIt should be noted that io_uring landed (in 5.1) a fair while after the whole Postgres back-and-forth started in 2018.
- masklinn 7y agoIt should also be noted that even sync_file_range's manpage warns you against using it, which tells you how reliable that thing is.
- the8472 7y agoIt's not really that you shouldn't use it because it is dangerous in itself. It's just doesn't do what one would naively expect it to do. It's still useful as "flush dirty pages" hinter while preparing the real fsync.
- the8472 7y agocorrection: I took a closer look at liburing and it also supports straight fsync[0], not just sync_file_range, this makes it even simpler. [0] http://git.kernel.dk/cgit/liburing/tree/src/io_uring.h?id=90619de90d35167b1adef0b1e000ffe307eaea03#n56 http://git.kernel.dk/cgit/liburing/tree/src/io_uring.h?id=90...
- the8472 7y agofsync != syncfs/sync The former is for a single file, the others are for one/all filesystems
- pknopf 7y agoI'm aware, but the analogy still applies, which was about page-cache in particular.