5 ms·
The secret life of fsync
- pyrolistical 4y agoFeels like this doesn’t actually answer the question? The hardware can still lie and say it was flushed so it benchmarks better. Would need to do real world test and yank the power after fsync. This https://news.ycombinator.com/item?id=10552218 https://news.ycombinator.com/item?id=10552218 article did just that. But unfortunately we would need this tested against all drive models/manufacturers
- emmelaich 4y agoThat's all covered in the last sentence. It addresses the main question well; is there a way to instruct the disk to flush its cache and is it used by fsync? Yes and yes.
- hardwaresofton 4y agoActually ran into this while doing some experiments with ZFS & Postgres, some non-enterprise drives will pretend they wrote data when they didn’t and you can corrupt a even a rock-steady database like Postgres. If you’re not afraid to lose some perf, you can disable volatile write cache on your non enterprise drive: https://vadosware.io/post/everything-ive-seen-on-optimizing-postgres-on-zfs-on-linux/#solution-force-proper-disk-syncing https://vadosware.io/post/everything-ive-seen-on-optimizing-...
- layer8 4y agoThe link currently gives me a 404. Archive link: https://archive.is/xVKhx https://archive.is/xVKhx
- hardwaresofton 4y agoThe site looks up to me -- did you get a 404 at the browser level? Cert issue? [EDIT] Just checked Qualys and I've an A+ rating (have had TLS cert issues in the past though)... Uptime tools not reporting anything either.
- layer8 4y agoHmm, it’s only on one device. It gives a white page with only the text “404 page not found” in fixed-width font. It works when prepending “www.” to the domain.
- hardwaresofton 4y agoSuuuper weird... This might be an issue with old certs being stuck... Found an old IPv6 record, so maybe that device was preferring IPv6, and hit that record.
- layer8 4y agoYes, I actually got a certificate warning at first.
- Avamander 4y agoI've actually had the opposite, I want ZFS to ignore the massive amount of fsync calls things like chromium or regular desktop applications make (without becoming generally unsafe). I've resorted to preloading libeatmydata instead. The disks are PLP and the machine is behind an UPS, things shouldn't/don't have to hit the disk all the time.
- hardwaresofton 4y agooh well if you want ZFS to ignore the fsync calls... you could disable sync and rely on txg_timeout (default 5s) to write your state to disk. Not recommended, but this is one of the things I tried (I tuned it to 1s to be slightly safer): https://vadosware.io/post/everything-ive-seen-on-optimizing-postgres-on-zfs-on-linux/#update-20211221-results-from-zfs_txg_timeout1syncdisabledlogbiaslatency https://vadosware.io/post/everything-ive-seen-on-optimizing-... But you probably want to be very sure that your desktop applications are somewhat safe with their data/can recover from crashes... TIL of libeatmydata[0], if you're comfortable using that, you'd probably be comfortable disabling fsync on the datasets for those apps. [0]: https://github.com/stewartsmith/libeatmydata https://github.com/stewartsmith/libeatmydata
- hyc_symas 4y agoYeah, I explicitly LD_PRELOAD libeatmydata for a few things, like my web browser and before running a software upgrade. So much junk being overwritten multiple times, manpages regenerated, init ramdisks regenerated, over and over again in a single update pass.
- Animats 4y agoFile system semantics need a rethink. Here's what I've proposed. This is not original with me; these are all things implemented in some non-UNIX-based systems. Files should come in several kinds: - Unit files. They're immutable, but replaceable. This is the normal case. You create them with "creat", write them, and when closed, they replace any previous version with the same name. Any program which opens one for reading sees a complete file. On a crash before close, the new file is dropped and the old version remains. If a program aborts, that also happens. An explicit close commits the file update. - Log files. These are append only. On a crash, you are guaranteed that the file will be intact to some point ending at a write boundary. The last transactions may be lost, but the file must not tail off into junk. For journals, log files, etc. - Temp files. These disappear on a crash. - Managed files. These are for databases. They can do everything files can do. Along with this, you need async I/O. This is primarily for log files and managed files. When you do a write, you get two callbacks, one when the buffer has been copied, and one when the data has been committed to permanent storage and will survive a crash. It is the job of the OS, drivers, and devices to not send the data-committed completion until the data is safe. Only a few programs need that, but they're the important ones - database managers. This turns the fsync problem around. Instead of doing an fsync to force write-out, databases wait for a completion before considering the data committed. It's not a flush-everything; it's a commit-complete for a single write. So if you have multiple I/O operations pending on a single file, a normal event in database land, there's no big stall for a cache flush. There's also no incentive for drive manufacturers to send commit-completes early. The only people who use them are the ones who will notice if someone cheats. Most bulk I/O is on unit files, where something is being copied. (There are mainframe systems that had this sort of thing half a century ago.)
- raggi 4y agoStopping at the block later sadly doesn’t answer much these days. Is it hitting a software ftl? Firmware? Is there caching? Is the ftl journaled/logged/whatever? Ideally we need a barrier command that goes all the way down. What I’d love to see is a top to bottom barrier available for user space but flushing be a privileged operation that requires explicit grants (you’d give this to db’s maybe, but not to browsers, for example)
- PaulBorrill 4y agoFsync doesn’t work. Never has. Good discussion yesterday on ItsAboutTime.club with Pat Helland and Prof. Edward Lee. https://www.clubhouse.com/room/P0Bkl3y9 https://www.clubhouse.com/room/P0Bkl3y9
- devnull3 4y agoThe block device may not be physical one. Instead it could be software like NBD. So there is really no way of knowing unless you know what exactly you are writing into, which actually breaks the abstraction.
- vrglvrglvrgl 4y ago[dead]