3 ms·
> Our findings show that although applications use many failure-handling strategies, none are sufficient: fsync failures can cause catastrophic outcomes such as
by iforgotpassword 4y ago
> Our findings show that although applications use many failure-handling strategies, none are sufficient: fsync failures can cause catastrophic outcomes such as data loss and corruption.
That makes it seem like an immediate abort might be the best action in most cases? Handling it wrong and then chugging along might amplify any corruption that has happened.
It might obviously depend on the application and use case, but I'd like to think projects like pgsql put a lot of effort into getting this right after fsyncgate. I've read quite a bit about it after that incident, but ultimately decided I'm too stupid to get that right and roll the "log error and bail out" route ever since.
- formerly_proven 4y agoIf you see EIO 99.999 % of the time you want to do just that - stop anything you're doing, don't attempt any further writes. Chances are they'll fail anyway.
- krkoch 4y agoThe storage just bought the farm, EIEIO.
- masklinn 4y ago> ultimately decided I'm too stupid to get that right and roll the "log error and bail out" route ever since. That's exactly where postgres ended up: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9ccdd7f66e3324d2b6d3dec282cfa9ff084083f1 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit... > PANIC on fsync() failure.
- eis 4y agoThe paper explains that even crashing after fsync fails you can end up with lost or corrupted data because after the next start you might get the wrong data from the page cache which survives the crash as it's not part of the programs memory but in the kernel. I recommend reading the paper or watching the video, it is very interesting.