4 ms·

The paper analyzes how file systems and PostgreSQL, LMDB, LevelDB, SQLite, and Redis react to fsync failures. It shows that although applications use many failu

The paper analyzes how file systems and PostgreSQL, LMDB, LevelDB, SQLite, and Redis react to fsync failures. It shows that although applications use many failure-handling strategies, none are sufficient: fsync failures can cause catastrophic outcomes such as data loss and corruption.

XorNot 4y ago
This sounds a lot like we need to come up with the correct API for this and switch to it.
- GordonS 4y ago
  I wonder if such an API would require hardware support in order to remain performant?
  - eis 4y ago
    I'd argue some modern hardware e.g. NVME SSDs already expose a hardware interface that would let a new transactional storage API be faster than what we have right now. Unfortunately it is a rare example in a see full of specs that do their best to avoid providing clear semantics and guarantees like atomic sector writes. I guess the next study would need to look into what manufacturers actually fully adhere to the spec. They've not exactly shown best behaviour in that regard in the past (lying about fsync etc) :(
    - xyzzy_plugh 4y ago
      Without the hardware being open, how do you prove this? In a past life I concluded and argued that there's no point in assuming hardware tells the truth. You have to expect the worst and have a plan for handling it, even if that plan is giving up and failing. Importantly this means you can't claim stronger semantics around e.g. atomicity. You can certainly work around most if not all issues, using redundancy, verification and distribution. But in isolation you cannot even properly observe extreme failure scenarios, you can only reduce their probability, and even that is limited.
      - eis 4y ago
        Fully agreed. You can never prevent all issues because there's always a chance for a fire, bitrot due to cosmic rays etc and thus you can only reduce the probability as you correctly concluded. Unfortunately the world we are in right now just makes the hardware issue even worse with APIs that are prone to introducing bugs in the programs. Bad hardware should result in the software being extra carefully crafted to balance it out a bit but somehow we ended up with bad hardware and bad software :(
hyc_symas 4y ago
The paper's analysis of LMDB is wrong. https://news.ycombinator.com/item?id=32462537 https://news.ycombinator.com/item?id=32462537

XorNot 4y ago

This sounds a lot like we need to come up with the correct API for this and switch to it.

GordonS 4y ago
I wonder if such an API would require hardware support in order to remain performant?
- eis 4y ago
  I'd argue some modern hardware e.g. NVME SSDs already expose a hardware interface that would let a new transactional storage API be faster than what we have right now. Unfortunately it is a rare example in a see full of specs that do their best to avoid providing clear semantics and guarantees like atomic sector writes. I guess the next study would need to look into what manufacturers actually fully adhere to the spec. They've not exactly shown best behaviour in that regard in the past (lying about fsync etc) :(
  - xyzzy_plugh 4y ago
    Without the hardware being open, how do you prove this? In a past life I concluded and argued that there's no point in assuming hardware tells the truth. You have to expect the worst and have a plan for handling it, even if that plan is giving up and failing. Importantly this means you can't claim stronger semantics around e.g. atomicity. You can certainly work around most if not all issues, using redundancy, verification and distribution. But in isolation you cannot even properly observe extreme failure scenarios, you can only reduce their probability, and even that is limited.
    - eis 4y ago
      Fully agreed. You can never prevent all issues because there's always a chance for a fire, bitrot due to cosmic rays etc and thus you can only reduce the probability as you correctly concluded. Unfortunately the world we are in right now just makes the hardware issue even worse with APIs that are prone to introducing bugs in the programs. Bad hardware should result in the software being extra carefully crafted to balance it out a bit but somehow we ended up with bad hardware and bad software :(

hyc_symas 4y ago

The paper's analysis of LMDB is wrong. https://news.ycombinator.com/item?id=32462537 https://news.ycombinator.com/item?id=32462537