4 ms·

Sqlite is much better than raw files to keep data intact on power failures, and be sure to study the options carefully If you truly need to use files you can t

Sqlite is much better than raw files to keep data intact on power failures, and be sure to study the options carefully If you truly need to use files you can take other steps such as mv the old file to .bck before mv the new file, but I really think you want sqlite

account42 2y ago
SQLite doesn't do any magic other than fsync. Using it to deal with power failures is nonsense.
- vlovich123 2y ago
  Well generally OP is correct that SQLite will more likely have written fsync code correctly and be fairly robust against power failures due to the WAL. Additionally, it’s not doing any renaming which you would need with plain files. So while it doesn’t do any other magic, it’s more likely to handle power failures correctly.
- cryptonector 2y ago
  `fsync()` is not necessarily enough if your on-disk format is complex because you still need to write recovery code. Of course, SQLite3's on-disk format is very complex, and so it requires correspondingly more complex power failure recovery code. But SQLite3 has an excellent test suite, and in particular they have an excellent power failure recovery test suite. Using SQLite3 to avoid power failure issues is pretty good advice. I don't see why GP is getting downvoted.
- nine_k 2y ago
  But the authors of SQLite have studied and correctly implemented the working fsync logic, so you don't have to. It ·may* be a correct approach if it's your implementation detail, and.nobody else expects that file you're replacing.
- Too 2y ago
  I think the idea was to store data in the db, instead of scattered over multiple files and directories. Then you only have to worry about one file to fsync (Two if using a wal).
- Dylan16807 2y ago
  > SQLite doesn't do any magic other than fsync. True. > Using it to deal with power failures is nonsense. Using a very well designed library for your use case is not nonsense.
  - jefecoon 2y ago
    > Using a very well designed library for your use case is not nonsense. Could you recommend your personal favorite(s) of such libraries? Enquiring minds want to know! Thx.
    - Dylan16807 2y ago
      When it comes to libraries that handle the messy details of putting data on disk, sqlite is the only one I can name off hand so that's the favorite.

account42 2y ago

SQLite doesn't do any magic other than fsync. Using it to deal with power failures is nonsense.

vlovich123 2y ago
Well generally OP is correct that SQLite will more likely have written fsync code correctly and be fairly robust against power failures due to the WAL. Additionally, it’s not doing any renaming which you would need with plain files. So while it doesn’t do any other magic, it’s more likely to handle power failures correctly.
cryptonector 2y ago
`fsync()` is not necessarily enough if your on-disk format is complex because you still need to write recovery code. Of course, SQLite3's on-disk format is very complex, and so it requires correspondingly more complex power failure recovery code. But SQLite3 has an excellent test suite, and in particular they have an excellent power failure recovery test suite. Using SQLite3 to avoid power failure issues is pretty good advice. I don't see why GP is getting downvoted.
nine_k 2y ago
But the authors of SQLite have studied and correctly implemented the working fsync logic, so you don't have to. It ·may* be a correct approach if it's your implementation detail, and.nobody else expects that file you're replacing.
Too 2y ago
I think the idea was to store data in the db, instead of scattered over multiple files and directories. Then you only have to worry about one file to fsync (Two if using a wal).
Dylan16807 2y ago
> SQLite doesn't do any magic other than fsync. True. > Using it to deal with power failures is nonsense. Using a very well designed library for your use case is not nonsense.
- jefecoon 2y ago
  > Using a very well designed library for your use case is not nonsense. Could you recommend your personal favorite(s) of such libraries? Enquiring minds want to know! Thx.
  - Dylan16807 2y ago
    When it comes to libraries that handle the messy details of putting data on disk, sqlite is the only one I can name off hand so that's the favorite.