43 ms·

> File system semantics need a rethink Normally when writing software that needs fopen/fsync, I would just use SQLite instead

josephg 4y ago
There's a host of reasons this isn't good enough: - SQLite itself would be simpler and probably faster if it could run on top of a better OS-level API - It doesn't make sense to build every other program on top of SQLite. Eg, other databases (redis, postgres), anything performance sensitive, etc. The filesystem is the system interface. Not sqlite. Aesthetically, its also pretty gross to make a mess in POSIX, then solve it in userland (via sqlite). Its much better to just have the OS expose an API that is actually nice to work with and avoid all the slowdown, complexity and bugs which can arise from the nasty fsync API. As an example, redis runs a whole extra userland thread dedicated to issuing fsync commands to the operating system. There's a whole signalling system needed to manage that thread. With an API like the GP proposed, none of that code would be needed.
- 1over137 4y ago
  >It doesn't make sense to build every other program on top of SQLite I dunno, it's not a crazy as building everything on top of Electron. :)
  - josephg 4y ago
    True - but that's not exactly a high bar for success. I don't want every program I run to be an electron style hodgepodge mess which takes 1800 dependencies to copy an iso to a USB stick (or whatever).
- mgaunard 4y ago
  Many databases (including SQLite) don't use the filesystem correctly. The fault doesn't lie with the filesystem but with the database.
  - josephg 4y ago
    Can you give some examples of ways databases misuse the filesystem?
    - blibble 4y ago
      there were several data loss inducing bugs in postgres with fsync/close
      - josephg 4y ago
        Right. When fsync returns an error code, it means (I think) that one of the earlier writes has failed or partially failed. This is not an obvious behaviour, and it wasn’t clear from the fsync documentation. And fsync doesn’t tell you which write failed! It’s spooky action at a distance. We can blame Postgres for not handling this case properly, but I think the design of fsync bears just as much blame here. A write completion callback (as suggested upthread) would be a much more sensible and intuitive way to pass write errors back to the database. I think if Linux exposed an api like that, there’s a much lower chance this bug would have happened at all.
        jstimpfle 4y ago
        It was always clear from what fsync does that it doesn't indicate which write failed. That's exactly the point why fsync exists -- write() doesn't write through, for obvious performance reasons. So any error reported from fsync() could apply to any write that happened since the last successful fsync(). In fact, storage errors that happen at the time of asynchronous writeback cannot be traced back to individual write() requests (write() requests can overwrite each other, can be collated, etc) so it is technically impossible for fsync() to report which write it was. But what may be surprising is that fsync() can report errors that were "caused" by other processes writing to the same file -- because, again, storage error in general cannot be traced back to individual write() calls.
        josephg 4y ago
blibble 4y ago
this is like deciding to go and use redis when you need an array internally

josephg 4y ago

There's a host of reasons this isn't good enough: - SQLite itself would be simpler and probably faster if it could run on top of a better OS-level API - It doesn't make sense to build every other program on top of SQLite. Eg, other databases (redis, postgres), anything performance sensitive, etc. The filesystem is the system interface. Not sqlite. Aesthetically, its also pretty gross to make a mess in POSIX, then solve it in userland (via sqlite). Its much better to just have the OS expose an API that is actually nice to work with and avoid all the slowdown, complexity and bugs which can arise from the nasty fsync API. As an example, redis runs a whole extra userland thread dedicated to issuing fsync commands to the operating system. There's a whole signalling system needed to manage that thread. With an API like the GP proposed, none of that code would be needed.

1over137 4y ago
>It doesn't make sense to build every other program on top of SQLite I dunno, it's not a crazy as building everything on top of Electron. :)
- josephg 4y ago
  True - but that's not exactly a high bar for success. I don't want every program I run to be an electron style hodgepodge mess which takes 1800 dependencies to copy an iso to a USB stick (or whatever).
mgaunard 4y ago
Many databases (including SQLite) don't use the filesystem correctly. The fault doesn't lie with the filesystem but with the database.
- josephg 4y ago
  Can you give some examples of ways databases misuse the filesystem?
  - blibble 4y ago
    there were several data loss inducing bugs in postgres with fsync/close
    - josephg 4y ago
      Right. When fsync returns an error code, it means (I think) that one of the earlier writes has failed or partially failed. This is not an obvious behaviour, and it wasn’t clear from the fsync documentation. And fsync doesn’t tell you which write failed! It’s spooky action at a distance. We can blame Postgres for not handling this case properly, but I think the design of fsync bears just as much blame here. A write completion callback (as suggested upthread) would be a much more sensible and intuitive way to pass write errors back to the database. I think if Linux exposed an api like that, there’s a much lower chance this bug would have happened at all.
      - jstimpfle 4y ago
        It was always clear from what fsync does that it doesn't indicate which write failed. That's exactly the point why fsync exists -- write() doesn't write through, for obvious performance reasons. So any error reported from fsync() could apply to any write that happened since the last successful fsync(). In fact, storage errors that happen at the time of asynchronous writeback cannot be traced back to individual write() requests (write() requests can overwrite each other, can be collated, etc) so it is technically impossible for fsync() to report which write it was. But what may be surprising is that fsync() can report errors that were "caused" by other processes writing to the same file -- because, again, storage error in general cannot be traced back to individual write() calls.
        josephg 4y ago

blibble 4y ago

this is like deciding to go and use redis when you need an array internally