4 ms·
The UNIX filesystem should not be used for applications, in my opinion. And for the underlying technologies, like databases, object stores, etc, it's the only o
by jeffdavis 11y ago
The UNIX filesystem should not be used for applications, in my opinion. And for the underlying technologies, like databases, object stores, etc, it's the only option (but not a great one).
Even for something simple like photos, it's not good:
* durability problems
* hierarchy doesn't work out well (browsing by dates vs people vs event types can't all work in the same hierarchy)
* you want storage on and accessibility from more than one machine
* something like content-addressed storage probably makes sense to avoid unnecessary duplication
Then, for underlying tech like databases it's not great either. Postgres used to initialize the data directory without syncing it. That makes sense for production, because who is going to put important data in it and crash within a few minutes? But it caused problems with testing, so I fixed it. To do such a trivial thing: write out a bunch of files and sync the whole directory, it was very non trivial. Syncing the files individually was very slow (because I have no clue what the most convenient order is for the OS to sync the files). I had to use the linux-specific sync_file_range() to get decent performance, and then it was still slow so we needed a way to optionally turn it off (e.g. for tests where it's not important). Horrible. It would have been much better to just say: do all of these things and sync them in the most convenient order, then tell me when you are finished. Kind of like a log based system.