4 ms·

The filesystem, and the file itself, are perhaps the best abstractions ever designed in computing. A database sometimes uses the file API when actually wants to

The filesystem, and the file itself, are perhaps the best abstractions ever designed in computing. A database sometimes uses the file API when actually wants to access the disk in a lower-level fashion, for which APIs are fragmented, non stable, non portable. So it's not a fault of the abstraction, but the fact that a different low-level abstraction to access the storage in a way databases need to do is missing from POSIX.

valenterry 6y ago
Not really... They are one of the _first_ abstractions, and they are quite fundamental. That's why they all still exist in the way they are. Just look at the name: "file". And "directory". Where do these terms come from? Right, from organizing things, mainly offices. And offices are organized like that because this is what is practical under the constraints of humans working with paper in a three dimensional space. But looking at it semantically, almost no one really wants to only use files or directories. We mostly want to use semantical tags. And one kind of tag can be a hierarchical "directory/file" tag - fair enough. But that's only a small part of the picture, because we also want to tag our objects/entities by date, creator, department, security clearance level, how long they have to be archived and so on. Using just a filesystem does not provide this, so we have to build on top of that. And you can see that this is changing. Look at phones nowadays. They don't use this anymore. Pictures are stored in collections/groups, can be tagged and sorted in various ways. They can be seen on a geographical map. And much more. No one has a directory/file structure in their head. So the question is: does the OS still need to support that? Actually, maybe not. Maybe SSDs today can work much more efficient if we don't assume everything must be a file. Just wait, we will get there.
- Spivak 6y ago
  I think Ceph has the right model. The only abstraction is physical disk to addressable binary/object storage. Everything else is implemented on top of that so you can do S3, filesystems, databases all close to the metal.
- pdimitar 6y ago
  Completely agreed. Every serious project I've been in has been striving to invent storage that's as close to their problem as possible. And I do the same in my hobby projects. The file/directory abstraction survived for a long time but nowadays it's often a limitation more than a convenience.

valenterry 6y ago

Not really... They are one of the _first_ abstractions, and they are quite fundamental. That's why they all still exist in the way they are. Just look at the name: "file". And "directory". Where do these terms come from? Right, from organizing things, mainly offices. And offices are organized like that because this is what is practical under the constraints of humans working with paper in a three dimensional space. But looking at it semantically, almost no one really wants to only use files or directories. We mostly want to use semantical tags. And one kind of tag can be a hierarchical "directory/file" tag - fair enough. But that's only a small part of the picture, because we also want to tag our objects/entities by date, creator, department, security clearance level, how long they have to be archived and so on. Using just a filesystem does not provide this, so we have to build on top of that. And you can see that this is changing. Look at phones nowadays. They don't use this anymore. Pictures are stored in collections/groups, can be tagged and sorted in various ways. They can be seen on a geographical map. And much more. No one has a directory/file structure in their head. So the question is: does the OS still need to support that? Actually, maybe not. Maybe SSDs today can work much more efficient if we don't assume everything must be a file. Just wait, we will get there.

Spivak 6y ago
I think Ceph has the right model. The only abstraction is physical disk to addressable binary/object storage. Everything else is implemented on top of that so you can do S3, filesystems, databases all close to the metal.
pdimitar 6y ago
Completely agreed. Every serious project I've been in has been striving to invent storage that's as close to their problem as possible. And I do the same in my hobby projects. The file/directory abstraction survived for a long time but nowadays it's often a limitation more than a convenience.