4 ms·

If you then expose that file system through a POSIX file system API, you have all of the same issues of underspecified or unclear behaviour that the article men

by jclulow 11y ago

If you then expose that file system through a POSIX file system API, you have all of the same issues of underspecified or unclear behaviour that the article mentions.

derefr 11y ago
If I were designing a userland from the ground up, I'd probably give processes a transactional MVCC object store, and make guarantees about that; and then implement a "POSIX compatibility layer" file system API in terms of that, but explicitly say that none of the same guarantees from the object-store layer apply. Some days I really do wish we weren't so inured to the particular 50-year-old systems-programming abstractions.
- hendzen 11y ago
  Microsoft actually tried something similar with WinFS [0] in the 00's, but the project failed. [0] - https://en.wikipedia.org/wiki/WinFS https://en.wikipedia.org/wiki/WinFS
  - derefr 11y ago
    Oddly, Apple's Core Data framework achieves pretty much all the goals WinFS set out to achieve—but does so sitting (for no good reason, really) on top of a regular filesystem. It'd actually be pretty easy to flip things around: give Core Data a raw block device persistence backend, and then write a filesystem driver on top of it. One of my recently-planned hobby projects was to write a FUSE filesystem for OSX that would take all the shell-level glop (LaunchServices UTI-associated verbs, NSFileManager attributes, Spotlight metadata, etc.) and stick it onto the files themselves as plain xattrs, so POSIX tools could effectively be made to work with shell-namespace "Documents" (including treating directory-packages as files, and traversing archive files and container-file-formats as if they were directory-packages.) It'd be quite interesting to combine the two experiments, actually; it'd result in something very much like a POSIX compatibility layer on top of a WinFS-like filesystem.
    - astrange 11y ago
      > but does so sitting (for no good reason, really) on top of a regular filesystem You don't like files? Files are great. Running Core Data over SQLite instead of a "real database" gets you backups for free, the iOS file-based security model, lets you delete an app's data when it goes away, network mounted home directories… really, the filesystem abstraction is pretty good still.
  - NotSammyHagar 11y ago
    I was a developer on WinFS. It turns out to be hard to have a performant system that also has transactional commits between something changing a regular file, and a change in a database. It was also stupid to write OS components in .net, when it would just crash if it ran out of memory, among many other problems. Many people working on it wrote reports about how this was infeasible or impractical, yet they just said keep going, it will be all right. I think we could have built something that worked well enough, but they grew cowardly after the general failure of the entire longhorn project. Don't try to change the compiler, the language, the os, the office application to use same "in motion" components, plus the database. It's probably little known outside of Microsoft, but they tried to build many different higher end object filesystems, including JAWS (Jim Alchin Windows Filesystem), and others I can't even remember.
- dap 11y ago
  How would it be different than the filesystem API? There was a great lightning talk a few years ago that I can't seem to find where the author described an API for storing blobs in a hierarchical namespace. Of course, halfway through, it became clear that it's just the POSIX API: you can "open" handles to objects, "rename" them, remove them, and so on. You'd end a transaction with "fsync()". (Okay, that one's a little more complicated, but I don't think it's as hard as the OP claims, at least for single files. Multiple files are more complicated, but that problem is intrinsically more complicated.)
  - derefr 11y ago
    Two key differences: 1. "MVCC": the ability to effectively get a handle on a static copy of the entire filesystem, perform mutations to many objects, and then submit the transaction, at which point it will either commit or rollback depending on whether any of those objects have been modified. Windows actually has this: https://en.wikipedia.org/wiki/Transactional_NTFS https://en.wikipedia.org/wiki/Transactional_NTFS allows for exactly the sort of MVCC I'm talking about—and presents a very different API than the POSIX filesystem one. 2. "Object store": as in, you don't interact with the filesystem by getting writable handles to the file's underlying backing store, where processes can effectively treat a file as persistent shared memory. Instead, you ask for a floating unnamed "buffer" object, fill it up, and then submit it to the filesystem to be atomically stored; or, vice-versa, you ask to retrieve a file, and are passed a buffer handle of the representation of the file at the point in time you asked for it (presumably, for optimization's sake, backed by copy-on-write mmap'ed pages from the latest MVCC-linearized copy of the file.) We actually have this today as well, but in an implicit and half-assed way. Files below some size threshold, in modern filesystems, don't use any FS extents for backing, but are instead stored as part of their filesystem directory-entry node. This basically makes the filesystem into an object store for those files—but without actually exposing any guarantee to userland that those files will be read/written as atomic operations. --- To be clearer, there's a problem with this fs-atop-object-store design, as I've stated it so far: An object store doesn't support every use-case a filesystem does, and a filesystem implemented only in terms of an object-store wouldn't be good for some things because of that. It would be terribly expensive to emulate a block device on top of an object store, for example—you'd have to make each emulated block an object, and mutate blocks by transactionally adding an updated block and removing the old block. This means that it wouldn't make sense to keep mountable read-write disk images on such a filesystem; nor would it make sense to keep database backing stores there. But both of those things are effectively things that take no advantage of existing on top of a filesystem in the first place—they do their own index-building, their own sparse-allocation, their own journaling, etc. Making the filesystem an object store just makes it obvious that for those use-cases, the filesystem itself is pure overhead. So, along with the MVCC object-store, the other part of my hypothetical system is a "buffer store": basically like a logical-volume manager, an API that consumes physical block devices and exposes handles to (cheap) logical resizable block devices. The object store could be implemented in terms of one of those logical block devices, but otherwise would completely ignore the existence of it. The filesystem compatibility layer, on the other hand, would allow you to request that a given filesystem object be backed by a persistent buffer (newly-created logical block device) instead of an object; and then whenever you requested that object from the filesystem, you'd get an IO handle to the logical block device, instead of an IO handle to a transactionally-resubmittable copy-on-write mmap of some object.
- protomyth 11y ago
  You might look at probably the most popular system to ship with an object store, the Apple Newton. I would probably start there and work my up API wise.

by jclulow 11y ago

If you then expose that file system through a POSIX file system API, you have all of the same issues of underspecified or unclear behaviour that the article mentions.

derefr 11y ago

If I were designing a userland from the ground up, I'd probably give processes a transactional MVCC object store, and make guarantees about that; and then implement a "POSIX compatibility layer" file system API in terms of that, but explicitly say that none of the same guarantees from the object-store layer apply. Some days I really do wish we weren't so inured to the particular 50-year-old systems-programming abstractions.

hendzen 11y ago
Microsoft actually tried something similar with WinFS [0] in the 00's, but the project failed. [0] - https://en.wikipedia.org/wiki/WinFS https://en.wikipedia.org/wiki/WinFS
- derefr 11y ago
  Oddly, Apple's Core Data framework achieves pretty much all the goals WinFS set out to achieve—but does so sitting (for no good reason, really) on top of a regular filesystem. It'd actually be pretty easy to flip things around: give Core Data a raw block device persistence backend, and then write a filesystem driver on top of it. One of my recently-planned hobby projects was to write a FUSE filesystem for OSX that would take all the shell-level glop (LaunchServices UTI-associated verbs, NSFileManager attributes, Spotlight metadata, etc.) and stick it onto the files themselves as plain xattrs, so POSIX tools could effectively be made to work with shell-namespace "Documents" (including treating directory-packages as files, and traversing archive files and container-file-formats as if they were directory-packages.) It'd be quite interesting to combine the two experiments, actually; it'd result in something very much like a POSIX compatibility layer on top of a WinFS-like filesystem.
  - astrange 11y ago
    > but does so sitting (for no good reason, really) on top of a regular filesystem You don't like files? Files are great. Running Core Data over SQLite instead of a "real database" gets you backups for free, the iOS file-based security model, lets you delete an app's data when it goes away, network mounted home directories… really, the filesystem abstraction is pretty good still.
- NotSammyHagar 11y ago
  I was a developer on WinFS. It turns out to be hard to have a performant system that also has transactional commits between something changing a regular file, and a change in a database. It was also stupid to write OS components in .net, when it would just crash if it ran out of memory, among many other problems. Many people working on it wrote reports about how this was infeasible or impractical, yet they just said keep going, it will be all right. I think we could have built something that worked well enough, but they grew cowardly after the general failure of the entire longhorn project. Don't try to change the compiler, the language, the os, the office application to use same "in motion" components, plus the database. It's probably little known outside of Microsoft, but they tried to build many different higher end object filesystems, including JAWS (Jim Alchin Windows Filesystem), and others I can't even remember.
dap 11y ago
How would it be different than the filesystem API? There was a great lightning talk a few years ago that I can't seem to find where the author described an API for storing blobs in a hierarchical namespace. Of course, halfway through, it became clear that it's just the POSIX API: you can "open" handles to objects, "rename" them, remove them, and so on. You'd end a transaction with "fsync()". (Okay, that one's a little more complicated, but I don't think it's as hard as the OP claims, at least for single files. Multiple files are more complicated, but that problem is intrinsically more complicated.)
- derefr 11y ago
  Two key differences: 1. "MVCC": the ability to effectively get a handle on a static copy of the entire filesystem, perform mutations to many objects, and then submit the transaction, at which point it will either commit or rollback depending on whether any of those objects have been modified. Windows actually has this: https://en.wikipedia.org/wiki/Transactional_NTFS https://en.wikipedia.org/wiki/Transactional_NTFS allows for exactly the sort of MVCC I'm talking about—and presents a very different API than the POSIX filesystem one. 2. "Object store": as in, you don't interact with the filesystem by getting writable handles to the file's underlying backing store, where processes can effectively treat a file as persistent shared memory. Instead, you ask for a floating unnamed "buffer" object, fill it up, and then submit it to the filesystem to be atomically stored; or, vice-versa, you ask to retrieve a file, and are passed a buffer handle of the representation of the file at the point in time you asked for it (presumably, for optimization's sake, backed by copy-on-write mmap'ed pages from the latest MVCC-linearized copy of the file.) We actually have this today as well, but in an implicit and half-assed way. Files below some size threshold, in modern filesystems, don't use any FS extents for backing, but are instead stored as part of their filesystem directory-entry node. This basically makes the filesystem into an object store for those files—but without actually exposing any guarantee to userland that those files will be read/written as atomic operations. --- To be clearer, there's a problem with this fs-atop-object-store design, as I've stated it so far: An object store doesn't support every use-case a filesystem does, and a filesystem implemented only in terms of an object-store wouldn't be good for some things because of that. It would be terribly expensive to emulate a block device on top of an object store, for example—you'd have to make each emulated block an object, and mutate blocks by transactionally adding an updated block and removing the old block. This means that it wouldn't make sense to keep mountable read-write disk images on such a filesystem; nor would it make sense to keep database backing stores there. But both of those things are effectively things that take no advantage of existing on top of a filesystem in the first place—they do their own index-building, their own sparse-allocation, their own journaling, etc. Making the filesystem an object store just makes it obvious that for those use-cases, the filesystem itself is pure overhead. So, along with the MVCC object-store, the other part of my hypothetical system is a "buffer store": basically like a logical-volume manager, an API that consumes physical block devices and exposes handles to (cheap) logical resizable block devices. The object store could be implemented in terms of one of those logical block devices, but otherwise would completely ignore the existence of it. The filesystem compatibility layer, on the other hand, would allow you to request that a given filesystem object be backed by a persistent buffer (newly-created logical block device) instead of an object; and then whenever you requested that object from the filesystem, you'd get an IO handle to the logical block device, instead of an IO handle to a transactionally-resubmittable copy-on-write mmap of some object.
protomyth 11y ago
You might look at probably the most popular system to ship with an object store, the Apple Newton. I would probably start there and work my up API wise.