3 ms·

Very nice! I really like the design choices here. Though I (edit: am biased and) would personally have used Rust and https://github.com/xetdata/nfsserve https:/

Very nice! I really like the design choices here. Though I (edit: am biased and) would personally have used Rust and https://github.com/xetdata/nfsserve https://github.com/xetdata/nfsserve.

eru 3y ago
I've built something like this as well, but in FUSE for Linux, not via NFS. It really brings home how close git's design already is to being a file system.
- bmacho 3y ago
  A virtual file system? How usable are those? I want something that can store slightly different data (modifies data on the fly), and one, that I can use as a tagged file system (generates folder structure on the fly). Is that viable with FUSE/NFS? Do programs play nice with them (no caching, no recursive lookahead and such)? Is it hard to write one? Which language did you used?
  - eru 3y ago
    I wrote version in Python and Rust. (And ended up contributing to libfuse and its Rust equivalent along the way.) > Do programs play nice with them (no caching, no recursive lookahead and such)? Other programs play nice with my peculiar file systems as far as I can tell: they just look like ordinary files and directories and symlinks etc to them. I don't know how well that would work for your usecase. Most of the caching etc that I am aware of happens at the layer of the Linux kernel, and your FUSE daemon can configure what caching it wants or doesn't want. I don't know much about NFS. I only used FUSE directly. For me, presenting git as a filesystem was relatively easy to write. I think that comes down to two main factors: * Git's internal structure is very close to how Linux (and especially fuse) do filesystems. That's probably not a big surprise, given that git and the Linux kernel are both projects started by the same guy. * I exposed the git data as a read-only filesystem. In general, if you can avoid mutation, you can avoid a lot of complexity. (The main source of complexity in the Python version I wrote first was that we allowed wanted to reflect changes in the contents of git branches to be reflected in the filesystem side. That's mutability flowing in only one direction (from git to the filesystem), but it was already annoying enough to deal with.) On a more general note: I see filesystems mostly as one peculiar interface you can put in front of your data. So one some level git has an underlying database, and we just expose it. The same could be done for eg postgres or mongodb with a suitable fuse daemon. But you can also think of ext4 being such a database with a weird UI layer on top. The main difference in practice is that they run their 'fuse daemon' in the kernel. From what I've read the new bcachefs explicitly embraces this view of 'filesystems are just databases', and uses techniques borrowed from relational databases for its internal datastructures.
rapnie 3y ago
Functional but not complete. The project is looking for maintainers. Overall it is a pity how so many potentially great Rust projects are inactive and left in incomplete state.
- reverius42 3y ago
  It's not complete in its current state with respect to the NFS protocol, but is very usable as a base for things that work (for instance, https://github.com/xetdata/xet-core https://github.com/xetdata/xet-core uses it to mount git(-xet) repos as directories, similarly to the linked post but with some different design choices). It is maintained for this use case and contributors are welcome for other use cases that aren't covered yet.

Very nice! I really like the design choices here. Though I (edit: am biased and) would personally have used Rust and https://github.com/xetdata/nfsserve https://github.com/xetdata/nfsserve.

eru 3y ago

I've built something like this as well, but in FUSE for Linux, not via NFS. It really brings home how close git's design already is to being a file system.

bmacho 3y ago
A virtual file system? How usable are those? I want something that can store slightly different data (modifies data on the fly), and one, that I can use as a tagged file system (generates folder structure on the fly). Is that viable with FUSE/NFS? Do programs play nice with them (no caching, no recursive lookahead and such)? Is it hard to write one? Which language did you used?
- eru 3y ago
  I wrote version in Python and Rust. (And ended up contributing to libfuse and its Rust equivalent along the way.) > Do programs play nice with them (no caching, no recursive lookahead and such)? Other programs play nice with my peculiar file systems as far as I can tell: they just look like ordinary files and directories and symlinks etc to them. I don't know how well that would work for your usecase. Most of the caching etc that I am aware of happens at the layer of the Linux kernel, and your FUSE daemon can configure what caching it wants or doesn't want. I don't know much about NFS. I only used FUSE directly. For me, presenting git as a filesystem was relatively easy to write. I think that comes down to two main factors: * Git's internal structure is very close to how Linux (and especially fuse) do filesystems. That's probably not a big surprise, given that git and the Linux kernel are both projects started by the same guy. * I exposed the git data as a read-only filesystem. In general, if you can avoid mutation, you can avoid a lot of complexity. (The main source of complexity in the Python version I wrote first was that we allowed wanted to reflect changes in the contents of git branches to be reflected in the filesystem side. That's mutability flowing in only one direction (from git to the filesystem), but it was already annoying enough to deal with.) On a more general note: I see filesystems mostly as one peculiar interface you can put in front of your data. So one some level git has an underlying database, and we just expose it. The same could be done for eg postgres or mongodb with a suitable fuse daemon. But you can also think of ext4 being such a database with a weird UI layer on top. The main difference in practice is that they run their 'fuse daemon' in the kernel. From what I've read the new bcachefs explicitly embraces this view of 'filesystems are just databases', and uses techniques borrowed from relational databases for its internal datastructures.

rapnie 3y ago

Functional but not complete. The project is looking for maintainers. Overall it is a pity how so many potentially great Rust projects are inactive and left in incomplete state.

reverius42 3y ago
It's not complete in its current state with respect to the NFS protocol, but is very usable as a base for things that work (for instance, https://github.com/xetdata/xet-core https://github.com/xetdata/xet-core uses it to mount git(-xet) repos as directories, similarly to the linked post but with some different design choices). It is maintained for this use case and contributors are welcome for other use cases that aren't covered yet.