12 ms·

Userspace FUSE for macOS

301 points by freebeer2 4y ago

vaxman 4y ago
This is a Bad idea. Do you expect Apple has any consideration that things like this are happening inside the usermode processes when its System suspends and compresses or decompresses their memory of or throttles them by flinging them back and forth to different classes of CPU cores or analyzes them to look for other efficiencies that they don’t talk about publicly (network, thermal, sandbox, etc). If Apple wants FUSE, it will build a carve-out (in the form of a framework) like it did for virtualization. If you still want to use FUSE, maybe try System76’s Pop!_OS or find The [Apple] Way to accomplish your client’s goal. https://youtu.be/1LHuEvw56F8 https://youtu.be/1LHuEvw56F8
freebeer2 4y ago
A new project aims to deliver libfuse for mac without using kernel extensions.
- keepquestioning 4y ago
  Could this be done using DriverKit
  - mdaniel 4y ago
    linky: https://developer.apple.com/documentation/driverkit https://developer.apple.com/documentation/driverkit > The drivers you build with DriverKit run in user space, rather than as kernel extensions, which improves system stability and security. You create your driver as an app extension and deliver it inside your existing app. makes it seem like every consumer who wants to use something FUSE-y would have to ship their own impl, right? and I shudder to think of the heartache required with signing or whatever Apple gatekeeping is going on nowadays
  - concerned_ctzn 4y ago
    The author here. DriverKit is really a no-go. A while back I was asked to do a project based on DriverKit, it took me nowhere because of countless bugs and semi-implemented features, worse yet, it caused system crashes (and it was supposed to be stable).
    - keepquestioning 4y ago
      You should send feedback
      - DannyBee 4y ago
        Maybe? Maybe they also shouldn't ship garbage that doesn't work and expect others to debug it? Maybe this should also happen before you deprecate and make it essentially impossible to install kext's? (It's possible, but trying to walk users through it is a great way to lose 99% of your user base instantly) (My experience with driverkit is identical to the parent's) The whole "we removed it for security reasons" is also a hilarious facade. The linux kernel has a much more expansive (heck, crazy!) driver interface and number of drivers. Yet, the rate of system compromise due to them vs applications/servers is probably 99 to 1 in favor of applications/servers.
        saagarjha 4y ago
        That’s because applications have a bunch of easy logic bugs to exploit rather than weird memory corruption you have to wrangle with in a driver, not because the drivers are any more secure.
concerned_ctzn 4y ago
https://github.com/macos-fuse-t/fuse-t https://github.com/macos-fuse-t/fuse-t
- mdaniel 4y ago
  That "repo" appears to exist solely for housing the release artifacts, for those who came to the comments looking for "the goods" Also, https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt appears to be "playing lawyer"
  - TimTheTinker 4y ago
    > Also, https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt appears to be "playing lawyer" Do you have any specific concerns with the text of the license? The text seems clear enough that there don't appear to be any potential liabilities from using the software in a personal capacity. As I understand it, the worst that could happen is that the license/software author could unknowingly incur liabilities from statutory or implied warranties - i.e. they left out some important exclusions that could be brought to bear if a plaintiff successfully construed the license as a contract. (IANAL, but I've read a lot about software licenses over the years and have dealt with a legal challenge related to dual commercial/OSS licensing)
    - rgovostes 4y ago
      For one thing, they copied the clauses from the BSD license, including references to “the following disclaimer”, but omitted the disclaimer itself.
    - mdaniel 4y ago
      Aside from the great sibling observation, I'll pile onto that by pointing out they seem to have truncated the URL for their LGPL repo citation IANAL-either but my mental model is that one should not "blaze trails" in making up licenses. I find it does not pass the straight-face test that no other license in the known world captures the author's intentions, so they had to make one up on the fly, typos and all
rgovostes 4y ago
Unfortunately, like macFUSE, this is not really open source. The license appears to be BSD-like for non-commercial use only. Also like macFUSE, the project uses GitHub but the source code is not available. https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt
- deleted 4y ago
  [deleted]
- skissane 4y ago
  The developer says they will release source code soon – https://github.com/macos-fuse-t/fuse-t/issues/1#issuecomment-1237429049 https://github.com/macos-fuse-t/fuse-t/issues/1#issuecomment... – although keeping the restrictive license. Apple is introducing user-space file system technology in macOS – LiveFS/UserFS/com.apple.filesystems.lifs – some of the infrastructure turned up in Monterey, in Ventura it is actually being used for mounting FAT/exFAT filesystems. It looks like for now Apple is keeping the API Apple-internal only (although I haven't looked at the Ventura SDK, so I could be wrong about that). But if Apple made the API public, it could be the death-knell of all these commercial FUSE-alternatives for macOS offerings. (Even if Apple's API isn't FUSE-compatible, if it is close enough, someone could easily open-source a translation layer – likely to be a lot simpler than bridging FUSE to an NFS server.)
  - ianlevesque 4y ago
    They have the right to use whatever licensing they want, but it is strange that this one area - FUSE on Mac - consistently attracts developers using weird license restrictions.
    - pjmlp 4y ago
      Because the Apple developer culture just like on the Windows side has always been welcoming to commercial software, that is how one keeps a paycheck doing desktop utilities.
    - 4y ago
EdSchouten 4y ago
What a coincidence! For a project that I maintain (Buildbarn, a distributed build cluster for Bazel) I recently generalized all of the FUSE code I had into a generic VFS that can both be exposed over FUSE and NFSv4. The intent was the same: to provide a better out of the box experience on macOS. Here's a design doc I wrote on that change. Slight warning that it's written with some Buildbarn knowledge in mind. https://github.com/buildbarn/bb-adrs/blob/master/0009-nfsv4.md https://github.com/buildbarn/bb-adrs/blob/master/0009-nfsv4.... Fortunately, fuse-t doesn't make any of my work unnecessary. Buildbarn uses go-fuse, which talks to the FUSE character directly instead of using libfuse. fuse-t would thus not be a drop-in replacement. Phew! PS: A bit unfortunate that fuse-t isn't Open Source. :-(
- EdSchouten 4y ago
  Replying to my own comment: I do wonder how this library deals with some of the fundamental differences beween FUSE and NFSv4. For example, with FUSE the kernel and server share intimate knowledge on which part of the file system lives in the kernel's inode cache. Only when the kernel issues a FORGET call, may the server drop information corresponding to a given nodeid. As NFSv4 is designed to be stateless, servers may need to be able to process requests containing arbitrary file handles, regardless of how long ago they were returned as part of some prior request. You therefore see that in-kernel implementations of NFS servers rely on special file system methods to resolve objects by file handle, in addition to being able to resolve by path. This means that if you implement FUSE on top of NFSv4, you will most likely not be able to purge any state. Your FUSE file system's FORGET method will probably never be called. This means that memory usage of fuse-t will most likely just keep on growing as time progresses? Or it announces itself as using file handle mode FH4_VOLATILE_*, but UNIX-like NFSv4 clients hardly ever know how to deal with that.
  - boulos 4y ago
    Nit: NFSv3 is stateless, V4 isn't (sessionid and clientid permit stateful handling of locks and shares). I would assume this person is just doing an in-memory "NFS server" that would keep all of that state around. So it's more like a FUSE-compatible layer that speaks NFSv4 as a "front end" (since NFS clients are better than "ha ha, surprise! This FUSE backend is networked and can fail on weird ways"). I'm not sure how they chose to translate things like DELEGATE, SEQUENCE, and so on (probably just reply with an error). But for basic OPEN, READ, WRITE, etc. it's all fairly straightforward. I had hoped they used nfs-ganesha for the NFS server / frontend, but the attributions file suggests they probably rolled their own (and thus is going to be full of bugs and immature for quite some time).
mdaniel 4y ago
I don't know what's going on with the google.com/url encoding of the links on the submitted article Also, one will want to be aware of the currently unsupported features <https://github.com/macos-fuse-t/fuse-t/wiki#unsupported-features https://github.com/macos-fuse-t/fuse-t/wiki#unsupported-feat...> since the submitted article appears to be aspirational
- tech234a 4y ago
  The page appears to be hosted on the new Google Sites.
HHad3 4y ago
This is great! I hope that this project enables those many FUSE-related applications to return to Homebrew since an open-source FUSE provider is now available again. I am curious though why a NFSv4 server was chosen over wrapping Apple's File Provider API [1], which seems to be the native method for providing virtual file systems from user space on macOS since macOS 11.5. After glancing over the API I guess it's because FPEs are too high-level to implement FUSE properly, but I'd be glad if you can share any details to satisfy my curiosity. [1] https://developer.apple.com/documentation/fileprovider https://developer.apple.com/documentation/fileprovider EDIT: Errata, I assumed it was open-source, but it is not. Too bad :( -- but at least this will eventually provide a more stable FUSE experience on macOS.
- mdaniel 4y ago
  It seems based on the associated WWDC video (https://developer.apple.com/videos/play/wwdc2021/10182/ https://developer.apple.com/videos/play/wwdc2021/10182/) that API appears to be for making your own Dropbox/GDrive/OneDrive client, and not "present this totally fake filesystem" scenario
tambourine_man 4y ago
macOS really needs a decent FUSE implementation. That and some sort of container story. Such a shame that it's not really open source. The idea of using NFSv4 is awesome though.
- mdaniel 4y ago
  > That and some sort of container story. While off-topic for this thread, I don't think that's going to do what you expect. If you snapped your fingers and XNU suddenly had cgroups and other namespace trickery required to make containers operate, you'd still have the grave problem of "containers are not virtual machines" and thus an XNU container would only run Darwin binaries, so you're back to the old days of "run one exe locally, run another in prod"
  - clhodapp 4y ago
    You're assuming that everyone's "prod" is a Linux server. I would say that MacOS needs containers for developing Darwin software.
    - mdaniel 4y ago
      Yes, I think that's a perfectly reasonable assumption given that (AFAIK) the only current "containerization" (as GP used that word) strategy is on Linux. BSD has jails and Solaris has something similar, but as far as "fire this thing up with its own pid, network, and fs namespacing, and allow me to constrain it easily" that's just Linux. I guess put another way: you run Darwin in production? As for the latter, macOS actually does have what they call Containers (https://developer.apple.com/library/archive/documentation/Security/Conceptual/AppSandboxDesignGuide/AppSandboxInDepth/AppSandboxInDepth.html#//apple_ref/doc/uid/TP40011183-CH3-SW6 https://developer.apple.com/library/archive/documentation/Se...) but as best I can tell such a thing requires opt-in from the app, which kind of defeats the purpose of running untrusted software IMHO. I actually only learned about that Containers stuff from trying to find where in the hell 1Password 8 stores its actual sqlite file: `$HOME/Library/Group Containers/2BUA8C4S2C.com.1password/Library/Application Support/1Password/Data/1password.sqlite`
als0 4y ago
In sore need of FUSE for macOS on Apple Silicon, without downgrading system security. Clever trick to use NFS behind the scenes.
orangea 4y ago
Why not use the file provider API? https://developer.apple.com/documentation/fileprovider https://developer.apple.com/documentation/fileprovider
- lillecarl 4y ago
  Because your software was written against FUSE and you don't wanna spend time rewriting it to accommodate Apple users.
  - orangea 4y ago
    I meant why not make a project like this that emulates the FUSE API, but use the file provider API to present the file system to the OS instead of NFS. It would have the advantage that it would be better integrated with the OS, for example you can display upload progress indicators on files in Finder. Maybe that can also be done with NFS though, I don't know.
    - 4ad 4y ago
      It's impossible, see my other reply.
- mathiasgredal 4y ago
  I found the api incredibly hard to use as someone who hasn't used swift or macOS API's before. Because of the way some of the code runs as a separate plugin from the main app, you can't really use traditional logging as a debug tool. There is also a lot of split state to be handled. You have to provide sync anchors and diffed tree updates to change the folder contents, which just aren't available with most backends. I started on a program to mount a website resource directory on your computer using this api, but I gave up due to the restrictions of the api: https://github.com/mathiasgredal/Itslearning https://github.com/mathiasgredal/Itslearning
- 4ad 4y ago
  The File Provider API is very limited. In particular, it can't be used to implement file systems where the directory hierarchy is dynamic. It can't materialize directories on `chdir(1)`, for example. The design was obviously tailored for providers that implement real disk files, like Dropbox and Apple's own iCloud Drive. Btw, Apple also ships with an undocumented 9P implementation. It seems to be used for mounting the host filesystem in virtualized guests. It is unclear if it can be made to work over a normal (non-PCI) transport like TCP.
jbverschoor 4y ago
Why does everybody use tcp ports instead of file sockets for local communication?
- pstuart 4y ago
  2 guesses: ignorance, and an assumption that using tcp allows for seamless transition to another host.
  - jbverschoor 4y ago
    Api is still socket api.. AF_INET vs AF_UNIX. It takes a few lines of code to have to option to switch between. Ignorance makes sense.. I guess full stack developers means people who can read the top 5 lines of a stack trace
    - macintux 4y ago
      > I guess full stack developers means people who can read the top 5 lines of a stack trace C'mon. Do I really need to quote the guidelines to you?
      - jbverschoor 4y ago
        Oh was not meant as a personal attack or anything. Just general state of the industry
  - klabb3 4y ago
    I guess I'm ignorant then! When I looked up domain sockets and so on it turned out to be different APIs for different OS:s, and it's significantly nicer to rely on a single API surface from the std lib. Maybe it's a habit thing as well, but to me pipes are more esoteric and harder to find docs about than network sockets.
    - pstuart 4y ago
      > I guess I'm ignorant then! We all are in our own ways ;-). I pulled that comment out of my ass so I'm ok with being corrected as appropriate.
mrpippy 4y ago
macOS Ventura has moved some lesser-used file systems (FAT, exFAT) out of the kernel to user-space, I believe using a private framework developed for iOS (when that added USB mass-storage support). Hopefully they’ll open those APIs up for 3rd party usage on the Mac next year.
- saagarjha 4y ago
  My impression was that there were no plans to do this.
guruz 4y ago
ocsmount for WebDAV is also based on a local NFS server. https://ocsmount.com/ https://ocsmount.com/
schappim 4y ago
FUSE really demonstrates the unreasonable effectiveness of file systems as an interface.
- junon 4y ago
  Or perhaps the unreasonable ineffectiveness of current-day IPC mechanisms.
skissane 4y ago
What advantages does NFSv4 have over SMB/CIFS for this use case? I have had the same idea myself before, but with a more cross-platform scope (Linux and Windows too, not just macOS). Whatever it flaws, SMB/CIFS has the advantage of being supported out-of-the-box on more platforms than NFSv4 is.
- chungy 4y ago
  Are you sure about that? Things that support SMB out of the box: Windows, Mac Things that support NFSv4 out of the box: Windows, Mac, Linux, FreeBSD, OpenBSD, NetBSD, illumos
  - skissane 4y ago
    The Linux kernel contains both NFS and SMB clients – so in that sense, supports both equally "out of the box". Whether either or both are compiled-in (or available as loadable modules) will depend on the distribution. Windows has an NFS client, but it is an additional OS feature which isn't installed by default (and I believe its NFS version support is somewhat outdated?) Whether Linux distributions install the NFS and/or SMB client support by default, or require an additional package install for either or both, is really going to depend on the distribution (and its installation options) FreeBSD, NetBSD and Illumos also include SMB client support, but I don't know how easy it is to use them as compared to their NFS clients. (From what I understand, OpenBSD pointedly has first-class support for NFS but not SMB: NFS has an in-kernel client, for SMB you have to use a user-space NFS-to-SMB translation daemon, which is in their ports tree.)
    - chungy 4y ago
      > Windows has an NFS client, but it is an additional OS feature which isn't installed by default (and I believe its NFS version support is somewhat outdated?) Windows has both NFS server and client. It's not enabled by default, but it can be enabled. It has supported NFSv4 since Windows 8.
      - skissane 4y ago
realloc 4y ago
I built something similar for a personal project (though not using the FUSE API) with Samba running locally. Samba has a VFS API which isn’t terrible complicated and lets you accomplish mostly the same thing: https://wiki.samba.org/index.php/Writing_a_Samba_VFS_Module https://wiki.samba.org/index.php/Writing_a_Samba_VFS_Module The only problem I ran into was not being able to bind to 127.0.0.1:445, or connect to a different port, on Windows. I ended up writing a small pcap program that would look for packets going out to some other ip and replay them back at localhost on the port samba was running on, so Windows thought it was connecting to a remote machine. It was a ridiculous solution and I’m sure there’s a better way, but it worked for what I needed.
ninefathom 4y ago
"You're gonna microkernel whether you want to or not! Only our devs are allowed to crash the system." --Apple
- lproven 4y ago
  You say that like it's a bad thing.
0xedd 4y ago
Like buying a tricycle, for the price of a 250cc motorcycle, and wrenching a 20cc engine on it.
lvh 4y ago
This is super smart. I've been working on a pet project that requires a VFS, and I ended up with a FUSE impl for Linux (and macOS if so inclined) and a userspace (specifically, JVM) NFSv4 impl :D Really excited to throw out some code.
netheril96 4y ago
Do binaries compiled and linked to macfuse run unmodified with FUSE-T, or should they be recompiled?
opless 4y ago
If only the 9pfs protocol was wildly implemented, we'd not want fuse at all!

301 points by freebeer2 4y ago

This is a Bad idea. Do you expect Apple has any consideration that things like this are happening inside the usermode processes when its System suspends and compresses or decompresses their memory of or throttles them by flinging them back and forth to different classes of CPU cores or analyzes them to look for other efficiencies that they don’t talk about publicly (network, thermal, sandbox, etc). If Apple wants FUSE, it will build a carve-out (in the form of a framework) like it did for virtualization. If you still want to use FUSE, maybe try System76’s Pop!_OS or find The [Apple] Way to accomplish your client’s goal. https://youtu.be/1LHuEvw56F8 https://youtu.be/1LHuEvw56F8

freebeer2 4y ago

A new project aims to deliver libfuse for mac without using kernel extensions.

keepquestioning 4y ago
Could this be done using DriverKit
- mdaniel 4y ago
  linky: https://developer.apple.com/documentation/driverkit https://developer.apple.com/documentation/driverkit > The drivers you build with DriverKit run in user space, rather than as kernel extensions, which improves system stability and security. You create your driver as an app extension and deliver it inside your existing app. makes it seem like every consumer who wants to use something FUSE-y would have to ship their own impl, right? and I shudder to think of the heartache required with signing or whatever Apple gatekeeping is going on nowadays
- concerned_ctzn 4y ago
  The author here. DriverKit is really a no-go. A while back I was asked to do a project based on DriverKit, it took me nowhere because of countless bugs and semi-implemented features, worse yet, it caused system crashes (and it was supposed to be stable).
  - keepquestioning 4y ago
    You should send feedback
    - DannyBee 4y ago
      Maybe? Maybe they also shouldn't ship garbage that doesn't work and expect others to debug it? Maybe this should also happen before you deprecate and make it essentially impossible to install kext's? (It's possible, but trying to walk users through it is a great way to lose 99% of your user base instantly) (My experience with driverkit is identical to the parent's) The whole "we removed it for security reasons" is also a hilarious facade. The linux kernel has a much more expansive (heck, crazy!) driver interface and number of drivers. Yet, the rate of system compromise due to them vs applications/servers is probably 99 to 1 in favor of applications/servers.
      - saagarjha 4y ago
        That’s because applications have a bunch of easy logic bugs to exploit rather than weird memory corruption you have to wrangle with in a driver, not because the drivers are any more secure.

concerned_ctzn 4y ago

https://github.com/macos-fuse-t/fuse-t https://github.com/macos-fuse-t/fuse-t

mdaniel 4y ago
That "repo" appears to exist solely for housing the release artifacts, for those who came to the comments looking for "the goods" Also, https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt appears to be "playing lawyer"
- TimTheTinker 4y ago
  > Also, https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt appears to be "playing lawyer" Do you have any specific concerns with the text of the license? The text seems clear enough that there don't appear to be any potential liabilities from using the software in a personal capacity. As I understand it, the worst that could happen is that the license/software author could unknowingly incur liabilities from statutory or implied warranties - i.e. they left out some important exclusions that could be brought to bear if a plaintiff successfully construed the license as a contract. (IANAL, but I've read a lot about software licenses over the years and have dealt with a legal challenge related to dual commercial/OSS licensing)
  - rgovostes 4y ago
    For one thing, they copied the clauses from the BSD license, including references to “the following disclaimer”, but omitted the disclaimer itself.
  - mdaniel 4y ago
    Aside from the great sibling observation, I'll pile onto that by pointing out they seem to have truncated the URL for their LGPL repo citation IANAL-either but my mental model is that one should not "blaze trails" in making up licenses. I find it does not pass the straight-face test that no other license in the known world captures the author's intentions, so they had to make one up on the fly, typos and all

rgovostes 4y ago

Unfortunately, like macFUSE, this is not really open source. The license appears to be BSD-like for non-commercial use only. Also like macFUSE, the project uses GitHub but the source code is not available. https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt https://github.com/macos-fuse-t/fuse-t/blob/main/License.txt

deleted 4y ago
[deleted]
skissane 4y ago
The developer says they will release source code soon – https://github.com/macos-fuse-t/fuse-t/issues/1#issuecomment-1237429049 https://github.com/macos-fuse-t/fuse-t/issues/1#issuecomment... – although keeping the restrictive license. Apple is introducing user-space file system technology in macOS – LiveFS/UserFS/com.apple.filesystems.lifs – some of the infrastructure turned up in Monterey, in Ventura it is actually being used for mounting FAT/exFAT filesystems. It looks like for now Apple is keeping the API Apple-internal only (although I haven't looked at the Ventura SDK, so I could be wrong about that). But if Apple made the API public, it could be the death-knell of all these commercial FUSE-alternatives for macOS offerings. (Even if Apple's API isn't FUSE-compatible, if it is close enough, someone could easily open-source a translation layer – likely to be a lot simpler than bridging FUSE to an NFS server.)
- ianlevesque 4y ago
  They have the right to use whatever licensing they want, but it is strange that this one area - FUSE on Mac - consistently attracts developers using weird license restrictions.
  - pjmlp 4y ago
    Because the Apple developer culture just like on the Windows side has always been welcoming to commercial software, that is how one keeps a paycheck doing desktop utilities.
  - 4y ago

EdSchouten 4y ago

What a coincidence! For a project that I maintain (Buildbarn, a distributed build cluster for Bazel) I recently generalized all of the FUSE code I had into a generic VFS that can both be exposed over FUSE and NFSv4. The intent was the same: to provide a better out of the box experience on macOS. Here's a design doc I wrote on that change. Slight warning that it's written with some Buildbarn knowledge in mind. https://github.com/buildbarn/bb-adrs/blob/master/0009-nfsv4.md https://github.com/buildbarn/bb-adrs/blob/master/0009-nfsv4.... Fortunately, fuse-t doesn't make any of my work unnecessary. Buildbarn uses go-fuse, which talks to the FUSE character directly instead of using libfuse. fuse-t would thus not be a drop-in replacement. Phew! PS: A bit unfortunate that fuse-t isn't Open Source. :-(

EdSchouten 4y ago
Replying to my own comment: I do wonder how this library deals with some of the fundamental differences beween FUSE and NFSv4. For example, with FUSE the kernel and server share intimate knowledge on which part of the file system lives in the kernel's inode cache. Only when the kernel issues a FORGET call, may the server drop information corresponding to a given nodeid. As NFSv4 is designed to be stateless, servers may need to be able to process requests containing arbitrary file handles, regardless of how long ago they were returned as part of some prior request. You therefore see that in-kernel implementations of NFS servers rely on special file system methods to resolve objects by file handle, in addition to being able to resolve by path. This means that if you implement FUSE on top of NFSv4, you will most likely not be able to purge any state. Your FUSE file system's FORGET method will probably never be called. This means that memory usage of fuse-t will most likely just keep on growing as time progresses? Or it announces itself as using file handle mode FH4_VOLATILE_*, but UNIX-like NFSv4 clients hardly ever know how to deal with that.
- boulos 4y ago
  Nit: NFSv3 is stateless, V4 isn't (sessionid and clientid permit stateful handling of locks and shares). I would assume this person is just doing an in-memory "NFS server" that would keep all of that state around. So it's more like a FUSE-compatible layer that speaks NFSv4 as a "front end" (since NFS clients are better than "ha ha, surprise! This FUSE backend is networked and can fail on weird ways"). I'm not sure how they chose to translate things like DELEGATE, SEQUENCE, and so on (probably just reply with an error). But for basic OPEN, READ, WRITE, etc. it's all fairly straightforward. I had hoped they used nfs-ganesha for the NFS server / frontend, but the attributions file suggests they probably rolled their own (and thus is going to be full of bugs and immature for quite some time).

mdaniel 4y ago

I don't know what's going on with the google.com/url encoding of the links on the submitted article Also, one will want to be aware of the currently unsupported features <https://github.com/macos-fuse-t/fuse-t/wiki#unsupported-features https://github.com/macos-fuse-t/fuse-t/wiki#unsupported-feat...> since the submitted article appears to be aspirational

tech234a 4y ago
The page appears to be hosted on the new Google Sites.

HHad3 4y ago

This is great! I hope that this project enables those many FUSE-related applications to return to Homebrew since an open-source FUSE provider is now available again. I am curious though why a NFSv4 server was chosen over wrapping Apple's File Provider API [1], which seems to be the native method for providing virtual file systems from user space on macOS since macOS 11.5. After glancing over the API I guess it's because FPEs are too high-level to implement FUSE properly, but I'd be glad if you can share any details to satisfy my curiosity. [1] https://developer.apple.com/documentation/fileprovider https://developer.apple.com/documentation/fileprovider EDIT: Errata, I assumed it was open-source, but it is not. Too bad :( -- but at least this will eventually provide a more stable FUSE experience on macOS.

mdaniel 4y ago
It seems based on the associated WWDC video (https://developer.apple.com/videos/play/wwdc2021/10182/ https://developer.apple.com/videos/play/wwdc2021/10182/) that API appears to be for making your own Dropbox/GDrive/OneDrive client, and not "present this totally fake filesystem" scenario

tambourine_man 4y ago

macOS really needs a decent FUSE implementation. That and some sort of container story. Such a shame that it's not really open source. The idea of using NFSv4 is awesome though.

mdaniel 4y ago
> That and some sort of container story. While off-topic for this thread, I don't think that's going to do what you expect. If you snapped your fingers and XNU suddenly had cgroups and other namespace trickery required to make containers operate, you'd still have the grave problem of "containers are not virtual machines" and thus an XNU container would only run Darwin binaries, so you're back to the old days of "run one exe locally, run another in prod"
- clhodapp 4y ago
  You're assuming that everyone's "prod" is a Linux server. I would say that MacOS needs containers for developing Darwin software.
  - mdaniel 4y ago
    Yes, I think that's a perfectly reasonable assumption given that (AFAIK) the only current "containerization" (as GP used that word) strategy is on Linux. BSD has jails and Solaris has something similar, but as far as "fire this thing up with its own pid, network, and fs namespacing, and allow me to constrain it easily" that's just Linux. I guess put another way: you run Darwin in production? As for the latter, macOS actually does have what they call Containers (https://developer.apple.com/library/archive/documentation/Security/Conceptual/AppSandboxDesignGuide/AppSandboxInDepth/AppSandboxInDepth.html#//apple_ref/doc/uid/TP40011183-CH3-SW6 https://developer.apple.com/library/archive/documentation/Se...) but as best I can tell such a thing requires opt-in from the app, which kind of defeats the purpose of running untrusted software IMHO. I actually only learned about that Containers stuff from trying to find where in the hell 1Password 8 stores its actual sqlite file: `$HOME/Library/Group Containers/2BUA8C4S2C.com.1password/Library/Application Support/1Password/Data/1password.sqlite`

als0 4y ago

In sore need of FUSE for macOS on Apple Silicon, without downgrading system security. Clever trick to use NFS behind the scenes.

orangea 4y ago

Why not use the file provider API? https://developer.apple.com/documentation/fileprovider https://developer.apple.com/documentation/fileprovider

lillecarl 4y ago
Because your software was written against FUSE and you don't wanna spend time rewriting it to accommodate Apple users.
- orangea 4y ago
  I meant why not make a project like this that emulates the FUSE API, but use the file provider API to present the file system to the OS instead of NFS. It would have the advantage that it would be better integrated with the OS, for example you can display upload progress indicators on files in Finder. Maybe that can also be done with NFS though, I don't know.
  - 4ad 4y ago
    It's impossible, see my other reply.
mathiasgredal 4y ago
I found the api incredibly hard to use as someone who hasn't used swift or macOS API's before. Because of the way some of the code runs as a separate plugin from the main app, you can't really use traditional logging as a debug tool. There is also a lot of split state to be handled. You have to provide sync anchors and diffed tree updates to change the folder contents, which just aren't available with most backends. I started on a program to mount a website resource directory on your computer using this api, but I gave up due to the restrictions of the api: https://github.com/mathiasgredal/Itslearning https://github.com/mathiasgredal/Itslearning
4ad 4y ago
The File Provider API is very limited. In particular, it can't be used to implement file systems where the directory hierarchy is dynamic. It can't materialize directories on `chdir(1)`, for example. The design was obviously tailored for providers that implement real disk files, like Dropbox and Apple's own iCloud Drive. Btw, Apple also ships with an undocumented 9P implementation. It seems to be used for mounting the host filesystem in virtualized guests. It is unclear if it can be made to work over a normal (non-PCI) transport like TCP.

jbverschoor 4y ago

Why does everybody use tcp ports instead of file sockets for local communication?

pstuart 4y ago
2 guesses: ignorance, and an assumption that using tcp allows for seamless transition to another host.
- jbverschoor 4y ago
  Api is still socket api.. AF_INET vs AF_UNIX. It takes a few lines of code to have to option to switch between. Ignorance makes sense.. I guess full stack developers means people who can read the top 5 lines of a stack trace
  - macintux 4y ago
    > I guess full stack developers means people who can read the top 5 lines of a stack trace C'mon. Do I really need to quote the guidelines to you?
    - jbverschoor 4y ago
      Oh was not meant as a personal attack or anything. Just general state of the industry
- klabb3 4y ago
  I guess I'm ignorant then! When I looked up domain sockets and so on it turned out to be different APIs for different OS:s, and it's significantly nicer to rely on a single API surface from the std lib. Maybe it's a habit thing as well, but to me pipes are more esoteric and harder to find docs about than network sockets.
  - pstuart 4y ago
    > I guess I'm ignorant then! We all are in our own ways ;-). I pulled that comment out of my ass so I'm ok with being corrected as appropriate.

mrpippy 4y ago

macOS Ventura has moved some lesser-used file systems (FAT, exFAT) out of the kernel to user-space, I believe using a private framework developed for iOS (when that added USB mass-storage support). Hopefully they’ll open those APIs up for 3rd party usage on the Mac next year.

saagarjha 4y ago
My impression was that there were no plans to do this.

guruz 4y ago

ocsmount for WebDAV is also based on a local NFS server. https://ocsmount.com/ https://ocsmount.com/

schappim 4y ago

FUSE really demonstrates the unreasonable effectiveness of file systems as an interface.

junon 4y ago
Or perhaps the unreasonable ineffectiveness of current-day IPC mechanisms.

skissane 4y ago

What advantages does NFSv4 have over SMB/CIFS for this use case? I have had the same idea myself before, but with a more cross-platform scope (Linux and Windows too, not just macOS). Whatever it flaws, SMB/CIFS has the advantage of being supported out-of-the-box on more platforms than NFSv4 is.

chungy 4y ago
Are you sure about that? Things that support SMB out of the box: Windows, Mac Things that support NFSv4 out of the box: Windows, Mac, Linux, FreeBSD, OpenBSD, NetBSD, illumos
- skissane 4y ago
  The Linux kernel contains both NFS and SMB clients – so in that sense, supports both equally "out of the box". Whether either or both are compiled-in (or available as loadable modules) will depend on the distribution. Windows has an NFS client, but it is an additional OS feature which isn't installed by default (and I believe its NFS version support is somewhat outdated?) Whether Linux distributions install the NFS and/or SMB client support by default, or require an additional package install for either or both, is really going to depend on the distribution (and its installation options) FreeBSD, NetBSD and Illumos also include SMB client support, but I don't know how easy it is to use them as compared to their NFS clients. (From what I understand, OpenBSD pointedly has first-class support for NFS but not SMB: NFS has an in-kernel client, for SMB you have to use a user-space NFS-to-SMB translation daemon, which is in their ports tree.)
  - chungy 4y ago
    > Windows has an NFS client, but it is an additional OS feature which isn't installed by default (and I believe its NFS version support is somewhat outdated?) Windows has both NFS server and client. It's not enabled by default, but it can be enabled. It has supported NFSv4 since Windows 8.
    - skissane 4y ago

realloc 4y ago

I built something similar for a personal project (though not using the FUSE API) with Samba running locally. Samba has a VFS API which isn’t terrible complicated and lets you accomplish mostly the same thing: https://wiki.samba.org/index.php/Writing_a_Samba_VFS_Module https://wiki.samba.org/index.php/Writing_a_Samba_VFS_Module The only problem I ran into was not being able to bind to 127.0.0.1:445, or connect to a different port, on Windows. I ended up writing a small pcap program that would look for packets going out to some other ip and replay them back at localhost on the port samba was running on, so Windows thought it was connecting to a remote machine. It was a ridiculous solution and I’m sure there’s a better way, but it worked for what I needed.

ninefathom 4y ago

"You're gonna microkernel whether you want to or not! Only our devs are allowed to crash the system." --Apple

lproven 4y ago
You say that like it's a bad thing.

0xedd 4y ago

Like buying a tricycle, for the price of a 250cc motorcycle, and wrenching a 20cc engine on it.

lvh 4y ago

This is super smart. I've been working on a pet project that requires a VFS, and I ended up with a FUSE impl for Linux (and macOS if so inclined) and a userspace (specifically, JVM) NFSv4 impl :D Really excited to throw out some code.

netheril96 4y ago

Do binaries compiled and linked to macfuse run unmodified with FUSE-T, or should they be recompiled?

opless 4y ago

If only the 9pfs protocol was wildly implemented, we'd not want fuse at all!