4 ms·

It bothers me that the Linux community still hasn't found a better way to mount file systems over a network than the Microsofty case-insensitive SMB. NFS is a

by dheera 5y ago

It bothers me that the Linux community still hasn't found a better way to mount file systems over a network than the Microsofty case-insensitive SMB. NFS is a lost cause if you want any sort of permissions or security, NFS with Kerberos and AFS are mind-bogglingly complex to set up for a home or small organization, SSHFS is slow, what else is there?

spijdar 5y ago
SMB isn't necessarily case-insensitive. You can configure Samba to be case-sensitive, and I believe it defaults to case sensitivity for UNIX-y clients. Aside from being "Microsofty", Samba is just a good overall option, and I imagine that's why no-one has really bothered to develop another robust fileserver in the same niche.
- mappu 5y ago
  As well as Samba, Samsung's CIFSD is headed for the mainline kernel: https://www.phoronix.com/scan.php?page=news_item&px=CIFSD-In-Kernel-SMB3-Next https://www.phoronix.com/scan.php?page=news_item&px=CIFSD-In...
jessedhillon 5y ago
sshfs has worked out fine for me, and feels more continuous with the *nix user experience.
- dheera 5y ago
  Continuous yes but it is horrible at dealing with hundreds of GB of data and accessing multiple files at once.
_delirium 5y ago
NFSv4 uses only a single port, which allows an option of skipping any complicated Kerberos setup and getting encryption by just tunneling the connection through stunnel: https://www.linuxjournal.com/content/encrypting-nfsv4-stunnel-tls https://www.linuxjournal.com/content/encrypting-nfsv4-stunne... A version of that scheme is in the process of being standardized as part of the official NFSv4 protocol: https://datatracker.ietf.org/doc/html/draft-ietf-nfsv4-rpc-tls https://datatracker.ietf.org/doc/html/draft-ietf-nfsv4-rpc-t... (see "Acknowledgments" at the end for a mention of the connection).
- denkmoon 5y ago
  And the permissions problem? My experience with NFS is that permissions fall over the moment you've got root on a box that is allowed to connect. You're essentially relying on network level access control to prevent naughtyness. If I can create a user that has the same UID as someone's files I want on a remote NFS, I can just go do that? Maybe I've always misconfigured NFS, but as far as I know outside of kerberos there is no way to have independent authorisation of users.
  - yrro 5y ago
    Yeah, you need kerberos. OTOH it's not as difficult as people seem to think. Maybe if you insist on DIY rather than getting started with FreeIPA or even whisper it Active Directory...
  - chasil 5y ago
    I wrote the article on stunnel encryption. NFSv3 was strictly numeric uid/gid, but this is not the case with NFSv4. It is still best to keep uid/gid values synchronized, but there are idmapd processes that negotiate permissions between client and server. Examine the output of "man idmapd.conf" for details. It even does LDAP.
    - dheera 5y ago
      > It is still best to keep uid/gid values synchronized That's impractical in a large organization. They really should have used a username@domain or UUID.
- yrro 5y ago
  Hmm, does that enable strong authentication of the users making RPC requests? Or does the server still have to trust that the client making RPC requests is being honest? Specifically I wonder if the client is compromised, is it able to start making RPC requests for any arbitrary user (as with AUTH_SYS), or only for any user who uses their own credentials to establish a security context (as with RPCSEC_GSS)? Forgive my imprecise terminology here, I'm clearly not an NFS/RPC expert! :) [edit] Ah, I see this is addressed in Sec. 7 - never mind. It looks like you still need to use GSS on top of this scheme in order to prevent a compromised client from impersonating any user.
mappu 5y ago
A cluster filesystem (ocfs2/gluster/lizard/...) with NBD? Minio and s3fs?
- selfhoster11 5y ago
  S3 mounts are not an alternative. Relying on FUSE S3 implementations is a recipe for losing files to unknown hang-ups during the upload stage. It's far better in my experience to use a dedicated S3 client application.
netflixandkill 5y ago
What absolute nonsense is this? NFS is far from perfect but aces the insane stupid complexity of SMB any day of the week and twice on Sunday
- throwaway2048 5y ago
  NFS is a security disaster, its basic security model is "trust the client when they say they are root", and using it on a network with packet loss (wireless networks in particular, but EVERY network in the long run) is a recipe for silent corruption, stale mounts and kernel panics. TCP vs UDP helps to a limited degree, but has annoying tradeoffs and isn't full proof against the problems.
  - unixhero 5y ago
    Stale mounts is for me the killer
    - deleted 5y ago
      [deleted]
    - stuaxo 5y ago
      Yep, a network file system that doesn't handle disconnections and reconnections well isn't great.
  - netflixandkill 5y ago
    NFS servers can remap remote UIDs to anything they want. This has been normal practice with less trusted clients for decades. If you're dealing with unreliable networks, you probably want a remote syncing system as opposed to any rpc oriented remote filesystem. Rclone works remarkably well for that purpose. In any case asserting that SMB is somehow preferable to any other option is insanity.
toast0 5y ago
9p maybe? I've never used it, but it would seem to fit.
NegativeLatency 5y ago
I use netatalk for AFP on Linux, it works ok, but I only have macOS clients I plan on migrating to SMB eventually Tried NFS for a while but getting permissions to work proved too difficult
unixhero 5y ago
Sure we have. Sshfs.
MayeulC 5y ago
I've been using NFSv4 over wireguard for a few weeks. It works really well, with speeds over 80 MBps for residential FTTH. Configuring a mesh wireguard network can take some time, though it looks like that's being worked on on and off. The main issue is having to update every config file when a new host joins. On the flip side, having each party authenticate the other means it should be pretty robust. You can bind NFS to the wireguard interface, and restrict by IP. Wireguard checks public keys against "authorizedips" before forwarding packets, so that should be pretty secure. I added the IPs to `/etc/hosts` so that I could allow a wildcard, that works kind of like netmasks, in a way. However, it seems not to handle servers coming and going that gracefully. My other remaining gripe is that there is no mount option to override gid and uid, besides setting up a mapping file; I don't need that granularity. Now, another option could be 9p (over wireguard), I haven't tried it, but it could be more flexible than NFS.
handrous 5y ago
I hate SMB, but the real problem, from my perspective, is that SMB is the only option if you want everything to be able to talk to your share. With NFS or SSHFS or anything else, you're cutting out a bunch client devices & software. Like FAT32 in the early days of USB drives, you use it because everything speaks it, not because it's the best thing around. FWIW the sanest way I've found to configure SMB shares on any OS is Docker on Linux. Concise for common use-cases, rock solid, config won't change out from under you for unclear reasons. I think an underrated benefit of Docker is that it strongly encourages cutting the bullshit out of configuration and getting straight to the point, for what most people want to do with the image. The config lines are gibberish, but at least they're very short gibberish. IIRC one option line per user, one option line per share (maybe one other per share to map the real directory to the container? Can't recall for sure), everything for each of them smushed together on that one line. No "what kind of user is meant here, again, since there are two kinds?" garbage, no "all three of these options look relevant, but which one is actually being applied in this particular case?" or "which section does this option go in?" that I recall from the hell of trying to configure Samba to do very normal, boring things over the years. 90% of the time all I want is "anonymous read, this username and password for writes, share directories x, y, and z" which is so normal a thing to want to do with it that the config shouldn't occupy much more space than it took to write that out—and, indeed, using the Docker image, it doesn't, and you don't have to deal with outside-the-config user state because the image takes care of that for you, so the config is an easily portable, tiny text file that depends on nothing else but the existence of the directories you're telling it to mount & share, unlike a normal samba config file.
Crontab 5y ago
I've been considering trying to wrap NFS with Colin Percival's spiped[1]. It should work if you only use TCP. [1] https://www.tarsnap.com/spiped.html https://www.tarsnap.com/spiped.html

by dheera 5y ago

spijdar 5y ago

SMB isn't necessarily case-insensitive. You can configure Samba to be case-sensitive, and I believe it defaults to case sensitivity for UNIX-y clients. Aside from being "Microsofty", Samba is just a good overall option, and I imagine that's why no-one has really bothered to develop another robust fileserver in the same niche.

mappu 5y ago
As well as Samba, Samsung's CIFSD is headed for the mainline kernel: https://www.phoronix.com/scan.php?page=news_item&px=CIFSD-In-Kernel-SMB3-Next https://www.phoronix.com/scan.php?page=news_item&px=CIFSD-In...

jessedhillon 5y ago

sshfs has worked out fine for me, and feels more continuous with the *nix user experience.

dheera 5y ago
Continuous yes but it is horrible at dealing with hundreds of GB of data and accessing multiple files at once.

_delirium 5y ago

NFSv4 uses only a single port, which allows an option of skipping any complicated Kerberos setup and getting encryption by just tunneling the connection through stunnel: https://www.linuxjournal.com/content/encrypting-nfsv4-stunnel-tls https://www.linuxjournal.com/content/encrypting-nfsv4-stunne... A version of that scheme is in the process of being standardized as part of the official NFSv4 protocol: https://datatracker.ietf.org/doc/html/draft-ietf-nfsv4-rpc-tls https://datatracker.ietf.org/doc/html/draft-ietf-nfsv4-rpc-t... (see "Acknowledgments" at the end for a mention of the connection).

denkmoon 5y ago
And the permissions problem? My experience with NFS is that permissions fall over the moment you've got root on a box that is allowed to connect. You're essentially relying on network level access control to prevent naughtyness. If I can create a user that has the same UID as someone's files I want on a remote NFS, I can just go do that? Maybe I've always misconfigured NFS, but as far as I know outside of kerberos there is no way to have independent authorisation of users.
- yrro 5y ago
  Yeah, you need kerberos. OTOH it's not as difficult as people seem to think. Maybe if you insist on DIY rather than getting started with FreeIPA or even whisper it Active Directory...
- chasil 5y ago
  I wrote the article on stunnel encryption. NFSv3 was strictly numeric uid/gid, but this is not the case with NFSv4. It is still best to keep uid/gid values synchronized, but there are idmapd processes that negotiate permissions between client and server. Examine the output of "man idmapd.conf" for details. It even does LDAP.
  - dheera 5y ago
    > It is still best to keep uid/gid values synchronized That's impractical in a large organization. They really should have used a username@domain or UUID.
yrro 5y ago
Hmm, does that enable strong authentication of the users making RPC requests? Or does the server still have to trust that the client making RPC requests is being honest? Specifically I wonder if the client is compromised, is it able to start making RPC requests for any arbitrary user (as with AUTH_SYS), or only for any user who uses their own credentials to establish a security context (as with RPCSEC_GSS)? Forgive my imprecise terminology here, I'm clearly not an NFS/RPC expert! :) [edit] Ah, I see this is addressed in Sec. 7 - never mind. It looks like you still need to use GSS on top of this scheme in order to prevent a compromised client from impersonating any user.

mappu 5y ago

A cluster filesystem (ocfs2/gluster/lizard/...) with NBD? Minio and s3fs?

selfhoster11 5y ago
S3 mounts are not an alternative. Relying on FUSE S3 implementations is a recipe for losing files to unknown hang-ups during the upload stage. It's far better in my experience to use a dedicated S3 client application.

netflixandkill 5y ago

What absolute nonsense is this? NFS is far from perfect but aces the insane stupid complexity of SMB any day of the week and twice on Sunday

throwaway2048 5y ago
NFS is a security disaster, its basic security model is "trust the client when they say they are root", and using it on a network with packet loss (wireless networks in particular, but EVERY network in the long run) is a recipe for silent corruption, stale mounts and kernel panics. TCP vs UDP helps to a limited degree, but has annoying tradeoffs and isn't full proof against the problems.
- unixhero 5y ago
  Stale mounts is for me the killer
  - deleted 5y ago
    [deleted]
  - stuaxo 5y ago
    Yep, a network file system that doesn't handle disconnections and reconnections well isn't great.
- netflixandkill 5y ago
  NFS servers can remap remote UIDs to anything they want. This has been normal practice with less trusted clients for decades. If you're dealing with unreliable networks, you probably want a remote syncing system as opposed to any rpc oriented remote filesystem. Rclone works remarkably well for that purpose. In any case asserting that SMB is somehow preferable to any other option is insanity.

toast0 5y ago

9p maybe? I've never used it, but it would seem to fit.

NegativeLatency 5y ago

I use netatalk for AFP on Linux, it works ok, but I only have macOS clients I plan on migrating to SMB eventually Tried NFS for a while but getting permissions to work proved too difficult

unixhero 5y ago

Sure we have. Sshfs.

MayeulC 5y ago

I've been using NFSv4 over wireguard for a few weeks. It works really well, with speeds over 80 MBps for residential FTTH. Configuring a mesh wireguard network can take some time, though it looks like that's being worked on on and off. The main issue is having to update every config file when a new host joins. On the flip side, having each party authenticate the other means it should be pretty robust. You can bind NFS to the wireguard interface, and restrict by IP. Wireguard checks public keys against "authorizedips" before forwarding packets, so that should be pretty secure. I added the IPs to `/etc/hosts` so that I could allow a wildcard, that works kind of like netmasks, in a way. However, it seems not to handle servers coming and going that gracefully. My other remaining gripe is that there is no mount option to override gid and uid, besides setting up a mapping file; I don't need that granularity. Now, another option could be 9p (over wireguard), I haven't tried it, but it could be more flexible than NFS.

handrous 5y ago

I hate SMB, but the real problem, from my perspective, is that SMB is the only option if you want everything to be able to talk to your share. With NFS or SSHFS or anything else, you're cutting out a bunch client devices & software. Like FAT32 in the early days of USB drives, you use it because everything speaks it, not because it's the best thing around. FWIW the sanest way I've found to configure SMB shares on any OS is Docker on Linux. Concise for common use-cases, rock solid, config won't change out from under you for unclear reasons. I think an underrated benefit of Docker is that it strongly encourages cutting the bullshit out of configuration and getting straight to the point, for what most people want to do with the image. The config lines are gibberish, but at least they're very short gibberish. IIRC one option line per user, one option line per share (maybe one other per share to map the real directory to the container? Can't recall for sure), everything for each of them smushed together on that one line. No "what kind of user is meant here, again, since there are two kinds?" garbage, no "all three of these options look relevant, but which one is actually being applied in this particular case?" or "which section does this option go in?" that I recall from the hell of trying to configure Samba to do very normal, boring things over the years. 90% of the time all I want is "anonymous read, this username and password for writes, share directories x, y, and z" which is so normal a thing to want to do with it that the config shouldn't occupy much more space than it took to write that out—and, indeed, using the Docker image, it doesn't, and you don't have to deal with outside-the-config user state because the image takes care of that for you, so the config is an easily portable, tiny text file that depends on nothing else but the existence of the directories you're telling it to mount & share, unlike a normal samba config file.

Crontab 5y ago

I've been considering trying to wrap NFS with Colin Percival's spiped[1]. It should work if you only use TCP. [1] https://www.tarsnap.com/spiped.html https://www.tarsnap.com/spiped.html