3 ms·

Rachel's advice, "Kick NFS to the curb", is good. My last startup, blekko (2007-now), managed to have no NFS usage, ever. Didn't miss it. Peak of 50 devs, 1500

Rachel's advice, "Kick NFS to the curb", is good. My last startup, blekko (2007-now), managed to have no NFS usage, ever. Didn't miss it. Peak of 50 devs, 1500 servers, no NFS. Even before that, in a supercomputing context (2002-), when writing MPI "wrapper" software that would be used on end-user clusters with all kinds of unfortunate network setups, I avoided a lot of trouble by copying the executable to the local node before running it. This isn't a new issue. And the traditional solutions work.

lmm 9y ago
I've found that even for unix-unix, samba is just more reliable with more reasonable error behaviour.
- WHoWHo 9y ago
  To me samba has just too much magic, especially when mixing unix, OSX and windows clients.
  - pnutjam 9y ago
    magic is just technology we don't understand. I use cifs over nfs as well.
    - WHoWHo 9y ago
      With excel locking, os x trashes, etc.? What good configuration guide would you suggest for a mixed Linux, OS X and Windows environment with MS Office? This magic only solves have the problems fruit:resource = xattr fruit:metadata = netatalk fruit:locking = none fruit:encoding = private delete veto files = yes veto files = /.snapshot/.windows/.mac/.TemporaryItems/.DS_Store
deleted 9y ago
[deleted]
gnufx 9y ago
On the other hand, I've run multi-hundred-node HPC stateless HPC clusters long term off an (rpm-based) NFS root with little trouble once I could get the cluster vendors out of the picture. Places like the Tri-Labs appear to -- or have done? -- with larger systems, even if they need something like https://computation.llnl.gov/projects/spindle https://computation.llnl.gov/projects/spindle at scale. What flattened NFS wasn't executables but i/o of data files, BLAST being the worst offender; staging those in array jobs was definitely worthwhile. I strongly prefer the trades-off of stateless systems in that context, though I might prefer to do it off a parallel filesystem, and clearly your kilometreage varies. [From experience ~10 years ago, I'd say avoid GPFS more than NFS, but doubtless it's improved, and I hold no brief for NFS.]

lmm 9y ago

I've found that even for unix-unix, samba is just more reliable with more reasonable error behaviour.

WHoWHo 9y ago
To me samba has just too much magic, especially when mixing unix, OSX and windows clients.
- pnutjam 9y ago
  magic is just technology we don't understand. I use cifs over nfs as well.
  - WHoWHo 9y ago
    With excel locking, os x trashes, etc.? What good configuration guide would you suggest for a mixed Linux, OS X and Windows environment with MS Office? This magic only solves have the problems fruit:resource = xattr fruit:metadata = netatalk fruit:locking = none fruit:encoding = private delete veto files = yes veto files = /.snapshot/.windows/.mac/.TemporaryItems/.DS_Store

deleted 9y ago

[deleted]

gnufx 9y ago

On the other hand, I've run multi-hundred-node HPC stateless HPC clusters long term off an (rpm-based) NFS root with little trouble once I could get the cluster vendors out of the picture. Places like the Tri-Labs appear to -- or have done? -- with larger systems, even if they need something like https://computation.llnl.gov/projects/spindle https://computation.llnl.gov/projects/spindle at scale. What flattened NFS wasn't executables but i/o of data files, BLAST being the worst offender; staging those in array jobs was definitely worthwhile. I strongly prefer the trades-off of stateless systems in that context, though I might prefer to do it off a parallel filesystem, and clearly your kilometreage varies. [From experience ~10 years ago, I'd say avoid GPFS more than NFS, but doubtless it's improved, and I hold no brief for NFS.]