3 ms·

I have really mixed feelings about things like NFS, remote desktop, etc. The idea of having everything remote to save resources (or for other reasons) does soun

I have really mixed feelings about things like NFS, remote desktop, etc. The idea of having everything remote to save resources (or for other reasons) does sound really appealing in theory, and, when it works, is truly great. However in practice it's really hard to make these things be worth it, because of latency. E.g. for network block storage and for NFS the performance is usually abysmal compared to even a relatively cheap modern SSD in terms of latency, and many applications now expect a low latency file system, and perform really poorly otherwise.

zh3 1y ago
Fairly obviously a 1Gbps network is not going to compete with 5Gbps SATA or 20Gbps NVME. Having said that, for real performance we load stuff over the network into local RAM and then generally run from that (faster than all other options). On the internal network the server also has a large RAM disk shared over NFS/SMB, and the performance PC's have plenty of RAM - so really it's a tradeoff, and the optimum is going to depend on how the system is used.
- Palomides 1y ago
  want to emphasize, for those who haven't been following, a nice used 25Gb ethernet card is like $25 now
  - zokier 1y ago
    But how much is a 25GbE (or 40GbE) switch?
    - Palomides 1y ago
      maybe around $400 for 25Gbe depending on your noise and power tolerance, and 40Gbe is dirt cheap now if you only have two or three devices that need a fast connection you can just do point to point, of course
    - bombcar 1y ago
      10gb is cheap as free → CRS304-4XG-IN If I needed more than that, I’d probably do a direct link.
heavyset_go 1y ago
I can saturate both a 1 and 2.5 Gbps links with WireGuard encrypted NFSv4 on thin clients that are relatively old. I also use it for shared storage for my cluster and NAS, and I don't think NFS itself has ever been the bottleneck. Latency-wise, the overhead is negligible on via LAN, though can be noticeable when doing big builds or running VMs.

zh3 1y ago

Fairly obviously a 1Gbps network is not going to compete with 5Gbps SATA or 20Gbps NVME. Having said that, for real performance we load stuff over the network into local RAM and then generally run from that (faster than all other options). On the internal network the server also has a large RAM disk shared over NFS/SMB, and the performance PC's have plenty of RAM - so really it's a tradeoff, and the optimum is going to depend on how the system is used.

Palomides 1y ago
want to emphasize, for those who haven't been following, a nice used 25Gb ethernet card is like $25 now
- zokier 1y ago
  But how much is a 25GbE (or 40GbE) switch?
  - Palomides 1y ago
    maybe around $400 for 25Gbe depending on your noise and power tolerance, and 40Gbe is dirt cheap now if you only have two or three devices that need a fast connection you can just do point to point, of course
  - bombcar 1y ago
    10gb is cheap as free → CRS304-4XG-IN If I needed more than that, I’d probably do a direct link.

heavyset_go 1y ago

I can saturate both a 1 and 2.5 Gbps links with WireGuard encrypted NFSv4 on thin clients that are relatively old. I also use it for shared storage for my cluster and NAS, and I don't think NFS itself has ever been the bottleneck. Latency-wise, the overhead is negligible on via LAN, though can be noticeable when doing big builds or running VMs.