25Gb NICs are getting into the affordable range, but SFP28 modules are still very pricey in comparison. I’m a big fan of the Mellanox CX4 LX cards for their low

25Gb NICs are getting into the affordable range, but SFP28 modules are still very pricey in comparison. I’m a big fan of the Mellanox CX4 LX cards for their low power draw (11w max). OEM cards come cheap and can be crossflashed to generic Mellanox firmware. The bigger issue with 25G is that it’s well into the range where any OS’s default TCP settings won’t provide anywhere near line speed, and once you solve that it can expose other non-network bottlenecks. I have dual-SFP28 cards in both my workstation and NAS, both connected to the network via a10G Mikrotik switch and directly to each other via a 25G point-to-point link. Now after all that tuning I have a fast network but run into the SAS bus bottleneck for any file transfers exceeding the size of my app take write cache :p