3 ms·
It's a shame you can't get a cable to link computers via PCIe (supports cables over 3m[0]). Multiple x16 connections that would lead to some decent bandwidth wh
by velox_io 9y ago
It's a shame you can't get a cable to link computers via PCIe (supports cables over 3m[0]). Multiple x16 connections that would lead to some decent bandwidth while being low-latency, without the network overhead or shelling out for high-end switches.
You can skip halfway.
[0]https://www.youtube.com/watch?v=q5xvwPa3r7M https://www.youtube.com/watch?v=q5xvwPa3r7M
- semi-extrinsic 9y agoUsing dual port adapters, you can do a three-node Infiniband ring with no routers involved and get full speed. Dual port FDR (40 Gbit/s) cards are about $150 on Ebay nowadays. If you've bought the hardware to warrant needing such a setup (i.e. 3 dual-socket Xeon servers), that's a very negligible cost.
- wmf 9y agoPCIe fabrics exist but they're not as great as you might expect. Because of the slow 8 Gbps lane speed the cables are enormous and the switches support very few ports. And the DMA engine may not give you the performance you expect if it doesn't have NIC-like features. https://www.broadcom.com/applications/datacenter-networking/expressfabric https://www.broadcom.com/applications/datacenter-networking/...