For a relatively modern cluster the standard is FDR Infiniband, which gives you 56 Gbit/s throughput. The latency for a small (< 32 kbytes) packet is less than

For a relatively modern cluster the standard is FDR Infiniband, which gives you 56 Gbit/s throughput. The latency for a small (< 32 kbytes) packet is less than 10 microseconds, down towards 1 microsecond for a 8 byte packet. Glenn Lockwood has a nice blogpost with much more detail: http://glennklockwood.blogspot.com/2013/05/fdr-infiniband-vs-dual-rail-qdr.html http://glennklockwood.blogspot.com/2013/05/fdr-infiniband-vs...