3 ms·

Infiniband used to be great but in an era of 40 or 100 gig ethernet it's not particularly special anymore.

• DannyBee 8y ago
  Right. As https://en.wikipedia.org/wiki/InfiniBand#cite_note-20 https://en.wikipedia.org/wiki/InfiniBand#cite_note-20 says "2009: of the top 500 supercomputers in the world, Gigabit Ethernet is the internal interconnect technology in 259 installations, compared with 181 using InfiniBand.[20]"
  • gnufx 8y ago
    There are (at least) two parameters that are relevant for the fabric, and latency is the critical one for HPC. Have you seen a materials science computation, of the sort that typically takes much of the time on HPC systems, running on 1GbE and SDR IB, even on three nodes? It sometimes happened on our system by mistake in the 2009 era. HPL is simply not a good measure of the utility of a system for HPC, which generally accepted amongst HPC people, who deserve to be listened to. There's a good reason you don't see petascale DFT calculations on one of these "cloud provider"s which comprise so much of the top 500. (That might work on Azure, or something else with an Infiniband-ish fabric, but I've still not seen it.) You can, and to some extent should, model network effects with systems like Simgrid or Dimemas.
  • InclinedPlane 8y ago
    And that was almost a decade ago.