Generally Infiniband [1] or Omni-Path [2] interconnects between the compute nodes - extremely high throughput and low latency communication on the order of 100

Generally Infiniband [1] or Omni-Path [2] interconnects between the compute nodes - extremely high throughput and low latency communication on the order of 100 Gb/s and 1 microsecond. These problems aren't trillions of completely independent operations which may work for cloud/distributed architecture. For these types of calculations compute node 6523 needs to know the data coming from compute node 2447 in near-realtime to continue working. https://en.wikipedia.org/wiki/InfiniBand https://en.wikipedia.org/wiki/InfiniBand https://en.wikipedia.org/wiki/Omni-Path https://en.wikipedia.org/wiki/Omni-Path