Based on a very shallow reading of Infiniband, I think some of the interconnects of supercomputers are essentially DMA-based rather than network-based. So it's

Based on a very shallow reading of Infiniband, I think some of the interconnects of supercomputers are essentially DMA-based rather than network-based. So it's like writing something massively multithreaded (your barrier is one step above shared memory) versus multihosted. I could be completely 100% wrong.