4 ms·
Distributed computation with message passing (and RDMA) is the essence of HPC systems. SGI systems supported multi-node Linux single system images up to ~1024
by gnufx 5y ago
Distributed computation with message passing (and RDMA) is the essence of HPC systems. SGI systems supported multi-node Linux single system images up to ~1024 cores a fair few years ago, but they depend on a coherent interconnect (NUMAlink, originally from the MPIS-based systems under Irix).
However, you don't ignore the distributed nature of even single HPC nodes unless you want to risk perhaps an order of magnitude performance loss. SMP these days doesn't stand for Symmetric Multi-Processing.
- zozbot234 5y agoDistributed shared memory is feasible in theory even via being provided in-software by the OS. You're right that this would not change the physical reality of message passing, but it would allow a single multi-processor application code to operate seamlessly using either shared memory on a single node, or distributed memory on a large cluster.
- gnufx 5y agoI talk about the practice in HPC, not theory, and this stuff is literally standard (remote memory of various types and the same thing running the same, modulo performance and resources, on a 32-core node as on one core each of 32 nodes). However, you still need to consider network non-uniformity at levels from at least NUMA nodes up, at least if you want performance in general.