3 ms·
If you have high bandwidth, you can do many of the same things as with shared memory. HPC users have historically used message passing on moderately communicati
by fulafel 10y ago
If you have high bandwidth, you can do many of the same things as with shared memory. HPC users have historically used message passing on moderately communication heavy codes.
True, it has not been used for "general purpouse computing" but historically shared memory multiprocessors hadn't been either. We are just now starting to see things like parallelism in web browsers, 10-20 years after shared memory MP became common on desktop. Software is just slow to change.
- Sean1708 10y agoHPC is usually a balancing act between the performance you gain by adding another node and the time you lose passing messages around. You can't just keep adding nodes without introducing some form of shared memory because you will very quickly become limited by your bandwidth.
- jabl 10y agoWhat is shared memory, if not message passing (with some particular semantics) implemented at the hardware level? I.e. messages == cache lines, semantics == cache coherency protocol. Particularly so when you go to (nowadays) multi-socket systems, where everything is NUMA. There is a smallish sub-segment of the HPC market where customers really want large CC-NUMA systems, AFAIK SGI (whoever owns them these days, HPE?) is more or less the only supplier. But the vast majority of HPC is clusters using message passing. If I were to choose some awfully general statement about HPC, rather than yours I'd choose "supercomputers are not tools for solving small problems faster, but rather tools that make solving larger problems possible". I.e. the entire weak vs. strong scaling thing etc.
- Sean1708 10y agoTo be honest, yours is probably the far better statement.