3 ms·
Synchronisation between two cores is fastest with hardware support, so shared memory with cache coherence protocol. Cache coherence protocol are all designed t
by qznc 7y ago
Synchronisation between two cores is fastest with hardware support, so shared memory with cache coherence protocol.
Cache coherence protocol are all designed to keep everybody in sync. So if you scale to thousands of cores, you better sync selectively and application specific. Thus it makes sense to use message passing (actor model).
It is unclear where the it makes sense to switch. Somewhere between 2 and 1000.
Hybrids are also an option. Supercomputers often use OpenMP (shared memory) and MPI (message passing) in a two level way.
- adev_ 7y ago> Hybrids are also an option. Supercomputers often use OpenMP (shared memory) and MPI (message passing) in a two level way. Not some, almost all supercomputer nowadays use an hybrid model. For three main reasons: - Memory is a precious resource and separated process space + messaging is more memory hungry than a shared hybrid process space - Hybrid machines with CPU + GPU on every node are everywhere and they requires a shared memory model. - Debugging a pure MPI model at scale is a huge pain in the ass that take some of the most complex debugger in the planet.