3 ms·

No amount of transistors can make shared memory monoliths fast on multicore systems, because the problem is in architecture unfriendly to such systems, not in h

No amount of transistors can make shared memory monoliths fast on multicore systems, because the problem is in architecture unfriendly to such systems, not in hardware. Monoliths can be fast if they drop shared memory and use actor model, not atomics and mutexes and programming with shared memory. But then it would no longer make them monoliths. Basically what makes monoliths slow on multicore systems is also the thing that makes them not microkernels. I mean if you split everything into actors, why even run all of them in kernel? Group them into isolated OS processes.

gpderetta 7y ago
And yet the cell and the transputer have been relegated to the annals of computer history while NUMAcc is king.
qznc 7y ago
Synchronisation between two cores is fastest with hardware support, so shared memory with cache coherence protocol. Cache coherence protocol are all designed to keep everybody in sync. So if you scale to thousands of cores, you better sync selectively and application specific. Thus it makes sense to use message passing (actor model). It is unclear where the it makes sense to switch. Somewhere between 2 and 1000. Hybrids are also an option. Supercomputers often use OpenMP (shared memory) and MPI (message passing) in a two level way.
- adev_ 7y ago
  > Hybrids are also an option. Supercomputers often use OpenMP (shared memory) and MPI (message passing) in a two level way. Not some, almost all supercomputer nowadays use an hybrid model. For three main reasons: - Memory is a precious resource and separated process space + messaging is more memory hungry than a shared hybrid process space - Hybrid machines with CPU + GPU on every node are everywhere and they requires a shared memory model. - Debugging a pure MPI model at scale is a huge pain in the ass that take some of the most complex debugger in the planet.

gpderetta 7y ago

And yet the cell and the transputer have been relegated to the annals of computer history while NUMAcc is king.

qznc 7y ago

Synchronisation between two cores is fastest with hardware support, so shared memory with cache coherence protocol. Cache coherence protocol are all designed to keep everybody in sync. So if you scale to thousands of cores, you better sync selectively and application specific. Thus it makes sense to use message passing (actor model). It is unclear where the it makes sense to switch. Somewhere between 2 and 1000. Hybrids are also an option. Supercomputers often use OpenMP (shared memory) and MPI (message passing) in a two level way.

adev_ 7y ago
> Hybrids are also an option. Supercomputers often use OpenMP (shared memory) and MPI (message passing) in a two level way. Not some, almost all supercomputer nowadays use an hybrid model. For three main reasons: - Memory is a precious resource and separated process space + messaging is more memory hungry than a shared hybrid process space - Hybrid machines with CPU + GPU on every node are everywhere and they requires a shared memory model. - Debugging a pure MPI model at scale is a huge pain in the ass that take some of the most complex debugger in the planet.