> Hybrids are also an option. Supercomputers often use OpenMP (shared memory) and MPI (message passing) in a two level way. Not some, almost all supercomputer

> Hybrids are also an option. Supercomputers often use OpenMP (shared memory) and MPI (message passing) in a two level way. Not some, almost all supercomputer nowadays use an hybrid model. For three main reasons: - Memory is a precious resource and separated process space + messaging is more memory hungry than a shared hybrid process space - Hybrid machines with CPU + GPU on every node are everywhere and they requires a shared memory model. - Debugging a pure MPI model at scale is a huge pain in the ass that take some of the most complex debugger in the planet.