> Some supercomputers use partially shared memory, but I'm not familiar with that space. Some supercomputers do have some shared memory architecture, but a who

> Some supercomputers use partially shared memory, but I'm not familiar with that space. Some supercomputers do have some shared memory architecture, but a whole lot more use Message Passing Interface (MPI) for distributed memory architecture. Shared memory starts to make less sense when your data can be terabytes or larger in size. It is a lot more scalable to just avoid a shared memory architecture and assume a distributed memory one. It becomes easier to program assuming that each thread just does not have access to the entire data set and has to send data back and forth between threads (pass messages).