6 ms·

Message passing is usually a "better" method of IPC anyway. The situations that genuinely require shared memory are rare.

• deleted 14y ago
  [deleted]
• scott_s 14y ago
  Perhaps in your line of work, but in scientific computing, it's quite common. Scientific codes have parallelism at multiple levels, but loop-level parallelism is extremely common. Simply, you have large vectors or matrices that are operated on by a loop. The iterations of the loop are (mostly) independent, so you can use multiple threads to do them in parallel. Assuming shared memory, of course; the communication cost of transferring the vectors or matrices to another process over typical IPC mechanisms would kill the benefit of the parallelism. OpenMP (http://openmp.org/wp/ http://openmp.org/wp/) for C, C++ and Fortran is the most common means of exploiting such parallelism in the high performance computing world.
  • montecarl 14y ago
    I'm curious about what field of scientific computing you work in. I am only familiar with MPI and hybrid MPI/OpenMP codes. I haven't come across any OpenMP only programs ever in chemistry or physics. If you have a loop with independent iterations then you can use message passing or shared memory to approach your problem (as well as loop vectorization). The main problem with message passing is that for very large datasets you can run into memory limitations.
    • scott_s 14y ago
      I don't, really. I do system software for high performance and distributed computing. The scientific applications are given to me, and are largely benchmarks. (Although, now, I don't do much actual HPC, but high-throughput, low-latency stream processing.) I have a friend who has a galaxy simulation that uses OpenMP but not MPI. The reason is simple: she doesn't have the expertise to make it distributed. Slapping a few OpenMP directives on the most expensive loops is easier than figuring out how to make it distributed using message passing. How much parallelism you extract out of a program is often a function of how much time and effort you can invest into it. Some people get "good enough" performance improvements by scaling on a single node.
  • wtallis 14y ago
    It's worth pointing out that those algorithms are also generally the ones that most benefit from being ported to a GPU, which removes the need for a shared-memory mechanism in the host-side language.
    • scott_s 14y ago
      Yes, if the latency of transferring the data and results to and back from GPU memory is low compared to the speedup.