4 ms·

APIs sorted by complexity (from most hard to learn to easier to learn): * OpenCL / CUDA * Pthreads * MPI * TBB * OpenMP

scott_s 16y ago
I consider message passing programs harder to learn and develop scalable applications for than shared memory. While the errors in a shared memory program (this includes Pthreads, TBB, OpenMP - any programming model that uses multithreading in a shared address space) can be more subtle, you don't have to have as deep an understanding of the runtime behavior of your algorithms. It's easier to rely on shared memory in hardware than to use explicit send/receive on all of your data.
- nivertech 16y ago
  I disagree, in MPI you don't deal with locks and race conditions. It's like saying, that Erlang programming is harder than multi-threading in C/C++. I'll take Erlang over Pthreads any day. MPI applications can be a little bit harder algorithmically, but a concept of Collective Communications (i.e. send/recv/scatter/gather/bcast/reduce) is easier to understand than threading and locking. The whole concept of Shared/Distributed memory models doesn't mutter in this case.
  - scott_s 16y ago
    There's two issues: debugging and scaling. I agree that multithreading at the Pthread level is harder to debug. But I think it's easier to design a program that will scale using Pthreads than MPI because with one you rely on the hardware to distribute data, and with the other you have to do all of that yourself.
    - nivertech 16y ago
      According to this logic MPI for Shared Memory will be easier to understand than OpenMP for clusters.
      - scott_s 16y ago
        You'll have to explain that, I don't follow. I haven't brought up OpenMP since we're both in agreement it's easier to develop for than MPI. My point, in case it is not clear: having to explicitly send and receive data and achieve scalability is inherently more difficult than relying on shared memory. Getting a working program may not be difficult relying only on message passing. But getting a working program that scales requires a deep understanding of the data needs of your algorithms. Of course, message passing programs have the potential for higher performance because they're not limited to a single node. But realizing that potential is harder. And I'm not including debugging, where I agree that shared memory is rife with problems.

I consider message passing programs harder to learn and develop scalable applications for than shared memory. While the errors in a shared memory program (this includes Pthreads, TBB, OpenMP - any programming model that uses multithreading in a shared address space) can be more subtle, you don't have to have as deep an understanding of the runtime behavior of your algorithms. It's easier to rely on shared memory in hardware than to use explicit send/receive on all of your data.

nivertech 16y ago
I disagree, in MPI you don't deal with locks and race conditions. It's like saying, that Erlang programming is harder than multi-threading in C/C++. I'll take Erlang over Pthreads any day. MPI applications can be a little bit harder algorithmically, but a concept of Collective Communications (i.e. send/recv/scatter/gather/bcast/reduce) is easier to understand than threading and locking. The whole concept of Shared/Distributed memory models doesn't mutter in this case.
- scott_s 16y ago
  There's two issues: debugging and scaling. I agree that multithreading at the Pthread level is harder to debug. But I think it's easier to design a program that will scale using Pthreads than MPI because with one you rely on the hardware to distribute data, and with the other you have to do all of that yourself.
  - nivertech 16y ago
    According to this logic MPI for Shared Memory will be easier to understand than OpenMP for clusters.
    - scott_s 16y ago
      You'll have to explain that, I don't follow. I haven't brought up OpenMP since we're both in agreement it's easier to develop for than MPI. My point, in case it is not clear: having to explicitly send and receive data and achieve scalability is inherently more difficult than relying on shared memory. Getting a working program may not be difficult relying only on message passing. But getting a working program that scales requires a deep understanding of the data needs of your algorithms. Of course, message passing programs have the potential for higher performance because they're not limited to a single node. But realizing that potential is harder. And I'm not including debugging, where I agree that shared memory is rife with problems.