4 ms·

The difference, as I understand it, is actually pretty simple... This suggests to me that the confusion is due to the synonymy of the terms in common usage. Wi

by slunk 14y ago

The difference, as I understand it, is actually pretty simple... This suggests to me that the confusion is due to the synonymy of the terms in common usage. Without making any claims about the quality of this presentation specifically, I wonder why anyone would insist on using them and inevitably have to lecture perfectly smart people about why they don't mean the same thing...

virtuabhi 14y ago
I listened to his presentation. IMHO the presenter laid too much stress on parallelism and concurrency being different, but never explained the difference clearly and succinctly. But from the presenter's side, I can see that he needs to sell his product(golang). Therefore he assumed that - these definitions are often confused, it is a big issue, and go is 42.
- seanmcdirmid 14y ago
  These terms have been confused since the early 90s at least. It is one of the first things we systems people learn: concurrency is doing multiple things at once because you must, parallelism is doing multiple things at once because you can. Infrastructure and techniques that support concurrency (e.g. actors) are almost always totally unsuitable to parallelism (e.g. SIMD ala GPUs), and the other way around. Likewise, skills don't transfer very well between the two fields. So when someone talks about this technology being useful to your use case, and they've confused the terms, its very annoying (you should use actors to get more parallelism, WTF????).
  - greatzebu 14y ago
    You're taking a pretty narrow view of parallel computing. For example, there are plenty of systems that use essentially actor models for large-scale parallel computation, e.g. Charm++ and HPX.
    - seanmcdirmid 14y ago
      Isn't Charm++ infrastructure comparable to MPI? In that case, its not clear what the goals are. I think it was during the MPI-phase that we began to realize that we had two very different problems on our hands. Fine-grained message passing has been obsolete in the parallel field at least for awhile now.
      - nivertech 14y ago
        Do you say that MPI is obsolete? I thought most legacy codes in HPC based on MPI?
        seanmcdirmid 14y ago
        You know, its rude to say something is obsolete in this field, so we just say...there are other choices and hope the hint is taken. Nothing ever dies quickly, it just fades away off into the sunset. There is a lot of legacy code running with MPI for sure, but the big data and almost big data trends are clear, and CUDA has been absolutely disruptive in the scientific computing field (though some people are beginning to use MapReduce here also).
        nivertech 14y ago
        yeah, I always map CUDA blocks to racks and CUDA threads to blades ;) GPUs, Xeon Phi, FPGAs and other accelerators still need somebody managing them. Nobody said it should be MPI, but there are still many hybrid architectures with MPI handling distribution and actual compute done using OpenMP or accelerators. In my system I use Erlang/OTP to handle distribution and concurrency and OpenCL for data-parallel compute.
        seanmcdirmid 14y ago
        > yeah, I always map CUDA blocks to racks and CUDA threads to blades ;) Sounds like Jeff Dean. > Nobody said it should be MPI, but there are still many hybrid architectures with MPI handling distribution and actual compute done using OpenMP or accelerators. MPI or even RPC works fine as a control mechanism, just not as a critical performance-sensitive abstraction, where we care about the width and speed of the pipe, and MPI is nothing like a pipe! > In my system I use Erlang/OTP to handle distribution and concurrency and OpenCL for data-parallel compute. This is quite reasonable. Once one understands the difference between concurrency and parallelism, they can pick appropriate tools to deal with each. As long as they confuse the issues, they'll make bad choices.
- dsymonds 14y ago
  Rob's been trying to get people to understand the difference since the early 1990s (Alef, etc.). It's not something new, nor unique to Go. In fact Go is a response to the desire for concurrency; it's not that Go popped out of nothing and then Rob decided that he consequently needed to "sell" it by pushing concurrency.
  - nivertech 14y ago
    I think roots of Go in Limbo programming language, running on Dis VM on Inferno OS. Inferno itself is based on Plan 9 ideas. Essentially Go just Limbo, but native instead of VM and with more conventional syntax. Inferno also had built-in distribution and hot code loading: I.e. you could load module from remote location and run a function locally or you could execute function remotely as RPC. Erlang/OTP never had this level of integration as Inferno, but now we have some efforts to running Erlang on bar metal, I.e. Erlang-on-Xen.

by slunk 14y ago

virtuabhi 14y ago

I listened to his presentation. IMHO the presenter laid too much stress on parallelism and concurrency being different, but never explained the difference clearly and succinctly. But from the presenter's side, I can see that he needs to sell his product(golang). Therefore he assumed that - these definitions are often confused, it is a big issue, and go is 42.

seanmcdirmid 14y ago
These terms have been confused since the early 90s at least. It is one of the first things we systems people learn: concurrency is doing multiple things at once because you must, parallelism is doing multiple things at once because you can. Infrastructure and techniques that support concurrency (e.g. actors) are almost always totally unsuitable to parallelism (e.g. SIMD ala GPUs), and the other way around. Likewise, skills don't transfer very well between the two fields. So when someone talks about this technology being useful to your use case, and they've confused the terms, its very annoying (you should use actors to get more parallelism, WTF????).
- greatzebu 14y ago
  You're taking a pretty narrow view of parallel computing. For example, there are plenty of systems that use essentially actor models for large-scale parallel computation, e.g. Charm++ and HPX.
  - seanmcdirmid 14y ago
    Isn't Charm++ infrastructure comparable to MPI? In that case, its not clear what the goals are. I think it was during the MPI-phase that we began to realize that we had two very different problems on our hands. Fine-grained message passing has been obsolete in the parallel field at least for awhile now.
    - nivertech 14y ago
      Do you say that MPI is obsolete? I thought most legacy codes in HPC based on MPI?
      - seanmcdirmid 14y ago
        You know, its rude to say something is obsolete in this field, so we just say...there are other choices and hope the hint is taken. Nothing ever dies quickly, it just fades away off into the sunset. There is a lot of legacy code running with MPI for sure, but the big data and almost big data trends are clear, and CUDA has been absolutely disruptive in the scientific computing field (though some people are beginning to use MapReduce here also).
        nivertech 14y ago
        yeah, I always map CUDA blocks to racks and CUDA threads to blades ;) GPUs, Xeon Phi, FPGAs and other accelerators still need somebody managing them. Nobody said it should be MPI, but there are still many hybrid architectures with MPI handling distribution and actual compute done using OpenMP or accelerators. In my system I use Erlang/OTP to handle distribution and concurrency and OpenCL for data-parallel compute.
        seanmcdirmid 14y ago
        > yeah, I always map CUDA blocks to racks and CUDA threads to blades ;) Sounds like Jeff Dean. > Nobody said it should be MPI, but there are still many hybrid architectures with MPI handling distribution and actual compute done using OpenMP or accelerators. MPI or even RPC works fine as a control mechanism, just not as a critical performance-sensitive abstraction, where we care about the width and speed of the pipe, and MPI is nothing like a pipe! > In my system I use Erlang/OTP to handle distribution and concurrency and OpenCL for data-parallel compute. This is quite reasonable. Once one understands the difference between concurrency and parallelism, they can pick appropriate tools to deal with each. As long as they confuse the issues, they'll make bad choices.
dsymonds 14y ago
Rob's been trying to get people to understand the difference since the early 1990s (Alef, etc.). It's not something new, nor unique to Go. In fact Go is a response to the desire for concurrency; it's not that Go popped out of nothing and then Rob decided that he consequently needed to "sell" it by pushing concurrency.
- nivertech 14y ago
  I think roots of Go in Limbo programming language, running on Dis VM on Inferno OS. Inferno itself is based on Plan 9 ideas. Essentially Go just Limbo, but native instead of VM and with more conventional syntax. Inferno also had built-in distribution and hot code loading: I.e. you could load module from remote location and run a function locally or you could execute function remotely as RPC. Erlang/OTP never had this level of integration as Inferno, but now we have some efforts to running Erlang on bar metal, I.e. Erlang-on-Xen.