4 ms·
He references the post "Parallelism /= Concurrency," which we previously discussed here: http://news.ycombinator.com/item?id=2457307 http://news.ycombinator.com
by scott_s 15y ago
He references the post "Parallelism /= Concurrency," which we previously discussed here: http://news.ycombinator.com/item?id=2457307 http://news.ycombinator.com/item?id=2457307
That was largely a semantic issue, which I responded with:
Parallelism: simultaneous calculations executing in the service of a single problem, usually with the goal of improved performance.
Concurrency: executions in the same time granularity, but not necessarily simultaneous. Also not necessarily in the service of the same problem, but some form of synchronization is required.
What I will add this time is that there are different kinds of parallelism, and I think he's considering "data parallelism" the real parallelism. I think this is a false distinction. What he calls "concurrency," I and others call either task parallelism (for distinct tasks that don't need to communicate much) or pipeline parallelism (for tasks that work on data in stages).
- jlouis 15y agoYep, you are right that I am missing out on the fact that there is different types of parallelism in the post. But it was getting too big of a post already :/ You can also classify parallelism. An example is to compare that you annotate parallel execution on the data structure (this is a parallel array) or on the control-flow (when operating on this array, do it in parallel). Another is if the parallel distinction is largely dynamic (worker pool) or static (we know we have 16 cores so the optimal communication structure assuming a double torus pairing configuration is ...). Yet another is to look at the data themselves. Are they a flat array or a nested tree structure (Nested Data Level parallelism is cool btw). And finally, to stop all the examples, we could look at declaratively making the code parallel and doing so explicitly yourself from primitives (think rolling your own database versus writing queries in SQL for an existing one - and hope the query optimizer can help). The main point is that Erlang has taken a choice here. And that choice only covers some uses of what people may want to do. For the other things, people tend to use Erlang as a controller and then write their FPGA code and run it separately. This is what the original telephony switches were doing with Erlang.
- philwelch 15y agoParallelism is an algorithms problem: given n processors, design an algorithm that runs faster as n increases. Concurrency is a systems problem: design a system that safely runs n separate but interacting contexts of execution. Erlang's solution to concurrency--no communication between processes other than asymmetric message passing, no shared state other than the process registry--is a great solution to concurrency in terms of safety and distributon across multiple hosts, but it provides limited help to parallelism. Erlang's reputation as a great language for concurrent programming on multicore processors is largely by accident--it has a great concurrency model for networks of distributed machines, and almost any concurrency model is better than mutexes and threads, but on multicore machines, Erlang wins only by default.
- scott_s 15y agono communication between processes other than asymmetric message passing, no shared state other than the process registry--is a great solution to concurrency in terms of safety and distributon across multiple hosts, but it provides limited help to parallelism. That's the same model of executions that MPI supports, and the most high-performance, parallel programs ever written have been written with MPI - scaling to thousands of processors across hundreds of nodes. I feel that the people who are trying to define "parallelism" and "concurrency" are clashing with existing definitions from the already established high-performance computing community.