5 ms·

Every time this article or a similar subject is brought up, the thread quickly fills up with people all giving different definitions of concurrency and parallel

Every time this article or a similar subject is brought up, the thread quickly fills up with people all giving different definitions of concurrency and parallelism. Nobody can agree on them so let's just assume they mean the same thing and move to more interesting discussions, shall we?

seanmcdirmid 11y ago
> Nobody can agree on them so let's just assume they mean the same thing and move to more interesting discussions, shall we? Concurrency and parallelism need different abstractions; try using actors for parallelism or MPI for concurrency! If they mean the same thing, how do we distinguish between them as is absolutely necessary?
- stefantalpalaru 11y ago
  > try using actors for parallelism Erlang > or MPI for concurrency Go uses channels as a message passing interface for the "composition of independently executing processes"
  - seanmcdirmid 11y ago
    Erlang is about concurrency, not parallelism; ditto for Go Channels. MPI was designed for parallelism in the HPC sector, not concurrency for distributed servers. Saying concurrency and parallelism are equivalent basically invalidates all design decisions made by these systems.
    - stefantalpalaru 11y ago
      > Erlang is about concurrency, not parallelism They make full use of all CPU cores since they implemented SMP support: http://erlang.2086793.n4.nabble.com/Some-facts-about-Erlang-and-SMP-td2108770.html http://erlang.2086793.n4.nabble.com/Some-facts-about-Erlang-... > ditto for Go Channels Go has a user-space M:N scheduler that distributes its "goroutines" among multiple OS threads and therefore multiple cores.
    - rdtsc 11y ago
      > Erlang is about concurrency, not parallelism; Erlang is about both. Those are not either or. Concurrency sits at a higher level of abstraction -- at the problem domain. "I'll create an actor for this request, another actor for that, maybe another one for monitoring". These are concurrent units. Parallelism happens if my machine has more than one CPU and these concurrency units will actaully execute at the same time. Or maybe I have a single CPU only. I'll have not parallelism. They'll be sequenced and scheduled in and out of the single CPU in some small or larger time slices. But they'll not execute in parallel. So maybe I am running my machine in 1989 (Erlang was doing work back then too) with only a single CPU. I still want to create many concurrent units of work. Or maybe I am running in 2015 and I have 64 core machine and I specifically choose Erlang because I expect to get lots of parallelism for cheap as my concurrent units are largely independent of each other.
      - seanmcdirmid 11y ago
        It really depends on what you are doing. If you are processing requests concurrently, parallelism is a nice benefit, but the requirement is to do things at the same time correctly with resource contentions involved. Parallelism is more about breaking a job into bits so it goes faster...e.g. what is done in the HPC and Big Data worlds with MPI, CUDA, MapReduce, ... Concurrency is necessary property of the work as defined, parallelism is an opportunistic property. Confusing the two concepts can lead to people trying to do HPC work with Erlang. If you are working on an HPC workload, you wouldn't choose Erlang, but you might get a happy speed up for a standard distributed application. Parallelism is strictly about performance, concurrency is about correctness. When you go off and do a HPC or big data program, you might play with MPI, CUDA, MapReduce, or maybe just roll something yourself. No one goes...we should do that in Erlang! Likewise, when you go off and do a concurrent distributed system, Actors make a lot of sense given the concurrency requirements involved (who would think of using CUDA for that?). Perhaps the problem is not about concurrency and parallelism, but different kinds of computations.
        Xixi 11y ago
        It's a semantic debat, but you seem to redefine parallelism as HPC. I respectfully disagree, parallelism is not necessarily about getting faster, it is really about actually executing computations simultaneously. Erlang uses parallelism for fault-tolerance purposes, because you need several physical machines running in parallel on different sites to achieve outstanding fault-tolerance. MPI/CUDA/MapReduce uses parallelism for performance reasons. I do not know if it is true as I couldn't find the source, but I remember reading that aircrafts run all computations in parallel on three different architectures (Power and two others), using three different algorithms, to ensure that the results are not compromised either by a faulty algorithm, or a CPU bug. In that case parallelism is used for safety.
rdtsc 11y ago
> so let's just assume they mean the same thing and move to more interesting discussions, shall we? I don't think it is that complicated. Concurrency is a property of the algorithm or problem space. Things can happen independent of each other. Two or more client requests can read a file. Logically they don't have to wait for each other. There is no sequential dependence. These could be represented in code as two threads, two futures, promises, Goroutines, Tasks, callback chains, actors etc <insert favorite concurrency primitive>. Parallelism is a property of the execution environment. "Can the concurrent parts of the program execute at the same time?" The machine happens to have 2 CPU cores and two requests arriving at the same time can execute at the some time. Or it has only one CPU and even though they can execute at the same time, they don't. One might be blocked waiting for another or they might be preempted by the scheduler every milliseconds, execute a bit, then switch to another. EDIT: Here is Joe Armstrong's interpretation. Which is similar to this and includes a cute diagram with coffee machines: http://joearms.github.io/2013/04/05/concurrent-and-parallel-programming.html http://joearms.github.io/2013/04/05/concurrent-and-parallel-... He knows what he is talking about, he is the father of Erlang.
buttproblem 11y ago
While I agree with you, it is better to focus on practical things other than pedantic names, you can really notice the difference more when you look at the foundation of Go's language level concurrency paradigm. Concurrency in Go is based on Communicating Sequential Processes (CSP) by Hoare; this was originally published in 1978. Reading CSP you see all the ideas are abstractly formalized in math terms: there really is no notion of multiple streams of execution occurring at the same time (i.e., multiple hardware threads). So, a distinction is made between things cooperating (concurrency in their parlance) and things running at the same time (parallelism). To continue this idea, parallelism, again using their terms, really is an implementation of the more abstract concept of concurrency. That is, given a description of concurrent processes communicating you can run them in parallel (and reason about their correctness). If you do not make the distinction, regardless of the terms you use, you remove the layers of abstraction necessary to reason about complex systems (separation of high level concepts and their implementation).
ternaryoperator 11y ago
Furthermore, for the many conversations I've had on the topic (and I've had hundreds), I don't recall a single instance where someone used the wrong term and it caused the slightest bit of confusion.
bottled_poe 11y ago
Not sure if troll, but... Just because the majority cannot agree does not mean the terms are not defined. Concurrency and parallelism are quite distinct concepts (though not entirely) [1,2,3]. 1. http://en.wikipedia.org/wiki/Concurrent_computing#Introduction http://en.wikipedia.org/wiki/Concurrent_computing#Introducti... 2. http://docs.oracle.com/cd/E19455-01/806-5257/6je9h032b/index.html http://docs.oracle.com/cd/E19455-01/806-5257/6je9h032b/index... 3. https://wiki.haskell.org/Parallelism_vs._Concurrency https://wiki.haskell.org/Parallelism_vs._Concurrency
- zak_mc_kracken 11y ago
  > Just because the majority cannot agree does not mean the terms are not defined I never said that, just that a lot of people have different definitions for these words, all contradicting each other. So in effect, these words have become meaningless.
curryhoward 11y ago
> let's just assume they mean the same thing and move to more interesting discussions They don't mean the same thing, and that is a poisonous attitude to have. Parallelism is about performance. Concurrency is about synchronization. The difference might not matter to you, but for certain things (programming language design, library design, system architecture) it is paramount. One thing nearly everyone agrees on: a single-core processor cannot do parallelism, but it's perfectly capable of concurrency. How do you reconcile that with your view that they are the same?
- zak_mc_kracken 11y ago
  > The difference might not matter to you The difference matters very much to me, what is silly is that nobody agrees on which one is concurrency and which one is parallelism. In this thread alone, I found two people with a different definition from yours.
  - deleted 11y ago
    [deleted]

seanmcdirmid 11y ago

> Nobody can agree on them so let's just assume they mean the same thing and move to more interesting discussions, shall we? Concurrency and parallelism need different abstractions; try using actors for parallelism or MPI for concurrency! If they mean the same thing, how do we distinguish between them as is absolutely necessary?

stefantalpalaru 11y ago
> try using actors for parallelism Erlang > or MPI for concurrency Go uses channels as a message passing interface for the "composition of independently executing processes"
- seanmcdirmid 11y ago
  Erlang is about concurrency, not parallelism; ditto for Go Channels. MPI was designed for parallelism in the HPC sector, not concurrency for distributed servers. Saying concurrency and parallelism are equivalent basically invalidates all design decisions made by these systems.
  - stefantalpalaru 11y ago
    > Erlang is about concurrency, not parallelism They make full use of all CPU cores since they implemented SMP support: http://erlang.2086793.n4.nabble.com/Some-facts-about-Erlang-and-SMP-td2108770.html http://erlang.2086793.n4.nabble.com/Some-facts-about-Erlang-... > ditto for Go Channels Go has a user-space M:N scheduler that distributes its "goroutines" among multiple OS threads and therefore multiple cores.
  - rdtsc 11y ago
    > Erlang is about concurrency, not parallelism; Erlang is about both. Those are not either or. Concurrency sits at a higher level of abstraction -- at the problem domain. "I'll create an actor for this request, another actor for that, maybe another one for monitoring". These are concurrent units. Parallelism happens if my machine has more than one CPU and these concurrency units will actaully execute at the same time. Or maybe I have a single CPU only. I'll have not parallelism. They'll be sequenced and scheduled in and out of the single CPU in some small or larger time slices. But they'll not execute in parallel. So maybe I am running my machine in 1989 (Erlang was doing work back then too) with only a single CPU. I still want to create many concurrent units of work. Or maybe I am running in 2015 and I have 64 core machine and I specifically choose Erlang because I expect to get lots of parallelism for cheap as my concurrent units are largely independent of each other.
    - seanmcdirmid 11y ago
      It really depends on what you are doing. If you are processing requests concurrently, parallelism is a nice benefit, but the requirement is to do things at the same time correctly with resource contentions involved. Parallelism is more about breaking a job into bits so it goes faster...e.g. what is done in the HPC and Big Data worlds with MPI, CUDA, MapReduce, ... Concurrency is necessary property of the work as defined, parallelism is an opportunistic property. Confusing the two concepts can lead to people trying to do HPC work with Erlang. If you are working on an HPC workload, you wouldn't choose Erlang, but you might get a happy speed up for a standard distributed application. Parallelism is strictly about performance, concurrency is about correctness. When you go off and do a HPC or big data program, you might play with MPI, CUDA, MapReduce, or maybe just roll something yourself. No one goes...we should do that in Erlang! Likewise, when you go off and do a concurrent distributed system, Actors make a lot of sense given the concurrency requirements involved (who would think of using CUDA for that?). Perhaps the problem is not about concurrency and parallelism, but different kinds of computations.
      - Xixi 11y ago
        It's a semantic debat, but you seem to redefine parallelism as HPC. I respectfully disagree, parallelism is not necessarily about getting faster, it is really about actually executing computations simultaneously. Erlang uses parallelism for fault-tolerance purposes, because you need several physical machines running in parallel on different sites to achieve outstanding fault-tolerance. MPI/CUDA/MapReduce uses parallelism for performance reasons. I do not know if it is true as I couldn't find the source, but I remember reading that aircrafts run all computations in parallel on three different architectures (Power and two others), using three different algorithms, to ensure that the results are not compromised either by a faulty algorithm, or a CPU bug. In that case parallelism is used for safety.

rdtsc 11y ago

> so let's just assume they mean the same thing and move to more interesting discussions, shall we? I don't think it is that complicated. Concurrency is a property of the algorithm or problem space. Things can happen independent of each other. Two or more client requests can read a file. Logically they don't have to wait for each other. There is no sequential dependence. These could be represented in code as two threads, two futures, promises, Goroutines, Tasks, callback chains, actors etc <insert favorite concurrency primitive>. Parallelism is a property of the execution environment. "Can the concurrent parts of the program execute at the same time?" The machine happens to have 2 CPU cores and two requests arriving at the same time can execute at the some time. Or it has only one CPU and even though they can execute at the same time, they don't. One might be blocked waiting for another or they might be preempted by the scheduler every milliseconds, execute a bit, then switch to another. EDIT: Here is Joe Armstrong's interpretation. Which is similar to this and includes a cute diagram with coffee machines: http://joearms.github.io/2013/04/05/concurrent-and-parallel-programming.html http://joearms.github.io/2013/04/05/concurrent-and-parallel-... He knows what he is talking about, he is the father of Erlang.

buttproblem 11y ago

While I agree with you, it is better to focus on practical things other than pedantic names, you can really notice the difference more when you look at the foundation of Go's language level concurrency paradigm. Concurrency in Go is based on Communicating Sequential Processes (CSP) by Hoare; this was originally published in 1978. Reading CSP you see all the ideas are abstractly formalized in math terms: there really is no notion of multiple streams of execution occurring at the same time (i.e., multiple hardware threads). So, a distinction is made between things cooperating (concurrency in their parlance) and things running at the same time (parallelism). To continue this idea, parallelism, again using their terms, really is an implementation of the more abstract concept of concurrency. That is, given a description of concurrent processes communicating you can run them in parallel (and reason about their correctness). If you do not make the distinction, regardless of the terms you use, you remove the layers of abstraction necessary to reason about complex systems (separation of high level concepts and their implementation).

ternaryoperator 11y ago

Furthermore, for the many conversations I've had on the topic (and I've had hundreds), I don't recall a single instance where someone used the wrong term and it caused the slightest bit of confusion.

bottled_poe 11y ago

Not sure if troll, but... Just because the majority cannot agree does not mean the terms are not defined. Concurrency and parallelism are quite distinct concepts (though not entirely) [1,2,3]. 1. http://en.wikipedia.org/wiki/Concurrent_computing#Introduction http://en.wikipedia.org/wiki/Concurrent_computing#Introducti... 2. http://docs.oracle.com/cd/E19455-01/806-5257/6je9h032b/index.html http://docs.oracle.com/cd/E19455-01/806-5257/6je9h032b/index... 3. https://wiki.haskell.org/Parallelism_vs._Concurrency https://wiki.haskell.org/Parallelism_vs._Concurrency

zak_mc_kracken 11y ago
> Just because the majority cannot agree does not mean the terms are not defined I never said that, just that a lot of people have different definitions for these words, all contradicting each other. So in effect, these words have become meaningless.

curryhoward 11y ago

> let's just assume they mean the same thing and move to more interesting discussions They don't mean the same thing, and that is a poisonous attitude to have. Parallelism is about performance. Concurrency is about synchronization. The difference might not matter to you, but for certain things (programming language design, library design, system architecture) it is paramount. One thing nearly everyone agrees on: a single-core processor cannot do parallelism, but it's perfectly capable of concurrency. How do you reconcile that with your view that they are the same?

zak_mc_kracken 11y ago
> The difference might not matter to you The difference matters very much to me, what is silly is that nobody agrees on which one is concurrency and which one is parallelism. In this thread alone, I found two people with a different definition from yours.
- deleted 11y ago
  [deleted]