4 ms·

I don't think either definition really defines what concurrency and parallelism are really. Concurrency is the study of how processes interact (i.e. communicati

by 14113 12y ago

I don't think either definition really defines what concurrency and parallelism are really. Concurrency is the study of how processes interact (i.e. communication, locking etc) while parallelism is studying how to physically perform more things at once. In this way, concurrency and parallelism (almost) form two axes within which you can judge programs: You can have highly concurrent, but serial program (e.g. a browser, with different threads for networking, UI, etc), and highly parallel, but non-concurrent (or with little concurrency), like image processing, or other "embarrassingly parallel" problems.

scott_s 12y ago
What you call concurrency, I call concurrency control. But more often, I just call it synchronization. I am unaware of any widespread acceptance of your definition.
- 14113 12y ago
  I hate doing this, but both the wikipedia articles for concurrency and parallelism agree with me: Concurrency > In computer science, concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. Parallelism > Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). In my opinion there's really not that much more to concurrency that's hard, apart from safely organising the communication between concurrent processes/threads. It's then the opposite problem with parallelism; the difficulty is finding and implementing the physically separate execution of different threads of execution in a program, while (generally) the communication is seen as a concurrency problem.
  - scott_s 12y ago
    I don't think that definition of concurrency agrees with either of us. You defined concurrency as interaction itself, while Wikipedia defines it in terms of time and execution. I would agree with the Wikipedia definition if it changed "simultaneously" with "concurrently".
    - 14113 12y ago
      Hmm, that's a fair point. Thinking about it a bit more, I think my definitions fit closer with the idea of concurrent or parallel programs rather than the abstract ideas of concurrency or parallelism themselves. In any case, I completely disagree with your idea that parallel systems must be concurrent. Concurrency arises completely due to the interactions between separate threads of execution, whether they be executing at the same time on separate processors, or scheduled one after the other on a single processor. This is why we have various process calculii (csp, pi-calculus etc), and things such as session types, in order to try and reason about it and manage it. Parallelism assumes nothing about the interactions between processes, only that they are assumed to be running on separate hardware. As another poster mentioned, this fits fairly well with what the Haskell, Go, and general programming language research community considers to be the definitions of concurrency and parallelism. Look at any paper that focusses on concurrency, and you'll see it's about managing and making safe communication between separate processes. Look at a paper on parallelism, and it'll be on accelerating a parallelisable program on multicore or distributed hardware.
      - scott_s 12y ago
        As someone who works in parallel and distributed systems (which means I write and read papers on parallelism), I find your distinction an empty one. Actually getting good performance from parallelism requires careful consideration of the synchronization between the parallel parts - and in my work, the "careful consideration" is usually done by me. My goal is often to make abstractions so that users don't have to implement or even reason about such synchronization themselves, but it's still a central part of the work. I also disagree with your assertion that the programming language research community disagrees with my definitions - I'm more of a systems person, but I do end up in their playground sometimes.
- stonemetal 12y ago
  The Go and Haskell language communities have been pushing those definitions. For the most part they seem to be winning as those definitions are trotted out every time the subject comes up. https://www.haskell.org/haskellwiki/Parallelism_vs._Concurrency https://www.haskell.org/haskellwiki/Parallelism_vs._Concurre... http://blog.golang.org/concurrency-is-not-parallelism http://blog.golang.org/concurrency-is-not-parallelism
  - taeric 12y ago
    Any attempts to hijack common words for specific meaning in programming will abound with problems. I find it hardly surprising that many people view parrallelism as concurrently executing similar operations. The key being "similar operations." That actually fits with common uses of the words everywhere else. Consider, what does it mean to run parallel with something else? Means you are running a similar path. Not even necessarily at the same time. What does it mean to run concurrently with something else? Simply that you are running when it is.
    - stonemetal 12y ago
      I agree it is odd to say that something is running concurrently but with out concurrency. The problem is threads and locks and what not are well known as concurrency primitives, programming with them known as concurrent programming, etc. So what do we call concurrent programming that doesn't make use of any of the concurrency primitives, concurrent programming techniques? We want to call it something but we don't want to co-opt current nomenclature. Parallel isn't "taken" and does an OK job of expressing what they are doing.
      - taeric 12y ago
        I would have thought something like "independent" or "n-agent" would have caught on. Though, I'm fine with some ambiguity, to be honest. And I am not at all "in the field." My point is really that it is a lost cause to try and use such an outside common word to have a specific meaning.

by 14113 12y ago

scott_s 12y ago

What you call concurrency, I call concurrency control. But more often, I just call it synchronization. I am unaware of any widespread acceptance of your definition.

14113 12y ago
I hate doing this, but both the wikipedia articles for concurrency and parallelism agree with me: Concurrency > In computer science, concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. Parallelism > Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). In my opinion there's really not that much more to concurrency that's hard, apart from safely organising the communication between concurrent processes/threads. It's then the opposite problem with parallelism; the difficulty is finding and implementing the physically separate execution of different threads of execution in a program, while (generally) the communication is seen as a concurrency problem.
- scott_s 12y ago
  I don't think that definition of concurrency agrees with either of us. You defined concurrency as interaction itself, while Wikipedia defines it in terms of time and execution. I would agree with the Wikipedia definition if it changed "simultaneously" with "concurrently".
  - 14113 12y ago
    Hmm, that's a fair point. Thinking about it a bit more, I think my definitions fit closer with the idea of concurrent or parallel programs rather than the abstract ideas of concurrency or parallelism themselves. In any case, I completely disagree with your idea that parallel systems must be concurrent. Concurrency arises completely due to the interactions between separate threads of execution, whether they be executing at the same time on separate processors, or scheduled one after the other on a single processor. This is why we have various process calculii (csp, pi-calculus etc), and things such as session types, in order to try and reason about it and manage it. Parallelism assumes nothing about the interactions between processes, only that they are assumed to be running on separate hardware. As another poster mentioned, this fits fairly well with what the Haskell, Go, and general programming language research community considers to be the definitions of concurrency and parallelism. Look at any paper that focusses on concurrency, and you'll see it's about managing and making safe communication between separate processes. Look at a paper on parallelism, and it'll be on accelerating a parallelisable program on multicore or distributed hardware.
    - scott_s 12y ago
      As someone who works in parallel and distributed systems (which means I write and read papers on parallelism), I find your distinction an empty one. Actually getting good performance from parallelism requires careful consideration of the synchronization between the parallel parts - and in my work, the "careful consideration" is usually done by me. My goal is often to make abstractions so that users don't have to implement or even reason about such synchronization themselves, but it's still a central part of the work. I also disagree with your assertion that the programming language research community disagrees with my definitions - I'm more of a systems person, but I do end up in their playground sometimes.
stonemetal 12y ago
The Go and Haskell language communities have been pushing those definitions. For the most part they seem to be winning as those definitions are trotted out every time the subject comes up. https://www.haskell.org/haskellwiki/Parallelism_vs._Concurrency https://www.haskell.org/haskellwiki/Parallelism_vs._Concurre... http://blog.golang.org/concurrency-is-not-parallelism http://blog.golang.org/concurrency-is-not-parallelism
- taeric 12y ago
  Any attempts to hijack common words for specific meaning in programming will abound with problems. I find it hardly surprising that many people view parrallelism as concurrently executing similar operations. The key being "similar operations." That actually fits with common uses of the words everywhere else. Consider, what does it mean to run parallel with something else? Means you are running a similar path. Not even necessarily at the same time. What does it mean to run concurrently with something else? Simply that you are running when it is.
  - stonemetal 12y ago
    I agree it is odd to say that something is running concurrently but with out concurrency. The problem is threads and locks and what not are well known as concurrency primitives, programming with them known as concurrent programming, etc. So what do we call concurrent programming that doesn't make use of any of the concurrency primitives, concurrent programming techniques? We want to call it something but we don't want to co-opt current nomenclature. Parallel isn't "taken" and does an OK job of expressing what they are doing.
    - taeric 12y ago
      I would have thought something like "independent" or "n-agent" would have caught on. Though, I'm fine with some ambiguity, to be honest. And I am not at all "in the field." My point is really that it is a lost cause to try and use such an outside common word to have a specific meaning.