4 ms·

A review of concurrency vs parallelism[1] maybe useful when reviewing this link. [1] http://joearms.github.io/2013/04/05/concurrent-and-parallel-programming.ht

A review of concurrency vs parallelism[1] maybe useful when reviewing this link. [1] http://joearms.github.io/2013/04/05/concurrent-and-parallel-programming.html http://joearms.github.io/2013/04/05/concurrent-and-parallel-...

scott_s 12y ago
That's good, but does not quite capture the relationship in how I define it. From what I have said several times on HN now: Concurrency is a statement about a duration of time. Parallelism is a statement about a moment in time. During a one second window, did both of the tasks execute? If yes, then they are concurrent. If no, then they are not concurrent in that window. Was there any moment in time when both tasks were executing simultaneously? If yes, then they executed in parallel. If no, then they did not execute in parallel. From this, it's easy to see that if something is parallel, it must also be concurrent - in order for two tasks to execute during the same moment in time, they must also both have executed during a window of time. Concurrency without parallelism happens on machines with a single core. Processes and threads must time-share the only core. Frequently, some of these processes will have multiple threads, so the threads need to synchronize with each other, even though they don't execute at the same time.
- 14113 12y ago
  I don't think either definition really defines what concurrency and parallelism are really. Concurrency is the study of how processes interact (i.e. communication, locking etc) while parallelism is studying how to physically perform more things at once. In this way, concurrency and parallelism (almost) form two axes within which you can judge programs: You can have highly concurrent, but serial program (e.g. a browser, with different threads for networking, UI, etc), and highly parallel, but non-concurrent (or with little concurrency), like image processing, or other "embarrassingly parallel" problems.
  - scott_s 12y ago
    What you call concurrency, I call concurrency control. But more often, I just call it synchronization. I am unaware of any widespread acceptance of your definition.
    - 14113 12y ago
      I hate doing this, but both the wikipedia articles for concurrency and parallelism agree with me: Concurrency > In computer science, concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. Parallelism > Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). In my opinion there's really not that much more to concurrency that's hard, apart from safely organising the communication between concurrent processes/threads. It's then the opposite problem with parallelism; the difficulty is finding and implementing the physically separate execution of different threads of execution in a program, while (generally) the communication is seen as a concurrency problem.
      - scott_s 12y ago
        I don't think that definition of concurrency agrees with either of us. You defined concurrency as interaction itself, while Wikipedia defines it in terms of time and execution. I would agree with the Wikipedia definition if it changed "simultaneously" with "concurrently".
        14113 12y ago
        Hmm, that's a fair point. Thinking about it a bit more, I think my definitions fit closer with the idea of concurrent or parallel programs rather than the abstract ideas of concurrency or parallelism themselves. In any case, I completely disagree with your idea that parallel systems must be concurrent. Concurrency arises completely due to the interactions between separate threads of execution, whether they be executing at the same time on separate processors, or scheduled one after the other on a single processor. This is why we have various process calculii (csp, pi-calculus etc), and things such as session types, in order to try and reason about it and manage it. Parallelism assumes nothing about the interactions between processes, only that they are assumed to be running on separate hardware. As another poster mentioned, this fits fairly well with what the Haskell, Go, and general programming language research community considers to be the definitions of concurrency and parallelism. Look at any paper that focusses on concurrency, and you'll see it's about managing and making safe communication between separate processes. Look at a paper on parallelism, and it'll be on accelerating a parallelisable program on multicore or distributed hardware.
- qznc 12y ago
  I consider it parallelism without concurrency, if tasks are executed at the same time, but do not interact / synchronize. A parallel map operation (without reduce) would an example. The important thing is: Concurrency introduces indeterminism, while parallelism does not.
  - scott_s 12y ago
    Even a parallel map operation must have some synchronization in the beginning and end. The code you write may not, but the parallel framework that you rely on does.

scott_s 12y ago

That's good, but does not quite capture the relationship in how I define it. From what I have said several times on HN now: Concurrency is a statement about a duration of time. Parallelism is a statement about a moment in time. During a one second window, did both of the tasks execute? If yes, then they are concurrent. If no, then they are not concurrent in that window. Was there any moment in time when both tasks were executing simultaneously? If yes, then they executed in parallel. If no, then they did not execute in parallel. From this, it's easy to see that if something is parallel, it must also be concurrent - in order for two tasks to execute during the same moment in time, they must also both have executed during a window of time. Concurrency without parallelism happens on machines with a single core. Processes and threads must time-share the only core. Frequently, some of these processes will have multiple threads, so the threads need to synchronize with each other, even though they don't execute at the same time.

14113 12y ago
I don't think either definition really defines what concurrency and parallelism are really. Concurrency is the study of how processes interact (i.e. communication, locking etc) while parallelism is studying how to physically perform more things at once. In this way, concurrency and parallelism (almost) form two axes within which you can judge programs: You can have highly concurrent, but serial program (e.g. a browser, with different threads for networking, UI, etc), and highly parallel, but non-concurrent (or with little concurrency), like image processing, or other "embarrassingly parallel" problems.
- scott_s 12y ago
  What you call concurrency, I call concurrency control. But more often, I just call it synchronization. I am unaware of any widespread acceptance of your definition.
  - 14113 12y ago
    I hate doing this, but both the wikipedia articles for concurrency and parallelism agree with me: Concurrency > In computer science, concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. Parallelism > Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). In my opinion there's really not that much more to concurrency that's hard, apart from safely organising the communication between concurrent processes/threads. It's then the opposite problem with parallelism; the difficulty is finding and implementing the physically separate execution of different threads of execution in a program, while (generally) the communication is seen as a concurrency problem.
    - scott_s 12y ago
      I don't think that definition of concurrency agrees with either of us. You defined concurrency as interaction itself, while Wikipedia defines it in terms of time and execution. I would agree with the Wikipedia definition if it changed "simultaneously" with "concurrently".
      - 14113 12y ago
        Hmm, that's a fair point. Thinking about it a bit more, I think my definitions fit closer with the idea of concurrent or parallel programs rather than the abstract ideas of concurrency or parallelism themselves. In any case, I completely disagree with your idea that parallel systems must be concurrent. Concurrency arises completely due to the interactions between separate threads of execution, whether they be executing at the same time on separate processors, or scheduled one after the other on a single processor. This is why we have various process calculii (csp, pi-calculus etc), and things such as session types, in order to try and reason about it and manage it. Parallelism assumes nothing about the interactions between processes, only that they are assumed to be running on separate hardware. As another poster mentioned, this fits fairly well with what the Haskell, Go, and general programming language research community considers to be the definitions of concurrency and parallelism. Look at any paper that focusses on concurrency, and you'll see it's about managing and making safe communication between separate processes. Look at a paper on parallelism, and it'll be on accelerating a parallelisable program on multicore or distributed hardware.
qznc 12y ago
I consider it parallelism without concurrency, if tasks are executed at the same time, but do not interact / synchronize. A parallel map operation (without reduce) would an example. The important thing is: Concurrency introduces indeterminism, while parallelism does not.
- scott_s 12y ago
  Even a parallel map operation must have some synchronization in the beginning and end. The code you write may not, but the parallel framework that you rely on does.