3 ms·

I'm curious. How would you define parallelism?

ErrantX 17y ago
I think the main problem is in the posters definition: A parallel program, on the other hand, is one that merely runs on multiple processors, with the goal of hopefully running faster than it would on a single CPU. It's an incredibly vague definition that actually encompasses concurrency too.. Wikipedia puts it quite succinctly: Parallel computing is a form of computation in which many calculations are carried out simultaneously,[1] operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). The WP article then goes on to discuss specific forms of parallelism; some concurrent, some not necessarily so. This article appears to be so heavily focused on the idea that concurrency and parallelism are discrete and disconnected that it seems to forget there is a bigger picture. The writer has a strong key point. Parallelism !== Concurrency (note the double =). However it's ignoring the fact that they DO intersect frequently.
- lucifer 17y ago
  The article's writer likely intended the "merely" to indicate that the "deterministic" nature of a program is not affected by the "mere" fact of the parallel execution of its parts. I believe this distinction between deterministic vs. non-deterministic execution is the critical aspect.
- winterkoninkje 17y ago
  Whatever the wording, there is a crucial difference between the two, irrespective of whether or how frequently they overlap or intersect. There is a fundamental difference between asynchronous separation of concerns (concurrency) and multiple simultaneous use of resources (parallelism). Concurrency is a conceptual organization, whereas parallelism is a physical organization. These are orthogonal and you're free to have one, the other, neither, or both (in different ways). Just because they can interact in interesting ways does nothing to diminish their difference. The point the OP was making is correct and well phrased, the comments here not withstanding. People who come from C land are used to dealing with threads (concurrency) as the primary means of doing anything non-serial. Some may be familiar with SPMD message-passing (parallelism), but they'll already know what's up. Unless you've programmed in Fortran or APL, are familiar with the Connection Machines, or do GPU coding, you won't have experienced SIMD (parallelism). People who've only ever used threads have an extremely impoverished view of the world of parallelism. Because they know threads==concurrency and they know threads can be used for parallelism, they assume the two are coextensive when they really are not.
- simonmar 17y ago
  Yes, a fair point. Concurrent and parallel do not describe disjoint sets of programs, of course. However, they really are different concepts. I think wikipedia's page on concurrency could use some work to make this clearer, maybe I'll try to gather enough references to make the case. If you want to rephrase the title, perhaps "Parallelism =/> Concurrency" would be better ("=/>" is "does not imply").

I think the main problem is in the posters definition: A parallel program, on the other hand, is one that merely runs on multiple processors, with the goal of hopefully running faster than it would on a single CPU. It's an incredibly vague definition that actually encompasses concurrency too.. Wikipedia puts it quite succinctly: Parallel computing is a form of computation in which many calculations are carried out simultaneously,[1] operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). The WP article then goes on to discuss specific forms of parallelism; some concurrent, some not necessarily so. This article appears to be so heavily focused on the idea that concurrency and parallelism are discrete and disconnected that it seems to forget there is a bigger picture. The writer has a strong key point. Parallelism !== Concurrency (note the double =). However it's ignoring the fact that they DO intersect frequently.

lucifer 17y ago
The article's writer likely intended the "merely" to indicate that the "deterministic" nature of a program is not affected by the "mere" fact of the parallel execution of its parts. I believe this distinction between deterministic vs. non-deterministic execution is the critical aspect.
winterkoninkje 17y ago
Whatever the wording, there is a crucial difference between the two, irrespective of whether or how frequently they overlap or intersect. There is a fundamental difference between asynchronous separation of concerns (concurrency) and multiple simultaneous use of resources (parallelism). Concurrency is a conceptual organization, whereas parallelism is a physical organization. These are orthogonal and you're free to have one, the other, neither, or both (in different ways). Just because they can interact in interesting ways does nothing to diminish their difference. The point the OP was making is correct and well phrased, the comments here not withstanding. People who come from C land are used to dealing with threads (concurrency) as the primary means of doing anything non-serial. Some may be familiar with SPMD message-passing (parallelism), but they'll already know what's up. Unless you've programmed in Fortran or APL, are familiar with the Connection Machines, or do GPU coding, you won't have experienced SIMD (parallelism). People who've only ever used threads have an extremely impoverished view of the world of parallelism. Because they know threads==concurrency and they know threads can be used for parallelism, they assume the two are coextensive when they really are not.
simonmar 17y ago
Yes, a fair point. Concurrent and parallel do not describe disjoint sets of programs, of course. However, they really are different concepts. I think wikipedia's page on concurrency could use some work to make this clearer, maybe I'll try to gather enough references to make the case. If you want to rephrase the title, perhaps "Parallelism =/> Concurrency" would be better ("=/>" is "does not imply").