5 ms·

I don't know who invented this nonsense distinction. First time I was introduced to this idea of "concurrent programming" being a separate thing when Go was rel

I don't know who invented this nonsense distinction. First time I was introduced to this idea of "concurrent programming" being a separate thing when Go was released. So, I associate this nonsense with Go, but it could have happened earlier, I simply never heard about it before then. Anyways. The way I see it used today, it's applied to language runtimes incapable or severely crippled when it comes to parallel / concurrent execution. Eg. Python, JavaScript etc. In such environments programmers are offered a mechanism that has many downsides of parallel / concurrent programming (eg. unpredictable order of execution) without the benefits of parallel / concurrent programming (ie. nothing actually happens at the same time, or only sleep is possible at the same time etc.) I feel like this distinction, while a nonsense idea at its core, became so popular due to the popularity of language runtimes with disabilities and their users needing to validate their worth by adding features to their languages their runtimes are inherently incapable of implementing. Similar situation happened with ML-style types. Python, for example, works very poorly with this add-on, but the desire to match features of other languages led Python developers to add those types anyways. Similarly, TypeScript and a bunch of similar languages, especially in Web.

vore 3y ago
How is this nonsense or anything to do with "language runtimes with disabilities"? An OS running on a single core processor cannot be parallel but it may be concurrent: it can never physically do two things at the same time, but it might be able to logically interleave different tasks. Parallelism is a physical thing, concurrency is a logical thing.
- crabbone 3y ago
  You have just repeated the nonsense I was talking about. The claim you repeat is meaningless. A program is either parallel / concurrent or not. The situation you describe (when there's a single processor core) isn't parallel or concurrent. In some sense, it emulates concurrent / parallel execution because it imitates the unpredictable ordering of code execution, which sure has its uses... but the whole point of dealing with this unpredictable ordering is that we actually want parallelism / concurrency. The emulation on its own is worthless.
  - vore 3y ago
    No it's not? How do you characterize running two programs on a single core without calling it concurrent but not parallel? There is a clear distinction between logical multitasking and physical multitasking that I think is useful to taxonomize.
  - stoniejohnson 3y ago
    bruh your OS scheduler wants a word with you
  - preseinger 3y ago
    a program (as written) is either concurrent or not, a program (as executed) is either parallel or not a program which is not written as concurrent can never be executed as parallel a program which is written as concurrent can be executed as parallel, or not > The situation you describe (when there's a single processor core) isn't parallel or concurrent. a program running on a single core can never be parallel, but it can be concurrent concurrent is a logical property, parallel is a physical property
    - wizofaus 3y ago
      "a program which is not written as concurrent can never be executed as parallel" Unless of course you're running multiple independent instances of it, each with different parameters, which I gather is a pretty common way of running long running CPU- intensive operations on large data sets. Presumably there's often some final separate step that may be needed to combine the results once they're all finished, which if run manually obviates any concurrency concerns at the software level.
      - preseinger 3y ago
        in which case that program is being executed N times, so none of this applies ("a program [as executed]" is a single instance (process) of a binary)
  - chlorion 3y ago
    Yes, concurrency on a single core CPU is simply not possible. That's why multitasking OS's didn't exist until multicore CPUs became a thing. If only these "Gophers" knew anything about computing history!
- CyberDildonics 3y ago
  Parallelism is a physical thing, concurrency is a logical thing. Fundamentally the difficulty is all about synchronization. People can try to split hairs and say there are two terms for two different things but ultimately it doesn't matter because the underlying problem is the same.
  - vore 3y ago
    Single core concurrency doesn't have to deal with hardware memory synchronization.
    - CyberDildonics 3y ago
      What point are you trying to make now? You just said: Parallelism is a physical thing, concurrency is a logical thing. So by your own definition, wouldn't multi-core concurrency be parallelism?
      - vore 3y ago
        Yes? Parallelism is inherently concurrent but concurrency is not inherently parallel. But single core concurrency still exists as a form of non-parallel concurrency. The point I'm trying to make is down to brass tacks, single core concurrency is a lot simpler than multi core concurrency, because you don't need any hardware cooperation.
        CyberDildonics 3y ago
        Before you were saying they are two different things, now you're saying they are the same when you have multiple cores and not the same when you have a single core. If they're the same when you have multiple cores, isn't there just single and multi-core concurrency by your own definitions? If so, why are you making a distinction of parallelism and concurrency?
        dkersten 3y ago
        Parallelism is concurrent. Concurrency isn’t necessarily parallel. There are still plenty of single core systems out there running concurrent software, even today, for example embedded systems.
phkahler 3y ago
>> don't know who invented this nonsense distinction ... It not nonsense. In C or C++ lots of code can be made parallel using OpenMP and inserting some #pragma statements above for loops. This does not work for things like running a UI in one thread and some other work in another thread, perhaps displaying results as they are found. These are quite different types of parallelism.
- Nathanba 3y ago
  I think it's nonsense to invent new terminology for this tiny minor difference in how you use threads and whether you wait on results or you don't wait.
  - ascar 3y ago
    but it's just not a tiny minor difference. The problems you end up dealing with are entirely different. The terminology used might be terrible since they're originally synonyms. Differentiating between the two is very useful though.
    - Nathanba 3y ago
      how are the problems different? In all cases you want the maximum performance. The only decision here is how you schedule your tasks. Ideally entirely in parallel and you only schedule them to wait if you can't find a way to make them 100% parallel
      - ascar 3y ago
        If there is no dependency between your results, you do not need synchronization. Parallelizing it just means putting them into a seperate stream of execution. That can be a thread or an entirely different computer and the two different execution streams need to synchronization or communication between each other. That is an entirely different set of problems than having to deal with a computation that has close dependencies and now you need synchronization and communication to progress the compuation. You don't only have to think about how to synchronize your computation, you also have to think about how to distribute your data to begin with to minimize the need for synchronization. An entire set of problems that just don't exist in the former case. To pick up a word you used: In one case you have many independent tasks and you want to run them as quickly as possible. In the other case you have a single task and you think about how to split that up to make it faster.
AmpsterMan 3y ago
Rob Pike has a discussion on the distinctions between parallelism and concurrency. Concurrency is closely related to co-routines which are a distinct invention from threads/processes which are more related to parallelism.
iamapart 3y ago
[dead]
sublinear 3y ago
> In such environments programmers are offered a mechanism that has many downsides of parallel / concurrent programming (eg. unpredictable order of execution) without the benefits of parallel / concurrent programming (ie. nothing actually happens at the same time, or only sleep is possible at the same time etc.) From the developer's perspective it's a massive upside to not have to manage low-level details and just define how the event loop will call their code. Any modern web browser has plenty of parallel execution behind the scenes, but the developer (and user) will just see concurrency which is much simpler to reason about. The order of execution doesn't matter if the things being executed aren't dependent. What matters more is that there's only one thread to think about. If they are dependent they shouldn't have been parallelized in the first place, so they're not.
- crabbone 3y ago
  What does this have to do with low or whatever other level?
mmcnl 3y ago
Just because you didn't know about the concept doesn't mean the distinction is nonsense. I think they are similar but not the same, exactly for the reasons laid out in the comment you are replying too. Just because you hate the languages that support concurrent programming doesn't mean concurrent programming is meaningless. Any language using async/await (basically all of them these days) support concurrent programming, including languages such as Swift and C# which are nothing like Python or JavaScript.
saltcured 3y ago
I'm not sure I can identify when it started, but these were already the concepts commonly in use when I did my CS undergraduate work in the early 90s. I.e. it was in textbooks and course titles as established jargon. Concurrency was the kind of thing worried about in OS design or Unix programming styles whether on a time-sharing system or some small scale multi-processing system. Coordination of heterogeneous sequential programs on some shared resources. Parallelism was the topic of high-performance computing with combined use of many hardware resources to accelerate a single algorithm. Of course these are simplifying abstractions, and real systems can get into the murky gray area that is both concurrent and parallel.

vore 3y ago

How is this nonsense or anything to do with "language runtimes with disabilities"? An OS running on a single core processor cannot be parallel but it may be concurrent: it can never physically do two things at the same time, but it might be able to logically interleave different tasks. Parallelism is a physical thing, concurrency is a logical thing.

crabbone 3y ago
You have just repeated the nonsense I was talking about. The claim you repeat is meaningless. A program is either parallel / concurrent or not. The situation you describe (when there's a single processor core) isn't parallel or concurrent. In some sense, it emulates concurrent / parallel execution because it imitates the unpredictable ordering of code execution, which sure has its uses... but the whole point of dealing with this unpredictable ordering is that we actually want parallelism / concurrency. The emulation on its own is worthless.
- vore 3y ago
  No it's not? How do you characterize running two programs on a single core without calling it concurrent but not parallel? There is a clear distinction between logical multitasking and physical multitasking that I think is useful to taxonomize.
- stoniejohnson 3y ago
  bruh your OS scheduler wants a word with you
- preseinger 3y ago
  a program (as written) is either concurrent or not, a program (as executed) is either parallel or not a program which is not written as concurrent can never be executed as parallel a program which is written as concurrent can be executed as parallel, or not > The situation you describe (when there's a single processor core) isn't parallel or concurrent. a program running on a single core can never be parallel, but it can be concurrent concurrent is a logical property, parallel is a physical property
  - wizofaus 3y ago
    "a program which is not written as concurrent can never be executed as parallel" Unless of course you're running multiple independent instances of it, each with different parameters, which I gather is a pretty common way of running long running CPU- intensive operations on large data sets. Presumably there's often some final separate step that may be needed to combine the results once they're all finished, which if run manually obviates any concurrency concerns at the software level.
    - preseinger 3y ago
      in which case that program is being executed N times, so none of this applies ("a program [as executed]" is a single instance (process) of a binary)
- chlorion 3y ago
  Yes, concurrency on a single core CPU is simply not possible. That's why multitasking OS's didn't exist until multicore CPUs became a thing. If only these "Gophers" knew anything about computing history!
CyberDildonics 3y ago
Parallelism is a physical thing, concurrency is a logical thing. Fundamentally the difficulty is all about synchronization. People can try to split hairs and say there are two terms for two different things but ultimately it doesn't matter because the underlying problem is the same.
- vore 3y ago
  Single core concurrency doesn't have to deal with hardware memory synchronization.
  - CyberDildonics 3y ago
    What point are you trying to make now? You just said: Parallelism is a physical thing, concurrency is a logical thing. So by your own definition, wouldn't multi-core concurrency be parallelism?
    - vore 3y ago
      Yes? Parallelism is inherently concurrent but concurrency is not inherently parallel. But single core concurrency still exists as a form of non-parallel concurrency. The point I'm trying to make is down to brass tacks, single core concurrency is a lot simpler than multi core concurrency, because you don't need any hardware cooperation.
      - CyberDildonics 3y ago
        Before you were saying they are two different things, now you're saying they are the same when you have multiple cores and not the same when you have a single core. If they're the same when you have multiple cores, isn't there just single and multi-core concurrency by your own definitions? If so, why are you making a distinction of parallelism and concurrency?
        dkersten 3y ago
        Parallelism is concurrent. Concurrency isn’t necessarily parallel. There are still plenty of single core systems out there running concurrent software, even today, for example embedded systems.

phkahler 3y ago

>> don't know who invented this nonsense distinction ... It not nonsense. In C or C++ lots of code can be made parallel using OpenMP and inserting some #pragma statements above for loops. This does not work for things like running a UI in one thread and some other work in another thread, perhaps displaying results as they are found. These are quite different types of parallelism.

Nathanba 3y ago
I think it's nonsense to invent new terminology for this tiny minor difference in how you use threads and whether you wait on results or you don't wait.
- ascar 3y ago
  but it's just not a tiny minor difference. The problems you end up dealing with are entirely different. The terminology used might be terrible since they're originally synonyms. Differentiating between the two is very useful though.
  - Nathanba 3y ago
    how are the problems different? In all cases you want the maximum performance. The only decision here is how you schedule your tasks. Ideally entirely in parallel and you only schedule them to wait if you can't find a way to make them 100% parallel
    - ascar 3y ago
      If there is no dependency between your results, you do not need synchronization. Parallelizing it just means putting them into a seperate stream of execution. That can be a thread or an entirely different computer and the two different execution streams need to synchronization or communication between each other. That is an entirely different set of problems than having to deal with a computation that has close dependencies and now you need synchronization and communication to progress the compuation. You don't only have to think about how to synchronize your computation, you also have to think about how to distribute your data to begin with to minimize the need for synchronization. An entire set of problems that just don't exist in the former case. To pick up a word you used: In one case you have many independent tasks and you want to run them as quickly as possible. In the other case you have a single task and you think about how to split that up to make it faster.

AmpsterMan 3y ago

Rob Pike has a discussion on the distinctions between parallelism and concurrency. Concurrency is closely related to co-routines which are a distinct invention from threads/processes which are more related to parallelism.

iamapart 3y ago

[dead]

sublinear 3y ago

> In such environments programmers are offered a mechanism that has many downsides of parallel / concurrent programming (eg. unpredictable order of execution) without the benefits of parallel / concurrent programming (ie. nothing actually happens at the same time, or only sleep is possible at the same time etc.) From the developer's perspective it's a massive upside to not have to manage low-level details and just define how the event loop will call their code. Any modern web browser has plenty of parallel execution behind the scenes, but the developer (and user) will just see concurrency which is much simpler to reason about. The order of execution doesn't matter if the things being executed aren't dependent. What matters more is that there's only one thread to think about. If they are dependent they shouldn't have been parallelized in the first place, so they're not.

crabbone 3y ago
What does this have to do with low or whatever other level?

mmcnl 3y ago

Just because you didn't know about the concept doesn't mean the distinction is nonsense. I think they are similar but not the same, exactly for the reasons laid out in the comment you are replying too. Just because you hate the languages that support concurrent programming doesn't mean concurrent programming is meaningless. Any language using async/await (basically all of them these days) support concurrent programming, including languages such as Swift and C# which are nothing like Python or JavaScript.

saltcured 3y ago

I'm not sure I can identify when it started, but these were already the concepts commonly in use when I did my CS undergraduate work in the early 90s. I.e. it was in textbooks and course titles as established jargon. Concurrency was the kind of thing worried about in OS design or Unix programming styles whether on a time-sharing system or some small scale multi-processing system. Coordination of heterogeneous sequential programs on some shared resources. Parallelism was the topic of high-performance computing with combined use of many hardware resources to accelerate a single algorithm. Of course these are simplifying abstractions, and real systems can get into the murky gray area that is both concurrent and parallel.