4 ms·

I find the distinction far less interesting than most people. I thing it's easier to think of parallelism simply as an interesting special case of concurrency,

by jerf 5y ago

I find the distinction far less interesting than most people. I thing it's easier to think of parallelism simply as an interesting special case of concurrency, and to think of runtimes and systems that are "concurrent" but can't run literally simultaneously, like Javascript or Python, as simply accidents of history not worth specially writing into the definitions of our terms. Every year "concurrent" code that can't be scheduled to run on multiple CPUs simultaneously is less and less interesting. And I don't anticipate any new language coming out in the future that will try to be "concurrent but can only run on one CPU", so this meaning is just going to fade into history as a particular quirk of some legacy runtimes. No, I do not consider any runtime that can't run on multiple CPUs simultaneously to have "good" concurrency support. It would, at best, be bad concurrency support. Better than "no" support, sure, but not good in 2021. If a new language came out with that support, nobody would call it "good" support, they'd call it failing to even raise the table stakes a new language needs nowadays.

Spivak 5y ago
What's your definition of running simultaneously because CPython's approach to running on multiple cores is multiprocessing which works honestly fine. The tooling to do it is pretty slick where you can ignore a lot of the typical pain of IPC. Because if "good" concurrency support means single-process multi-threaded on multiple cores running with enough locks that you can have multiple threads executing code simultaneously in a shared memory space then a lot of languages are going to fall down or punt all responsibility for doing that safely to the programmer which might as well be no support.
- jerf 5y ago
  Shared memory spaces. Your "slick tooling" is one of the hacks I mentioned that history will forget, by the standard of "if a new language emerged that tried to call that 'concurrency' nobody would take it seriously". I should say that "hack" here isn't necessarily a perjorative. There are reasons for communities to create and deploy those. There are plenty of cases where existing code can be leveraged to work better than it could without it, and that's the relevant standard for whether something is useful, not whether or not in a parallel universe the code could have been written in some completely different way or whether in some hypothetical sense if you could push a button and rewrite something for free you'd end up with something better. Hacks can be good, and every language will pick them up at some point as history evolves around it and some of the core assumptions a language/runtime made go out of date. But it's still a hack. While OS process boundaries do provide some nice features, they are also in general overkill. See Erlang and Pony for some alternate riffs on the idea, and if you look at it hard enough and understand it deeply enough, even what Rust does can provide some similar benefits without raising a full OS process boundary between bits of code.
  - catern 5y ago
    You're saying that shared memory concurrency is the future? I think you've got it completely wrong. Shared memory concurrency is the past. It was thought to be a good idea in the 80s and 90s, but we now know that it's both hard to program, and that it works poorly with highly-parallel, high-performance hardware. In the future, we'll see more and more programming systems which don't provide shared memory concurrency at all.
    - bottled_poe 5y ago
      I believe there will always be situations (though specific) where a shared-memory concurrency model is the most efficient and performant use of available resources. For this reason alone, shared-memory concurrency will always have a place. That said, I generally agree that the isolated parallel memory model is preferable for simplicity's sake.
    - jerf 5y ago
      Shared memory space, not necessarily shared memory. I referenced Erlang & Pony after all so it's not like I'm unaware of the issues here. You don't need a full OS process boundary to separate things; it is a very crude and blunt instrument and there are much better options available. Separation is nice but paying to cross OS boundaries with every message kills performance dead. A full OS process boundary between all concurrently-running threads would be insane overkill for a Rust, Erlang or Pony program, and in practice, with best practices and the use of some tooling, even for Go, even if it is just old-fashioned shared-memory at its core.
      - catern 5y ago
        "Memory space" is usually called "address space". And I think you're confused - context switching is expensive but there's no context switching involved when you're running in parallel across N cores. We're talking about parallelism, not concurrency.
        jerf 5y ago
simiones 5y ago
I have usually seen the distinction between concurrency and parallelism being drawn for precisely the opposite reason: parallelism without concurrency is a relatively commonly used niche, it is much simpler than concurrency, and it has special tools that make sense just for it. For example, CUDA exposes a parallel programming model with low support for actual concurrency. Similarly, OpenMP is mostly useful for parallel computations that rarely need coordination. C#'s Parallel package is another example. By contrast, concurrency often rears its ugly head even in single-threaded workflows, even ones that are part of larger multi-threaded programs.

by jerf 5y ago

Spivak 5y ago

What's your definition of running simultaneously because CPython's approach to running on multiple cores is multiprocessing which works honestly fine. The tooling to do it is pretty slick where you can ignore a lot of the typical pain of IPC. Because if "good" concurrency support means single-process multi-threaded on multiple cores running with enough locks that you can have multiple threads executing code simultaneously in a shared memory space then a lot of languages are going to fall down or punt all responsibility for doing that safely to the programmer which might as well be no support.

jerf 5y ago
Shared memory spaces. Your "slick tooling" is one of the hacks I mentioned that history will forget, by the standard of "if a new language emerged that tried to call that 'concurrency' nobody would take it seriously". I should say that "hack" here isn't necessarily a perjorative. There are reasons for communities to create and deploy those. There are plenty of cases where existing code can be leveraged to work better than it could without it, and that's the relevant standard for whether something is useful, not whether or not in a parallel universe the code could have been written in some completely different way or whether in some hypothetical sense if you could push a button and rewrite something for free you'd end up with something better. Hacks can be good, and every language will pick them up at some point as history evolves around it and some of the core assumptions a language/runtime made go out of date. But it's still a hack. While OS process boundaries do provide some nice features, they are also in general overkill. See Erlang and Pony for some alternate riffs on the idea, and if you look at it hard enough and understand it deeply enough, even what Rust does can provide some similar benefits without raising a full OS process boundary between bits of code.
- catern 5y ago
  You're saying that shared memory concurrency is the future? I think you've got it completely wrong. Shared memory concurrency is the past. It was thought to be a good idea in the 80s and 90s, but we now know that it's both hard to program, and that it works poorly with highly-parallel, high-performance hardware. In the future, we'll see more and more programming systems which don't provide shared memory concurrency at all.
  - bottled_poe 5y ago
    I believe there will always be situations (though specific) where a shared-memory concurrency model is the most efficient and performant use of available resources. For this reason alone, shared-memory concurrency will always have a place. That said, I generally agree that the isolated parallel memory model is preferable for simplicity's sake.
  - jerf 5y ago
    Shared memory space, not necessarily shared memory. I referenced Erlang & Pony after all so it's not like I'm unaware of the issues here. You don't need a full OS process boundary to separate things; it is a very crude and blunt instrument and there are much better options available. Separation is nice but paying to cross OS boundaries with every message kills performance dead. A full OS process boundary between all concurrently-running threads would be insane overkill for a Rust, Erlang or Pony program, and in practice, with best practices and the use of some tooling, even for Go, even if it is just old-fashioned shared-memory at its core.
    - catern 5y ago
      "Memory space" is usually called "address space". And I think you're confused - context switching is expensive but there's no context switching involved when you're running in parallel across N cores. We're talking about parallelism, not concurrency.
      - jerf 5y ago

simiones 5y ago

I have usually seen the distinction between concurrency and parallelism being drawn for precisely the opposite reason: parallelism without concurrency is a relatively commonly used niche, it is much simpler than concurrency, and it has special tools that make sense just for it. For example, CUDA exposes a parallel programming model with low support for actual concurrency. Similarly, OpenMP is mostly useful for parallel computations that rarely need coordination. C#'s Parallel package is another example. By contrast, concurrency often rears its ugly head even in single-threaded workflows, even ones that are part of larger multi-threaded programs.