4 ms·

If you actually have concurrency, and you have the hardware able to handle it, why do you not have parallelism? I.e., simply spinning up an actor obviously doe

If you actually have concurrency, and you have the hardware able to handle it, why do you not have parallelism? I.e., simply spinning up an actor obviously does not imply concurrency; the classic Erlang 'ping pong' example shows processes waiting to receive a message and responding when they do. Not actually concurrent, despite multiple actors and multiple processor cores it's a sequential program, not a concurrent one. Likewise, if you have 5 processes all adding a list of numbers (independently and sequentially, one element at a time), but only 4 cores, you would have only 4 processes running in parallel at a time; a hardware limit. Up the cores, you can have all 5. Can you give me an example of a concurrent program that is not also parallel, due to non-hardware reasons, in Erlang? Because switching the goalposts around - as you say, Erlang encourages you to write concurrent programs, even before CPUs allowed true parallelism, and because of that, once the CPU allowed true parallelism, it only required a VM tweak and suddenly Erlang programs were running in parallel. But you say no, that that wasn't the case; just because you wrote things to be concurrent, and now the hardware has changed to allow them to be in parallel, they aren't necessarily parallel. I'd love an example of that.

chrisseaton 5y ago
> If you actually have concurrency, and you have the hardware able to handle it, why do you not have parallelism? You can have concurrent tasks but with dependencies between them which means there is no parallelism available. > Can you give me an example of a concurrent program that is not also parallel, due to non-hardware reasons, in Erlang? For example.... > the classic Erlang 'ping pong' example shows processes waiting to receive a message and responding when they do. Not actually concurrent, despite multiple actors and multiple processor cores it's a sequential program, not a concurrent one. No that is actually concurrent. These are two concurrent tasks. They just don't have any parallelism between them because there is a sequential dependency and only one is able to make progress at any given time. Concurrent and parallel are not the same thing, if you didn't know. Concurrency is having work in progress on multiple tasks at the same time. You don't need to actually be advancing more than one task at the same time for it to be concurrent.
- lostcolony 5y ago
  "You can have concurrent tasks but with dependencies between them which means there is no parallelism available." - that isn't really concurrent then, is it? Concurrency implies running two things concurrently; you're just creating things to wait. No useful work is being done. You broke a sequential task across processes. Just because those processes are stuck on a receive loop doesn't mean they're doing anything. Might as well say your Java program is concurrent because you spun up a second thread that just waits. That was what I was getting at with my question; arbitrary processes that do their work in a serial fashion...isn't a concurrent program. It's a badly written serial one.
  - chrisseaton 5y ago
    > that isn't really concurrent then It is, using standard industry terminology. Concurrency does not require that concurrent tasks are able to make independent progress at the same time, just that they are in some point of progress at the same time. Back in 2012 when I was working on my PhD I wrote a paper about an example algorithm with very little inherent parallelism, where even if you have multiple concurrent tasks all making forward progress you may find it extremely hard to actually produce results in parallel, if you want to do a deep dive into this problem. https://chrisseaton.com/multiprog2012/seaton-multiprog2012.pdf https://chrisseaton.com/multiprog2012/seaton-multiprog2012.p... I wrote a poster about it as well for an easier introduction. https://chrisseaton.com/research-symposium-2012/seaton-irregular-parallelism-poster.pdf https://chrisseaton.com/research-symposium-2012/seaton-irreg...
    - lostcolony 5y ago
      Thank you; the example helped clarify for me. Yes, any time there are shared resources, concurrent tasks can hit synchronization points, and end up bottlenecking on them. Of course, I'd contend that a point of synchronization is a form of serialization, but I agree that the tasks being synchronized would still be described as concurrent (since they don't causally affect their ordering). But such a synchronization is a necessary part of the algorithm you're choosing to use (even prior to SMP support), or it would be completely unnatural to include it. I don't know that it really invalidates the OP's point, that the language didn't remove the bottlenecks you introduced?
Jtsummers 5y ago
Concurrency produces parallelism when the concurrent portions can actually run at the same time. That is, between synchronization points which force sequential behavior. It's very feasible to write a concurrent program that overuses (versus what is strictly needed) synchronization points in order to produce something which offers a better codebase (by some measure) but provides no parallelism. You have to remove/minimize these synchronization points in order to obtain parallelism. It's also possible that by design these synchronization points can't be eliminated. Erlang's message passing is inherently asynchronous, but you could conceive of similar synchronization points in a program that will reduce the potential parallelism or eliminate it depending on the overall structure. For instance: PID ! {self(), Message}, ;; forgot self() initially receive {PID, Response} -> end, ... You still have a concurrent design (which can be easier to reason about), but because you're waiting on a response you end up without any parallelism no matter how many cores you throw at it.
- lostcolony 5y ago
  Right, when I said "actually concurrent", I meant the code -could- run simultaneously (or more formally, that the items of execution aren't necessarily causally related). Any sequential operation could be spread out over an arbitrary number of Erlang processes, without it actually being concurrent. That design isn't concurrent, you just are involving a bunch of processes that wait. That said, you are right that a synch point around a controlled resource (i.e., "we only want to support processing 5 requests at a time") would still have concurrent tasks (i.e., ten requests come in; there is no casual relation between when they run). Of course...that's an intentional point of synchronization necessary for correctness; it still strikes me as a strawman to say that you don't get parallelization for concurrent code, just because you forced a bottleneck somewhere.

chrisseaton 5y ago

> If you actually have concurrency, and you have the hardware able to handle it, why do you not have parallelism? You can have concurrent tasks but with dependencies between them which means there is no parallelism available. > Can you give me an example of a concurrent program that is not also parallel, due to non-hardware reasons, in Erlang? For example.... > the classic Erlang 'ping pong' example shows processes waiting to receive a message and responding when they do. Not actually concurrent, despite multiple actors and multiple processor cores it's a sequential program, not a concurrent one. No that is actually concurrent. These are two concurrent tasks. They just don't have any parallelism between them because there is a sequential dependency and only one is able to make progress at any given time. Concurrent and parallel are not the same thing, if you didn't know. Concurrency is having work in progress on multiple tasks at the same time. You don't need to actually be advancing more than one task at the same time for it to be concurrent.

lostcolony 5y ago
"You can have concurrent tasks but with dependencies between them which means there is no parallelism available." - that isn't really concurrent then, is it? Concurrency implies running two things concurrently; you're just creating things to wait. No useful work is being done. You broke a sequential task across processes. Just because those processes are stuck on a receive loop doesn't mean they're doing anything. Might as well say your Java program is concurrent because you spun up a second thread that just waits. That was what I was getting at with my question; arbitrary processes that do their work in a serial fashion...isn't a concurrent program. It's a badly written serial one.
- chrisseaton 5y ago
  > that isn't really concurrent then It is, using standard industry terminology. Concurrency does not require that concurrent tasks are able to make independent progress at the same time, just that they are in some point of progress at the same time. Back in 2012 when I was working on my PhD I wrote a paper about an example algorithm with very little inherent parallelism, where even if you have multiple concurrent tasks all making forward progress you may find it extremely hard to actually produce results in parallel, if you want to do a deep dive into this problem. https://chrisseaton.com/multiprog2012/seaton-multiprog2012.pdf https://chrisseaton.com/multiprog2012/seaton-multiprog2012.p... I wrote a poster about it as well for an easier introduction. https://chrisseaton.com/research-symposium-2012/seaton-irregular-parallelism-poster.pdf https://chrisseaton.com/research-symposium-2012/seaton-irreg...
  - lostcolony 5y ago
    Thank you; the example helped clarify for me. Yes, any time there are shared resources, concurrent tasks can hit synchronization points, and end up bottlenecking on them. Of course, I'd contend that a point of synchronization is a form of serialization, but I agree that the tasks being synchronized would still be described as concurrent (since they don't causally affect their ordering). But such a synchronization is a necessary part of the algorithm you're choosing to use (even prior to SMP support), or it would be completely unnatural to include it. I don't know that it really invalidates the OP's point, that the language didn't remove the bottlenecks you introduced?

Jtsummers 5y ago

Concurrency produces parallelism when the concurrent portions can actually run at the same time. That is, between synchronization points which force sequential behavior. It's very feasible to write a concurrent program that overuses (versus what is strictly needed) synchronization points in order to produce something which offers a better codebase (by some measure) but provides no parallelism. You have to remove/minimize these synchronization points in order to obtain parallelism. It's also possible that by design these synchronization points can't be eliminated. Erlang's message passing is inherently asynchronous, but you could conceive of similar synchronization points in a program that will reduce the potential parallelism or eliminate it depending on the overall structure. For instance: PID ! {self(), Message}, ;; forgot self() initially receive {PID, Response} -> end, ... You still have a concurrent design (which can be easier to reason about), but because you're waiting on a response you end up without any parallelism no matter how many cores you throw at it.

lostcolony 5y ago
Right, when I said "actually concurrent", I meant the code -could- run simultaneously (or more formally, that the items of execution aren't necessarily causally related). Any sequential operation could be spread out over an arbitrary number of Erlang processes, without it actually being concurrent. That design isn't concurrent, you just are involving a bunch of processes that wait. That said, you are right that a synch point around a controlled resource (i.e., "we only want to support processing 5 requests at a time") would still have concurrent tasks (i.e., ten requests come in; there is no casual relation between when they run). Of course...that's an intentional point of synchronization necessary for correctness; it still strikes me as a strawman to say that you don't get parallelization for concurrent code, just because you forced a bottleneck somewhere.