41 ms·

> For communicating gigabytes of data back and forth you want shared memory anyway. Not really. You are better off communicating what to do with the data, rath

> For communicating gigabytes of data back and forth you want shared memory anyway. Not really. You are better off communicating what to do with the data, rather than sharing the data.

adwn 8y ago
> You are better off communicating what to do with the data, rather than sharing the data. Not if you actually want to distribute the work over several cores. Example: Your data is partitioned into 4 segments A, B, C, and D. In step 1, node X processes A&B and node Y processes C&D; in step 2, node X processes A&D and node Y processes C&B. If A, B, C, and D are large, this is a prime example where shared memory trumps message passing in terms of performance. If you think that's a contrived case: dense matrix multiplication behaves like this.
pcwalton 8y ago
Only if you don't have a data-parallel workload. Message passing is often a nonstarter for data-parallel workloads. Another way to look at it is that the hardware has a very efficient word-level message-passing interface built-in, via shared memory. Why not take advantage of it?
- lmm 8y ago
  > Another way to look at it is that the hardware has a very efficient word-level message-passing interface built-in, via shared memory. Why not take advantage of it? Because shared mutable state is the root of all evil, or at least very difficult to reason about. Better to have clear owners for all state; if you need to conduct multiple operations on the same data you should move the code to where the data is rather than sharing the data between different processing units (this has long been a truism of distributed system design; modern multicore processors behave much the same as distributed systems if you zoom in enough, and the same techniques are appropriate).
  - pcwalton 8y ago
    It may be "the root of all evil", but often there's no faster way to solve the problem. We need to accept that shared mutable state is sometimes necessary and work on how to tame it to be maintainable and usable.
    - a1369209993 8y ago
      Shared mutable state is like goto; yes, it is sometimes necessary, but that only happens when there's a deficiency in your language's concurrency primitives.

> For communicating gigabytes of data back and forth you want shared memory anyway. Not really. You are better off communicating what to do with the data, rather than sharing the data.

adwn 8y ago

> You are better off communicating what to do with the data, rather than sharing the data. Not if you actually want to distribute the work over several cores. Example: Your data is partitioned into 4 segments A, B, C, and D. In step 1, node X processes A&B and node Y processes C&D; in step 2, node X processes A&D and node Y processes C&B. If A, B, C, and D are large, this is a prime example where shared memory trumps message passing in terms of performance. If you think that's a contrived case: dense matrix multiplication behaves like this.

pcwalton 8y ago

Only if you don't have a data-parallel workload. Message passing is often a nonstarter for data-parallel workloads. Another way to look at it is that the hardware has a very efficient word-level message-passing interface built-in, via shared memory. Why not take advantage of it?

lmm 8y ago
> Another way to look at it is that the hardware has a very efficient word-level message-passing interface built-in, via shared memory. Why not take advantage of it? Because shared mutable state is the root of all evil, or at least very difficult to reason about. Better to have clear owners for all state; if you need to conduct multiple operations on the same data you should move the code to where the data is rather than sharing the data between different processing units (this has long been a truism of distributed system design; modern multicore processors behave much the same as distributed systems if you zoom in enough, and the same techniques are appropriate).
- pcwalton 8y ago
  It may be "the root of all evil", but often there's no faster way to solve the problem. We need to accept that shared mutable state is sometimes necessary and work on how to tame it to be maintainable and usable.
  - a1369209993 8y ago
    Shared mutable state is like goto; yes, it is sometimes necessary, but that only happens when there's a deficiency in your language's concurrency primitives.