5 ms·

> They were built to handle concurrency down to the core. And they weren't built to handle data parallelism at all.

rdtsc 10y ago
Not sure I know what you mean by "data parallelism". I wrote code not too long ago which had multiple processes reference the same binary blob. I have enough CPUs to ensure some processes were actually accessing it in parallel. In other case, used an ETS table to have multiple processes read and write data to and from it. Or maybe to put it another way, if you think I am not already doing data parallelism, sell me on why I need data parallelism.
- catnaroek 10y ago
  > reference the same binary blob Is there some way to share data whose structure is actually understood and checked (statically or dynamically) by the language? > to have multiple processes read and write data to and from it. How did you coordinate those reads and writes? A writer not having exclusive access to what it's writing to, at the time of the writing... sounds... very bad. > sell me on why I need data parallelism. Just to name one example: machine learning. If you're feeding real-time data to a ML algorithm, chances are you'll have to perform multiple independent calculations that use the same data as input. To speed up those calculations, you need to perform them in parallel.
di4na 10y ago
Nothing is. There was an interesting discussion between Simon Peyton Jones and Joe Armstrong on that. https://www.infoq.com/interviews/armstrong-peyton-jones-erlang-haskell https://www.infoq.com/interviews/armstrong-peyton-jones-erla... In general, the thing is... parallelism is hard. If not impossible to do right from automated tools.
- catnaroek 10y ago
  First we have to make it safe: make it inexpressible to do parallelism wrong. Only then can we discuss automating it. There is no value whatsoever in automating the wrong thing.
  - di4na 10y ago
    What do you mean by that. Because doing parallelism wrong is easy : do it. Parrallelism mean doing the same thing at the same time. The only way to not do it wrong is to not do it.
    - catnaroek 10y ago
      > Parrallelism mean doing the same thing at the same time. Yeah, but how to do it without making the program state go kaboom? > What do you mean by that. “Make it impossible to write to a data structure without (at least temporary) exclusive access to it.” At the risk of oversimplifying, there are basically three ways to do it: (0) Use data structures that are immutable after they are first written to. Pros: very easy to understand and use. Cons: limited to pure parallelism, without concurrency. (1) Clone all data structures when you need to share them. Pros: very easy to understand and use. Cons: not all data structures can be meaningfully cloned, e.g., how do you clone a file descriptor? (2) Use a static analysis (such as type checking) to enforce the precondition that writers must have exclusive access at the time of the writing. Pros: can handle both parallelism and concurrency. Cons: requires the ability to write code that a static analysis tool can understand. Since the last one is by far the most rare approach, let me give an example of it. Quoting the Rust book: > Here’s the rules about borrowing in Rust: > First, any borrow must last for a scope no greater than that of the owner. Second, you may have one or the other of these two kinds of borrows, but not both at the same time: > - one or more references (&T) to a resource, > - exactly one mutable reference (&mut T). https://doc.rust-lang.org/stable/book/references-and-borrowing.html https://doc.rust-lang.org/stable/book/references-and-borrowi...
      - di4na 10y ago
        Sure but 1 and 2 end up being the same. Because when you reach the limits you clone. The thing with parallelism is not reading. It is writing. So the best way ... is to have isolated computation then reconnect them at the end... and yes it is hard.. But parallelism has few interest in my view. Concurrency is far more interested.

rdtsc 10y ago

Not sure I know what you mean by "data parallelism". I wrote code not too long ago which had multiple processes reference the same binary blob. I have enough CPUs to ensure some processes were actually accessing it in parallel. In other case, used an ETS table to have multiple processes read and write data to and from it. Or maybe to put it another way, if you think I am not already doing data parallelism, sell me on why I need data parallelism.

catnaroek 10y ago
> reference the same binary blob Is there some way to share data whose structure is actually understood and checked (statically or dynamically) by the language? > to have multiple processes read and write data to and from it. How did you coordinate those reads and writes? A writer not having exclusive access to what it's writing to, at the time of the writing... sounds... very bad. > sell me on why I need data parallelism. Just to name one example: machine learning. If you're feeding real-time data to a ML algorithm, chances are you'll have to perform multiple independent calculations that use the same data as input. To speed up those calculations, you need to perform them in parallel.

di4na 10y ago

Nothing is. There was an interesting discussion between Simon Peyton Jones and Joe Armstrong on that. https://www.infoq.com/interviews/armstrong-peyton-jones-erlang-haskell https://www.infoq.com/interviews/armstrong-peyton-jones-erla... In general, the thing is... parallelism is hard. If not impossible to do right from automated tools.

catnaroek 10y ago
First we have to make it safe: make it inexpressible to do parallelism wrong. Only then can we discuss automating it. There is no value whatsoever in automating the wrong thing.
- di4na 10y ago
  What do you mean by that. Because doing parallelism wrong is easy : do it. Parrallelism mean doing the same thing at the same time. The only way to not do it wrong is to not do it.
  - catnaroek 10y ago
    > Parrallelism mean doing the same thing at the same time. Yeah, but how to do it without making the program state go kaboom? > What do you mean by that. “Make it impossible to write to a data structure without (at least temporary) exclusive access to it.” At the risk of oversimplifying, there are basically three ways to do it: (0) Use data structures that are immutable after they are first written to. Pros: very easy to understand and use. Cons: limited to pure parallelism, without concurrency. (1) Clone all data structures when you need to share them. Pros: very easy to understand and use. Cons: not all data structures can be meaningfully cloned, e.g., how do you clone a file descriptor? (2) Use a static analysis (such as type checking) to enforce the precondition that writers must have exclusive access at the time of the writing. Pros: can handle both parallelism and concurrency. Cons: requires the ability to write code that a static analysis tool can understand. Since the last one is by far the most rare approach, let me give an example of it. Quoting the Rust book: > Here’s the rules about borrowing in Rust: > First, any borrow must last for a scope no greater than that of the owner. Second, you may have one or the other of these two kinds of borrows, but not both at the same time: > - one or more references (&T) to a resource, > - exactly one mutable reference (&mut T). https://doc.rust-lang.org/stable/book/references-and-borrowing.html https://doc.rust-lang.org/stable/book/references-and-borrowi...
    - di4na 10y ago
      Sure but 1 and 2 end up being the same. Because when you reach the limits you clone. The thing with parallelism is not reading. It is writing. So the best way ... is to have isolated computation then reconnect them at the end... and yes it is hard.. But parallelism has few interest in my view. Concurrency is far more interested.