4 ms·

I wonder how long it will be before compilers/interpreters of async-aware languages just do this by default. CPUs and low-level language compilers jump through

I wonder how long it will be before compilers/interpreters of async-aware languages just do this by default. CPUs and low-level language compilers jump through all kinds of hoops of out-of-order execution, branch prediction, caching, parallel execution, etc. I picture a day maybe 10 years from now where developers in most languages don't even have to think about these things. All the old-timers will still be structuring their code "as though it didn't exist" whereas the new kids will fly along without even thinking about it. Kind of like garbage collection the first few years.

eru 9y ago
Even in Haskell you still have to write some boilerplate to get this by default. Languages that don't separate pure computation from IO (and other side-effects) make it even harder on the compiler. So to answer your question: implementations will do this by default only after pure/constant will become the default for all functions/variables, with side-effecting/mutable clearly marked.
- hughperkins 9y ago
  If one looks at it from a 'what a today's compiler can do', then sure, one needs to statically declare everything. If one looks at things from a 'any technical task can be handled by machine learning, sooner or later' point of view, there seems to be no obvious reason why parallelization, which is a purely technical task, not like say, writing music, could not sooner or later be handled by machine learning algorithms?
  - eru 9y ago
    Writing music is a pretty technical task, actually. Especially if you optimize for something as tangible as: please human ears, or "fit within the actions in the movie / computer game". The idea here is that we want to find a way to make automatic parallelization of computer programmes happen as much as possible long before we've solved general AI. So even in a language like Haskell you still have to pay some attention to make your code amenable to parallelism. See Guy Steele's advice (see https://vimeo.com/6624203 https://vimeo.com/6624203).
jerf 9y ago
Analysis of real software that has looked for the maximum theoretical implicit concurrency has found that software to have its implicit concurrency level top out very quickly, often not even making it up to 2x. You don't hear about this today not because nobody has tried it, but because it's been tried and it hasn't been found worth the bother. Amdahl's law is a real jerk sometimes. If you want concurrent code, you're going to have to write it with an awareness of concurrency issues. There are things that languages can do to make it easier, but in terms of doing it automatically even that hasn't had a great track record. Fortress [1] took a good run at it, but ultimately didn't seem to get that far, at least relative to their ambitions. Personally I suspect that while Haskell may not be that language today, and may not be that language ever necessarily, that the "magically concurrent" language of the future is going to end up looking a lot like Haskell, in particular, with the ability and need to consider the mathematical properties of your code for things like associativity and commutativity. It's going to take a pretty major sea change in a whole lot of attitudes before "the youngsters" are just casually reeling off group theory names while the "old fogies" are stuck going "What's a monoid again?" Not impossible, but hard to imagine how we get to there from here. [1]: https://en.wikipedia.org/wiki/Fortress_(programming_language) https://en.wikipedia.org/wiki/Fortress_(programming_language...
- solidsnack9000 9y ago
  Sadly it seems like the youngsters are going in the opposite direction these days. It's the old fogies who have to keep saying, I've got a functor, I've got a monoid, I've got a monoid functor = monad.
  - eru 9y ago
    Aren't Applicative Functors still all the rage? Monads are not all that well suited to automatic parallelization (as Simon Marlow points out, too).
    - solidsnack9000 9y ago
      Applicative seems to be the way of the future, yes.
DSrcl 9y ago
It's unsafe for a compiler to do this in general (i.e. without annotations) because it can't determine dependencies that are external to the program -- e.g. `one=get(); two=get()`. The dependency between one and two is not obvious to a compiler when IO is involved, so it has to assume the two `gets' has to be executed sequentially.
- daxfohl 9y ago
  Doh, my head-slap moment of the day. I totally overlooked that.
- eyelidlessness 9y ago
  > It's unsafe for a compiler to do this in general (i.e. without annotations) because it can't determine dependencies that are external to the program As I understood the talk, Haxl doesn't address this either. It depends on you describing an IO operation as a type with set of functionality that can be used to identify those dependencies. > e.g. `one=get(); two=get()`. The dependency between one and two is not obvious to a compiler when IO is involved, so it has to assume the two `gets' has to be executed sequentially. A "pure" language will address a lot of this due to referential transparency. Which is to say that if `get` takes no arguments, its instructions for where and how to perform an IO operation are entirely static. Given your literal example, the only way the result could be different is if there were some side effect or if a source of data required by `get` could change during execution time. Haxl's takes this a bit further by suggesting that if you are performing the same IO request twice "at the same time", you expect to get the same result both times, so it memoizes the request for the duration of your set of IO operations. Within a `do`, as far as I could grok from the talk and glancing at documentation; it's worth noting I don't know Haskell and have never heard of Haxl before today.
abecedarius 9y ago
Promise pipelining is an optimization of that sort: https://capnproto.org/rpc.html https://capnproto.org/rpc.html which was in the E language around 20 years ago: http://erights.org/elib/distrib/pipeline.html http://erights.org/elib/distrib/pipeline.html
CyberDildonics 9y ago
CPUs already have multiple concurrent instructions running. When people write software that runs serially on a macro scale, that is exactly what they get.

eru 9y ago

Even in Haskell you still have to write some boilerplate to get this by default. Languages that don't separate pure computation from IO (and other side-effects) make it even harder on the compiler. So to answer your question: implementations will do this by default only after pure/constant will become the default for all functions/variables, with side-effecting/mutable clearly marked.

hughperkins 9y ago
If one looks at it from a 'what a today's compiler can do', then sure, one needs to statically declare everything. If one looks at things from a 'any technical task can be handled by machine learning, sooner or later' point of view, there seems to be no obvious reason why parallelization, which is a purely technical task, not like say, writing music, could not sooner or later be handled by machine learning algorithms?
- eru 9y ago
  Writing music is a pretty technical task, actually. Especially if you optimize for something as tangible as: please human ears, or "fit within the actions in the movie / computer game". The idea here is that we want to find a way to make automatic parallelization of computer programmes happen as much as possible long before we've solved general AI. So even in a language like Haskell you still have to pay some attention to make your code amenable to parallelism. See Guy Steele's advice (see https://vimeo.com/6624203 https://vimeo.com/6624203).

jerf 9y ago

Analysis of real software that has looked for the maximum theoretical implicit concurrency has found that software to have its implicit concurrency level top out very quickly, often not even making it up to 2x. You don't hear about this today not because nobody has tried it, but because it's been tried and it hasn't been found worth the bother. Amdahl's law is a real jerk sometimes. If you want concurrent code, you're going to have to write it with an awareness of concurrency issues. There are things that languages can do to make it easier, but in terms of doing it automatically even that hasn't had a great track record. Fortress [1] took a good run at it, but ultimately didn't seem to get that far, at least relative to their ambitions. Personally I suspect that while Haskell may not be that language today, and may not be that language ever necessarily, that the "magically concurrent" language of the future is going to end up looking a lot like Haskell, in particular, with the ability and need to consider the mathematical properties of your code for things like associativity and commutativity. It's going to take a pretty major sea change in a whole lot of attitudes before "the youngsters" are just casually reeling off group theory names while the "old fogies" are stuck going "What's a monoid again?" Not impossible, but hard to imagine how we get to there from here. [1]: https://en.wikipedia.org/wiki/Fortress_(programming_language) https://en.wikipedia.org/wiki/Fortress_(programming_language...

solidsnack9000 9y ago
Sadly it seems like the youngsters are going in the opposite direction these days. It's the old fogies who have to keep saying, I've got a functor, I've got a monoid, I've got a monoid functor = monad.
- eru 9y ago
  Aren't Applicative Functors still all the rage? Monads are not all that well suited to automatic parallelization (as Simon Marlow points out, too).
  - solidsnack9000 9y ago
    Applicative seems to be the way of the future, yes.

DSrcl 9y ago

It's unsafe for a compiler to do this in general (i.e. without annotations) because it can't determine dependencies that are external to the program -- e.g. `one=get(); two=get()`. The dependency between one and two is not obvious to a compiler when IO is involved, so it has to assume the two `gets' has to be executed sequentially.

daxfohl 9y ago
Doh, my head-slap moment of the day. I totally overlooked that.
eyelidlessness 9y ago
> It's unsafe for a compiler to do this in general (i.e. without annotations) because it can't determine dependencies that are external to the program As I understood the talk, Haxl doesn't address this either. It depends on you describing an IO operation as a type with set of functionality that can be used to identify those dependencies. > e.g. `one=get(); two=get()`. The dependency between one and two is not obvious to a compiler when IO is involved, so it has to assume the two `gets' has to be executed sequentially. A "pure" language will address a lot of this due to referential transparency. Which is to say that if `get` takes no arguments, its instructions for where and how to perform an IO operation are entirely static. Given your literal example, the only way the result could be different is if there were some side effect or if a source of data required by `get` could change during execution time. Haxl's takes this a bit further by suggesting that if you are performing the same IO request twice "at the same time", you expect to get the same result both times, so it memoizes the request for the duration of your set of IO operations. Within a `do`, as far as I could grok from the talk and glancing at documentation; it's worth noting I don't know Haskell and have never heard of Haxl before today.

abecedarius 9y ago

Promise pipelining is an optimization of that sort: https://capnproto.org/rpc.html https://capnproto.org/rpc.html which was in the E language around 20 years ago: http://erights.org/elib/distrib/pipeline.html http://erights.org/elib/distrib/pipeline.html

CyberDildonics 9y ago

CPUs already have multiple concurrent instructions running. When people write software that runs serially on a macro scale, that is exactly what they get.