4 ms·

Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use o

Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture I have read this many times over the years and it sounds perfect. It would be great to have some form of programming that maps to parallel architectures without headaches and bug-prone extra work. Still, for GPU programming we're mainly stuck with things like OpenCL/CUDA, which are imperative like ever, and it's a lot of work to port non-trivial programs to them. Also, even with Haskell/Erlang/etc AFAIK you still have to do special handling to get programs to be efficient with manycore parallelism. And that's not even talking about GPU. Is this system ever implemented or does it just exist in theory?

gmt2027 15y ago
The basic problem in porting to OpenCL/CUDA is identifying the mapping and reduction operations buried in imperative code. That these data-parallel operations are fundamentally functional is clearly reflected in the primitives that the Thrust and CUDPP libraries provide. The real challenge emerges during program restructuring and performance tuning. This requires intimate architectural knowledge and frequent trade-offs between conflicting optimisations.
- wladimir 15y ago
  So we're already doing 'fundamentally functional' operations? That's an interesting way to look at it, but it's still driven from an imperative language. The grandparent post talked about functional programming in a functional programming language on a GPU. Is Theano a step in this direction? ( http://deeplearning.net/software/theano/ http://deeplearning.net/software/theano/ )
sedachv 15y ago
All of this work has already been done at the first massively parallel (64k processors in early 1980s) supercomputer startup: Danny Hillis' Thinking Machines. Look at StarLisp, Paralations, and NESL.
anamax 15y ago
>> Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture > It would be great to have some form of programming that maps to parallel architectures without headaches and bug-prone extra work. One of the advantages of age is that you can remember instead of predict. There are three problems - finding enough work that can be done in parallel, controlling the amount of parallel work that you find (yes, you can find too much), and managing the overhead. Functional languages help superficially with the first but the latter two are important as well. One reason why I write "superficially" is that writing in a functional language isn't enough - you have to write programs with the correct dependencies as well. Functional languages merely make it easier for the compiler/run-time to identify the dependencies in the program, which is a very different thing.
- Peaker 15y ago
  Functional programs encode their dependencies as arguments/return-values. In Haskell, using the par combinator is guaranteed to only affect performance, and not the actual result of the program. This is already a huge boost.
  - anamax 15y ago
    > This is already a huge boost. Like I said, I've been to this rodeo before. Heck, I even drank the Kool-aid one of the previous times that functional languages were going to enable parallel execution. Since there's nothing new this time, I think that it's safe to say that the result will be the same. Easy to find and nicely separable dependencies may be necessary, but they aren't anywhere close to sufficient. (BTW - the relevant criteria is not "huge boost" but significant boost, and I don't mean distiguishable from random chance.)
    - Peaker 15y ago
      What functional languages did you see before? Were they pure? Could they guarantee that parallelism did not affect the semantics of their programs like Haskell can? Did you actually see the use of the "par" combinator?

gmt2027 15y ago

The basic problem in porting to OpenCL/CUDA is identifying the mapping and reduction operations buried in imperative code. That these data-parallel operations are fundamentally functional is clearly reflected in the primitives that the Thrust and CUDPP libraries provide. The real challenge emerges during program restructuring and performance tuning. This requires intimate architectural knowledge and frequent trade-offs between conflicting optimisations.

wladimir 15y ago
So we're already doing 'fundamentally functional' operations? That's an interesting way to look at it, but it's still driven from an imperative language. The grandparent post talked about functional programming in a functional programming language on a GPU. Is Theano a step in this direction? ( http://deeplearning.net/software/theano/ http://deeplearning.net/software/theano/ )

sedachv 15y ago

All of this work has already been done at the first massively parallel (64k processors in early 1980s) supercomputer startup: Danny Hillis' Thinking Machines. Look at StarLisp, Paralations, and NESL.

anamax 15y ago

>> Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture > It would be great to have some form of programming that maps to parallel architectures without headaches and bug-prone extra work. One of the advantages of age is that you can remember instead of predict. There are three problems - finding enough work that can be done in parallel, controlling the amount of parallel work that you find (yes, you can find too much), and managing the overhead. Functional languages help superficially with the first but the latter two are important as well. One reason why I write "superficially" is that writing in a functional language isn't enough - you have to write programs with the correct dependencies as well. Functional languages merely make it easier for the compiler/run-time to identify the dependencies in the program, which is a very different thing.

Peaker 15y ago
Functional programs encode their dependencies as arguments/return-values. In Haskell, using the par combinator is guaranteed to only affect performance, and not the actual result of the program. This is already a huge boost.
- anamax 15y ago
  > This is already a huge boost. Like I said, I've been to this rodeo before. Heck, I even drank the Kool-aid one of the previous times that functional languages were going to enable parallel execution. Since there's nothing new this time, I think that it's safe to say that the result will be the same. Easy to find and nicely separable dependencies may be necessary, but they aren't anywhere close to sufficient. (BTW - the relevant criteria is not "huge boost" but significant boost, and I don't mean distiguishable from random chance.)
  - Peaker 15y ago
    What functional languages did you see before? Were they pure? Could they guarantee that parallelism did not affect the semantics of their programs like Haskell can? Did you actually see the use of the "par" combinator?