5 ms·

He quotes the guy saying "Computers function by executing a series of instructions". This is no longer true, and becoming less true all the time. Using a mass

by Peaker 15y ago

He quotes the guy saying "Computers function by executing a series of instructions". This is no longer true, and becoming less true all the time. Using a massive amount of GPU's, and many CPU cores is no longer a simple series of instructions. Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture. If we have even more radical changes in hardware architectures, it is almost certain that the more abstract models of programming will survive better than those tied to a particular hardware architecture (Von neumann model).

tree_of_item 15y ago
Adding lots of GPUs and CPUs doesn't make that not true. Those processing units still work by executing a series of instructions, they just do some fancy things with regards to the order of instructions and communication between units. It may be more productive to think of things in terms of a more abstract, functional model of computing, but in the end, for anything to actually happen, that abstract model must be translated in to some series of instructions. I think that's what the quote was trying to get at.
- Miky 15y ago
  It would be quite possible to design a processor that is not designed around series of instructions at all, but something more akin to functional programming. In this case, everything will “actually happen” without any kind of series of instructions.
  - tree_of_item 15y ago
    Maybe my imagination isn't strong enough, but I don't see how a processor could work without being reducible to a series of instructions. The lowest level of "code" you give it might be more "functional", but there's a direct mapping between that and a sequence of states the processor goes through to actually do anything. In looking for such a thing, I've seen http://en.wikipedia.org/wiki/SECD_machine http://en.wikipedia.org/wiki/SECD_machine and http://en.wikipedia.org/wiki/Graph_reduction_machine http://en.wikipedia.org/wiki/Graph_reduction_machine, but I'm not sure either of these fit your description. Do you have experience with them or another machine you could share?
wladimir 15y ago
Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture I have read this many times over the years and it sounds perfect. It would be great to have some form of programming that maps to parallel architectures without headaches and bug-prone extra work. Still, for GPU programming we're mainly stuck with things like OpenCL/CUDA, which are imperative like ever, and it's a lot of work to port non-trivial programs to them. Also, even with Haskell/Erlang/etc AFAIK you still have to do special handling to get programs to be efficient with manycore parallelism. And that's not even talking about GPU. Is this system ever implemented or does it just exist in theory?
- gmt2027 15y ago
  The basic problem in porting to OpenCL/CUDA is identifying the mapping and reduction operations buried in imperative code. That these data-parallel operations are fundamentally functional is clearly reflected in the primitives that the Thrust and CUDPP libraries provide. The real challenge emerges during program restructuring and performance tuning. This requires intimate architectural knowledge and frequent trade-offs between conflicting optimisations.
  - wladimir 15y ago
    So we're already doing 'fundamentally functional' operations? That's an interesting way to look at it, but it's still driven from an imperative language. The grandparent post talked about functional programming in a functional programming language on a GPU. Is Theano a step in this direction? ( http://deeplearning.net/software/theano/ http://deeplearning.net/software/theano/ )
- sedachv 15y ago
  All of this work has already been done at the first massively parallel (64k processors in early 1980s) supercomputer startup: Danny Hillis' Thinking Machines. Look at StarLisp, Paralations, and NESL.
- anamax 15y ago

by Peaker 15y ago

tree_of_item 15y ago

Adding lots of GPUs and CPUs doesn't make that not true. Those processing units still work by executing a series of instructions, they just do some fancy things with regards to the order of instructions and communication between units. It may be more productive to think of things in terms of a more abstract, functional model of computing, but in the end, for anything to actually happen, that abstract model must be translated in to some series of instructions. I think that's what the quote was trying to get at.

Miky 15y ago
It would be quite possible to design a processor that is not designed around series of instructions at all, but something more akin to functional programming. In this case, everything will “actually happen” without any kind of series of instructions.
- tree_of_item 15y ago
  Maybe my imagination isn't strong enough, but I don't see how a processor could work without being reducible to a series of instructions. The lowest level of "code" you give it might be more "functional", but there's a direct mapping between that and a sequence of states the processor goes through to actually do anything. In looking for such a thing, I've seen http://en.wikipedia.org/wiki/SECD_machine http://en.wikipedia.org/wiki/SECD_machine and http://en.wikipedia.org/wiki/Graph_reduction_machine http://en.wikipedia.org/wiki/Graph_reduction_machine, but I'm not sure either of these fit your description. Do you have experience with them or another machine you could share?

wladimir 15y ago

Then, the more abstract model of functional programming allows the implementation more freedom when distributing work on the many cores/GPUs, and can make use of this new architecture I have read this many times over the years and it sounds perfect. It would be great to have some form of programming that maps to parallel architectures without headaches and bug-prone extra work. Still, for GPU programming we're mainly stuck with things like OpenCL/CUDA, which are imperative like ever, and it's a lot of work to port non-trivial programs to them. Also, even with Haskell/Erlang/etc AFAIK you still have to do special handling to get programs to be efficient with manycore parallelism. And that's not even talking about GPU. Is this system ever implemented or does it just exist in theory?

gmt2027 15y ago
The basic problem in porting to OpenCL/CUDA is identifying the mapping and reduction operations buried in imperative code. That these data-parallel operations are fundamentally functional is clearly reflected in the primitives that the Thrust and CUDPP libraries provide. The real challenge emerges during program restructuring and performance tuning. This requires intimate architectural knowledge and frequent trade-offs between conflicting optimisations.
- wladimir 15y ago
  So we're already doing 'fundamentally functional' operations? That's an interesting way to look at it, but it's still driven from an imperative language. The grandparent post talked about functional programming in a functional programming language on a GPU. Is Theano a step in this direction? ( http://deeplearning.net/software/theano/ http://deeplearning.net/software/theano/ )
sedachv 15y ago
All of this work has already been done at the first massively parallel (64k processors in early 1980s) supercomputer startup: Danny Hillis' Thinking Machines. Look at StarLisp, Paralations, and NESL.
anamax 15y ago