4 ms·

Laziness (lazy IO, in this case) has nothing to do with a language being functional or not. It has everything to do with a language being strict or lazy. There

Laziness (lazy IO, in this case) has nothing to do with a language being functional or not. It has everything to do with a language being strict or lazy. There are strict functional languages. Most are, in fact.

m_mueller 10y ago
Lazy/strict is just one property that can be problematic in a high performance context. The biggest one to me is mutability. Basically it's a simple test: Can I swap pointers? (e.g. 'model_current' and 'model_next_timestep'). If not, I can't use it, it would slow down numerical solvers tremendously to allocate and free the required memory for each step. However, if I can just have the time iteration in an imperative shell that allows pointer swapping, while keeping the interesting mathematics in a purely functional core, that would be an interesting architecture (because it could make use of inherent parallelism in a better way). So far I haven't seen anything like that becoming truly competitive with Fortran/C/C++ in the HPC space, which I find a shame. Programming for HPC should be like programming in the future, as Alan Kay likes to say, but it seems to me there is a stark (and IMO unnecessary) disconnect between the worlds of HPC and desktop/web programming today.
- fulafel 10y ago
  There are various ways get the effect (with similar space and performance chacteristic), but there are different approaches for different FP languages. For example, persistent data structures, non-pure primitives (eg Clojure's atoms), monads, etc. For Haskell, these sound relevant if you're set on using strict dense matrices: https://wiki.haskell.org/Monad/ST https://wiki.haskell.org/Monad/ST https://hackage.haskell.org/package/bed-and-breakfast https://hackage.haskell.org/package/bed-and-breakfast I suspect HPC will always be more gnarly than elegant, since by definition HPC is about spending money on making code go fast in very specialized apps.
  - m_mueller 10y ago
    could you expand on persistent data structures? How do they give me 'free' memory with neither allocation nor mutable data? Or is that outside of pure FP?
    - fulafel 10y ago
      So persistent data structures are conceptually just an extension of the classic lisp cells: You can have a pointer to (a, b, c) and a pointer to (a, b, c, d) simultaneously, without using twice the memory or time. This can be extended to trees. And you can for example make an efficient tree-backed vector, with chunks of values at the nodes. With some thought you can even get good atomic properties so you can have safe parallel operations on the same persistent vector. Here's a good start about Clojure's persistent vector: http://hypirion.com/musings/understanding-persistent-vector-pt-1 http://hypirion.com/musings/understanding-persistent-vector-...
- mbrock 10y ago
  Pointer swapping would probably just be swapping the arguments to your loop function.
  - m_mueller 10y ago
    Well one needs to become the output, so if you only have immutable data structures it's afaik impossible.

m_mueller 10y ago

Lazy/strict is just one property that can be problematic in a high performance context. The biggest one to me is mutability. Basically it's a simple test: Can I swap pointers? (e.g. 'model_current' and 'model_next_timestep'). If not, I can't use it, it would slow down numerical solvers tremendously to allocate and free the required memory for each step. However, if I can just have the time iteration in an imperative shell that allows pointer swapping, while keeping the interesting mathematics in a purely functional core, that would be an interesting architecture (because it could make use of inherent parallelism in a better way). So far I haven't seen anything like that becoming truly competitive with Fortran/C/C++ in the HPC space, which I find a shame. Programming for HPC should be like programming in the future, as Alan Kay likes to say, but it seems to me there is a stark (and IMO unnecessary) disconnect between the worlds of HPC and desktop/web programming today.

fulafel 10y ago
There are various ways get the effect (with similar space and performance chacteristic), but there are different approaches for different FP languages. For example, persistent data structures, non-pure primitives (eg Clojure's atoms), monads, etc. For Haskell, these sound relevant if you're set on using strict dense matrices: https://wiki.haskell.org/Monad/ST https://wiki.haskell.org/Monad/ST https://hackage.haskell.org/package/bed-and-breakfast https://hackage.haskell.org/package/bed-and-breakfast I suspect HPC will always be more gnarly than elegant, since by definition HPC is about spending money on making code go fast in very specialized apps.
- m_mueller 10y ago
  could you expand on persistent data structures? How do they give me 'free' memory with neither allocation nor mutable data? Or is that outside of pure FP?
  - fulafel 10y ago
    So persistent data structures are conceptually just an extension of the classic lisp cells: You can have a pointer to (a, b, c) and a pointer to (a, b, c, d) simultaneously, without using twice the memory or time. This can be extended to trees. And you can for example make an efficient tree-backed vector, with chunks of values at the nodes. With some thought you can even get good atomic properties so you can have safe parallel operations on the same persistent vector. Here's a good start about Clojure's persistent vector: http://hypirion.com/musings/understanding-persistent-vector-pt-1 http://hypirion.com/musings/understanding-persistent-vector-...
mbrock 10y ago
Pointer swapping would probably just be swapping the arguments to your loop function.
- m_mueller 10y ago
  Well one needs to become the output, so if you only have immutable data structures it's afaik impossible.