4 ms·

It's interesting that in Norvig's talk you linked, while praising python, he also mentions that he is a supporter of the functional programming style for scient

It's interesting that in Norvig's talk you linked, while praising python, he also mentions that he is a supporter of the functional programming style for scientific work, both because it more transparently mirrors the mathematical formulation of the algorithm, and because it should make the use of massively parallel computations easier. I concede that there are currently few implementations of massively parallel, functional code (MapReduce and possibly Data Parallel Haskell come to mind) and it's true that the MPI-based Single Program, Multiple Data paradigm is dominant for now, the functional paradigm being but a very promising newcomer. Given that, and the fact that languages such as Lisp, Haskell and OCaml are more aimed at the functional paradigm, wouldn't you agree that there is possibly a general shift going on in scientific programming in general, from Fortran/C(++) imperative style, via python, towards finally any of Haskell, Lisp and/or OCaml? I don't really agree with your implication that the (at first) obscure way of doing things functionally makes the work less open, and that this makes it less desirable. If everyone worked in exactly the same way, communication would be easier, but there would never be any disruptive change. I think letting everyone figure out for themselves what language works best for his/her appliciation has more merits than trying to enforce a standard, be it python + numpy & scipy, or anything else. If there turns out to be one clear winner, people will gravitate to that automatically, but you shouldn't argue for harmonization for its own sake.

chancho 17y ago
> and because it should make the use of massively parallel computations easier. I concede that there are currently few implementations of massively parallel, functional code (MapReduce and possibly Data Parallel Haskell come to mind) Not only are there few, their numbers are growing much more slowly than traditional C/Fortan/MPI environments. The "massive" in massively parallel is getting more and more massive every year. Running computations at full speed at these scales, 100K+ cores in multiple levels of hierarchy (same die, same board, same blade, same switch, etc) is a hugely mundane architectural problem. Not only is there no architecture besides MPI which has nearly as much effort invested in this problem, none are currently making the investment, so none have a chance to catch up within the next decade. The only way you will see FP doing massive parallelism at speeds even close to current best is if they wrap the C MPI interface, and I have yet to see any functional message-passing APIs which aren't tarted-up imperative environments. (If anyone knows any please share.) At it's core, message passing programming involves managing data, deciding where it resides and where it needs to go, which is kind of antithetical to FP. You mention MapReduce, which I think is more of a data-center thing subtly but fundamentally different from HPC, involving less computation and more I/O. FP has a good shot there (cf. Erlang) but not much chance at cracking the HPC market.
- graphene 17y ago
  The only way you will see FP doing massive parallelism at speeds even close to current best is if they wrap the C MPI interface, and I have yet to see any functional message-passing APIs which aren't tarted-up imperative environments True, I would not be surprised if it evolved that way, but according to my (modest) knowlegde, it is possible for imperative routines to have a purely functional interface. The extreme case of course is the fact that any functional high-level code needs to be translated to assembly code to run at all, but I don't see why that boundary can't be on a higher level; As long as the developer programs functionally (and derives the benefits from that), does it matter that his code is being translated first into imperative MPI-ified C, and then assembly? This is all provided, of course, that you can actually leverage the parallelism-related benefits of FP this way, which I guess is not known as of yet. You mentioned "functional message-passing APIs which [are] tarted-up imperative environments", care to give an example? I'm intrigued... As for MapReduce, of course it's different from what the average HPC machine is used for, but being an embarrassingly parallel problem, it's one of the first problems where the FP approach has been shown to work. In other (eg typical HPC) applications, there's lots of work ahead in rethinking the algorithms so they can be expressed functionally, before you can even begin contemplating how to use the hardware.
sophacles 17y ago
I have always wanted: A. an easy way to call into Haskell or some other fp from python. B. the ability to embedd the above so that I can have equations in a nice mathy looking form but the rest in python. C. all of the above w/ R too.
- tetha 17y ago
  This post is a very, very nice proof of my feeling, that FP is usually not used due to the hassle of getting data into the program in a functional language. I mean, c'mon. Who was not confused by Haskells IO? And now, compare this with "foo = input()", or "bar = scanf("%d");". Bam. All the benefit of nice functional notation for the algorithms superseded by the hassle of getting data from the outside into the program (or, the very small hassle (seen relatively) to do so in imperative languages.

chancho 17y ago

> and because it should make the use of massively parallel computations easier. I concede that there are currently few implementations of massively parallel, functional code (MapReduce and possibly Data Parallel Haskell come to mind) Not only are there few, their numbers are growing much more slowly than traditional C/Fortan/MPI environments. The "massive" in massively parallel is getting more and more massive every year. Running computations at full speed at these scales, 100K+ cores in multiple levels of hierarchy (same die, same board, same blade, same switch, etc) is a hugely mundane architectural problem. Not only is there no architecture besides MPI which has nearly as much effort invested in this problem, none are currently making the investment, so none have a chance to catch up within the next decade. The only way you will see FP doing massive parallelism at speeds even close to current best is if they wrap the C MPI interface, and I have yet to see any functional message-passing APIs which aren't tarted-up imperative environments. (If anyone knows any please share.) At it's core, message passing programming involves managing data, deciding where it resides and where it needs to go, which is kind of antithetical to FP. You mention MapReduce, which I think is more of a data-center thing subtly but fundamentally different from HPC, involving less computation and more I/O. FP has a good shot there (cf. Erlang) but not much chance at cracking the HPC market.

graphene 17y ago
The only way you will see FP doing massive parallelism at speeds even close to current best is if they wrap the C MPI interface, and I have yet to see any functional message-passing APIs which aren't tarted-up imperative environments True, I would not be surprised if it evolved that way, but according to my (modest) knowlegde, it is possible for imperative routines to have a purely functional interface. The extreme case of course is the fact that any functional high-level code needs to be translated to assembly code to run at all, but I don't see why that boundary can't be on a higher level; As long as the developer programs functionally (and derives the benefits from that), does it matter that his code is being translated first into imperative MPI-ified C, and then assembly? This is all provided, of course, that you can actually leverage the parallelism-related benefits of FP this way, which I guess is not known as of yet. You mentioned "functional message-passing APIs which [are] tarted-up imperative environments", care to give an example? I'm intrigued... As for MapReduce, of course it's different from what the average HPC machine is used for, but being an embarrassingly parallel problem, it's one of the first problems where the FP approach has been shown to work. In other (eg typical HPC) applications, there's lots of work ahead in rethinking the algorithms so they can be expressed functionally, before you can even begin contemplating how to use the hardware.

sophacles 17y ago

I have always wanted: A. an easy way to call into Haskell or some other fp from python. B. the ability to embedd the above so that I can have equations in a nice mathy looking form but the rest in python. C. all of the above w/ R too.

tetha 17y ago
This post is a very, very nice proof of my feeling, that FP is usually not used due to the hassle of getting data into the program in a functional language. I mean, c'mon. Who was not confused by Haskells IO? And now, compare this with "foo = input()", or "bar = scanf("%d");". Bam. All the benefit of nice functional notation for the algorithms superseded by the hassle of getting data from the outside into the program (or, the very small hassle (seen relatively) to do so in imperative languages.