5 ms·

I'm all for trying something unfamiliar, and I could even argue for Haskell that way, but from a different angle. I would say that adopting a language for the

by mtrimpe 12y ago

I'm all for trying something unfamiliar, and I could even argue for Haskell that way, but from a different angle. I would say that adopting a language for the next 10 years that doesn't have purely functional data structures at its core is a fool's errand. Persistent data structures, i.e. efficiently storing every version of your data structure, is the natural reflection of the new reality in computing where: * storage is cheap * random reads are cheap * random writes are expensive * serial writes are cheap Whether you adopt this through Haskell, Clojure, scala or even Bloom e.a. is fairly inconsequential. What will matter is that persistent data structures will inevitable become commonplace over the next decade and that techniques for leveraging it (such as functional reactive programming) will be adopted in one form or another. @jacquesm: I live in Amsterdam so I can try to explain it to you in person once; virtually everyone still underestimates the impact this will have and (or perhaps because) it's very difficult to get that across in writing.

jacquesm 12y ago
Living in Romania right now, but I'll ping you when I'm back in NL (I go there at least once every few weeks), Clojure seems to be the most approachable from the languages you list. Besides your listed reasons mutable state is becoming a bigger and bigger bottle neck as the number of cores goes up and that's why functional programming is becoming more and more important. It also makes programs much harder to write in a bug-free manner and it makes testing much more difficult (if not impossible). It's one of the core reasons why I felt so bad about dropping Erlang from the list because I feel that it gives me that plus a battle tested environment to work with. Still tempted to put it back in. Your reasons add weight to an already very difficult decision, whether or not to pursue this opportunity as a way to get more into functional programming or to stay in my 'comfort zone'. I've been toying around with FP on and off for the last couple of years and it is one of the most important revivals the computing world has seen to date (imo of course).
- lmm 12y ago
  This may be my biases speaking but I'd say Scala is a more approachable option than Clojure (and more popular, and has a first-class web framework in Play). It keeps more of the traditional OO style and syntax - you can use scalaz and write scala that looks like Haskell, but you can also write code that corresponds line-by-line to java, whereas with Clojure you're forced to make a more immediate leap to the lisp-style syntax, immediately stop using inheritance, and so on.
  - ctdavies 12y ago
    That's a major appeal of Clojure, the Lisp (not Lisp-style) syntax, the functional paradigm (and the idioms are great). I really don't want to write code that corresponds line-by-line to Java. If I wanted to do that, I'd just write Java. However, Java interop with Clojure is straightforward. Furthermore, Clojure allows one to solve problems with an "orientation" toward objects, but as one learns to write Clojure well, the language guides one toward sublime and elegant functional solutions. An important thing about Clojure, however, is that it is not tied to the JVM; The original and most well-known implementation is for the JVM, but not only is there also Clojurescript, there are implementations that compile to CLR bytecode, Python, Ruby, and Actionscript. And with EDN, the extensible data notation, not only can you have your front-end and back-end codebases entirely in Clojure, you can transfer and store your data as Clojure too. I know not everyone has the same aesthetic tastes as I do, but I do like to think that there is something beautiful about this. Finally, I must say that while Scala does indeed have a first-class web framework in Play, web frameworks are first-class objects in Clojure. Joke. Sort of. I personally do not want to have a monolithic pre-packaged web framework to bow down to; the Clojure community is a community of hackers, we like to make things out of other things, and to take things apart. Clojure and the work of its community allows me to compose my own web framework as needed very very quickly–and to drastically change it just as fast. I'm not saying you should stop doing what you like to do, but if you gave Clojure a chance...dude: it's addictive.
    - lmm 12y ago
      > I really don't want to write code that corresponds line-by-line to Java. If I wanted to do that, I'd just write Java. The code I write now is nothing like Java, but it's a question of how you get from there to here. When I started Scala I wrote what I'd write in Java. In the first week I removed the semicolons. In the second week I started to use function literals. Then I started using match/case. Later I found out about for/yield with futures. And so on. I learnt the language at my job, and I was delivering real, user-facing features in it from day 1. I don't think I could've done the same with Clojure. (Reasonable people can disagree on syntax; I find that Scala's lets you get very close to the language of the domain (e.g. see the Spray "routing DSL") and that's worth the cost it imposes on the macro system)
seanmcdirmid 12y ago
Persistent structures are almost always O log N (because they have to be trees). You can fix your branch out for better performance at the cost of more wasted space. You will never be as fast as an array. Also, sometimes you want your updates to a data structure to be seen via an alias (happens often actually), then we are back at square one again. Systems people are much more pragmatic about this: they see the trends in hardware before we PL people do, and their solution has never been functional programming. Ok, well...maybe once with MapReduce (but Jeff and Sanjay are both systems PL people).
- mtrimpe 12y ago
  At n=32 the performance costs are as negligible as dynamic typing once was (even less I'd argue.) As far as I can tell that very much puts us at the point in time where mutable state is usually a premature optimization. Adopting it successfully does require separating essential from accidental state which naturally leads to some variant of FRP being adopted to manage (re-)derivation of accidental state; which is usually the hard to explain part. I am wondering what you do see systems people doing by the way? I'm always interested in broadening my perspective on this.
  - seanmcdirmid 12y ago
    n=32 leads can lead to lots of sparseness, memory pressure, bad locality (given sparseness); its not going to fly easily as the default. The shared memory aspect is even a bigger hurdle: people share memory for a reason, its not very often accidental. FRP is hardly the answer, but I work on a competing paradigm so mark me as biased. The systems people I work with do a lot of key value stores (sharded NoSQL-style databases), there is a lot of pipelining for data parallel processing (MapReduce, Pig), and there is still a lot of classic locking going on, throughput is important. Oh, and lots of Paxos, logging, state replication, etc... The normal stuff you could find at Google (note I don't work for Google). On the HPC side, the people that really care about performance, there is some map reduce via MPI, but most of the big gains are to be had in CUDA, which is an extremely imperative programming model with very synchronized core executions and memory accesses. I've learned that whenever the FP people talk about high performance, they usually mean something else (like concurrency that isn't so slow), rather than the real high performance stuff that the big kids (systems and HPC communities) are doing.
oskarth 12y ago
> virtually everyone still underestimates the impact this will have and (or perhaps because) it's very difficult to get that across in writing. Why is it hard to get across in writing?
- mtrimpe 12y ago
  I'd say because of two reasons: a) it requires putting together a fairly large number of concepts of moderate complexity (and you never know which ones your readers already understand.) b) many of the concepts sound trivial and people usually don't grasp their power until you've listed X amounts of things it would enable. And to be clear; it's hard for me to put in writing for now ;)
nwmcsween 12y ago
Persistent structures do not map well with how computers work in reality.

by mtrimpe 12y ago

jacquesm 12y ago

Living in Romania right now, but I'll ping you when I'm back in NL (I go there at least once every few weeks), Clojure seems to be the most approachable from the languages you list. Besides your listed reasons mutable state is becoming a bigger and bigger bottle neck as the number of cores goes up and that's why functional programming is becoming more and more important. It also makes programs much harder to write in a bug-free manner and it makes testing much more difficult (if not impossible). It's one of the core reasons why I felt so bad about dropping Erlang from the list because I feel that it gives me that plus a battle tested environment to work with. Still tempted to put it back in. Your reasons add weight to an already very difficult decision, whether or not to pursue this opportunity as a way to get more into functional programming or to stay in my 'comfort zone'. I've been toying around with FP on and off for the last couple of years and it is one of the most important revivals the computing world has seen to date (imo of course).

lmm 12y ago
This may be my biases speaking but I'd say Scala is a more approachable option than Clojure (and more popular, and has a first-class web framework in Play). It keeps more of the traditional OO style and syntax - you can use scalaz and write scala that looks like Haskell, but you can also write code that corresponds line-by-line to java, whereas with Clojure you're forced to make a more immediate leap to the lisp-style syntax, immediately stop using inheritance, and so on.
- ctdavies 12y ago
  That's a major appeal of Clojure, the Lisp (not Lisp-style) syntax, the functional paradigm (and the idioms are great). I really don't want to write code that corresponds line-by-line to Java. If I wanted to do that, I'd just write Java. However, Java interop with Clojure is straightforward. Furthermore, Clojure allows one to solve problems with an "orientation" toward objects, but as one learns to write Clojure well, the language guides one toward sublime and elegant functional solutions. An important thing about Clojure, however, is that it is not tied to the JVM; The original and most well-known implementation is for the JVM, but not only is there also Clojurescript, there are implementations that compile to CLR bytecode, Python, Ruby, and Actionscript. And with EDN, the extensible data notation, not only can you have your front-end and back-end codebases entirely in Clojure, you can transfer and store your data as Clojure too. I know not everyone has the same aesthetic tastes as I do, but I do like to think that there is something beautiful about this. Finally, I must say that while Scala does indeed have a first-class web framework in Play, web frameworks are first-class objects in Clojure. Joke. Sort of. I personally do not want to have a monolithic pre-packaged web framework to bow down to; the Clojure community is a community of hackers, we like to make things out of other things, and to take things apart. Clojure and the work of its community allows me to compose my own web framework as needed very very quickly–and to drastically change it just as fast. I'm not saying you should stop doing what you like to do, but if you gave Clojure a chance...dude: it's addictive.
  - lmm 12y ago
    > I really don't want to write code that corresponds line-by-line to Java. If I wanted to do that, I'd just write Java. The code I write now is nothing like Java, but it's a question of how you get from there to here. When I started Scala I wrote what I'd write in Java. In the first week I removed the semicolons. In the second week I started to use function literals. Then I started using match/case. Later I found out about for/yield with futures. And so on. I learnt the language at my job, and I was delivering real, user-facing features in it from day 1. I don't think I could've done the same with Clojure. (Reasonable people can disagree on syntax; I find that Scala's lets you get very close to the language of the domain (e.g. see the Spray "routing DSL") and that's worth the cost it imposes on the macro system)

seanmcdirmid 12y ago

Persistent structures are almost always O log N (because they have to be trees). You can fix your branch out for better performance at the cost of more wasted space. You will never be as fast as an array. Also, sometimes you want your updates to a data structure to be seen via an alias (happens often actually), then we are back at square one again. Systems people are much more pragmatic about this: they see the trends in hardware before we PL people do, and their solution has never been functional programming. Ok, well...maybe once with MapReduce (but Jeff and Sanjay are both systems PL people).

mtrimpe 12y ago
At n=32 the performance costs are as negligible as dynamic typing once was (even less I'd argue.) As far as I can tell that very much puts us at the point in time where mutable state is usually a premature optimization. Adopting it successfully does require separating essential from accidental state which naturally leads to some variant of FRP being adopted to manage (re-)derivation of accidental state; which is usually the hard to explain part. I am wondering what you do see systems people doing by the way? I'm always interested in broadening my perspective on this.
- seanmcdirmid 12y ago
  n=32 leads can lead to lots of sparseness, memory pressure, bad locality (given sparseness); its not going to fly easily as the default. The shared memory aspect is even a bigger hurdle: people share memory for a reason, its not very often accidental. FRP is hardly the answer, but I work on a competing paradigm so mark me as biased. The systems people I work with do a lot of key value stores (sharded NoSQL-style databases), there is a lot of pipelining for data parallel processing (MapReduce, Pig), and there is still a lot of classic locking going on, throughput is important. Oh, and lots of Paxos, logging, state replication, etc... The normal stuff you could find at Google (note I don't work for Google). On the HPC side, the people that really care about performance, there is some map reduce via MPI, but most of the big gains are to be had in CUDA, which is an extremely imperative programming model with very synchronized core executions and memory accesses. I've learned that whenever the FP people talk about high performance, they usually mean something else (like concurrency that isn't so slow), rather than the real high performance stuff that the big kids (systems and HPC communities) are doing.

oskarth 12y ago

> virtually everyone still underestimates the impact this will have and (or perhaps because) it's very difficult to get that across in writing. Why is it hard to get across in writing?

mtrimpe 12y ago
I'd say because of two reasons: a) it requires putting together a fairly large number of concepts of moderate complexity (and you never know which ones your readers already understand.) b) many of the concepts sound trivial and people usually don't grasp their power until you've listed X amounts of things it would enable. And to be clear; it's hard for me to put in writing for now ;)

nwmcsween 12y ago

Persistent structures do not map well with how computers work in reality.