4 ms·
You know when toddlers try to put a round peg in a square hole? that's how it feels using functional programming in a turing-machine-like CPU. It just don't fit
by chubot 13y ago
You know when toddlers try to put a round peg in a square hole? that's how it feels using functional programming in a turing-machine-like CPU. It just don't fit.
Yup, I agree. Functional languages failed in part because they modelled the machine poorly (Richard Gabriel has some reflections on Lisp in this area).
But the funny thing is that hardware has changed! Imperative programming languages are now the square peg in the round hole :)
Multicore machines are going to elaborate lengths to maintain the illusion of a uniform address space. But if we programmed with message passing and heterogeneous processes, then our programs would have more mechanical sympathy for the machine.
Shared memory and locks are basically the result of not wanting to disturb too much source code when converting from a single-threaded to a concurrent/parallel program. But it results in poor performance and scalability, not to mention untestable and unmaintainable programs. You can look at the evolution of multicore scalability in the Linux and FreeBSD kernels, or Python and Ruby GILs (coarse grained locks in a Turing machine model), for examples of this.
The other thing that has changed is that our hardware is distributed (not just clusters, but also client-server). Networks are fundamentally unreliable, and that means that idempotency of subprograms is a very important property. Idempotency is of course a word from the same area of mathematics that functional programming comes from (abstract algebra).
The other algebraic property that is important in distributed systems is commutativity. This basically has to do with physics[1]. Commuter[2] is a nice project which addresses commutativity in interface design.
It's not an accident that MapReduce and all the newer big data frameworks are based on the model of functional programming. They are using the imperative model at the scale of a single machine, where its appropriate, but the functional model at the scale of the cluster, so subprograms can be retried and executed in any order.
The alternative to explicitly thinking about these properties in your program design is to (only) use slow state machine protocols like Paxos. You could naively apply this transformation to your sequential programs, but you will be making very poor use of the hardware. There is a reason that they are only used for very small bits of state in distributed systems.
tl;dr Distributed systems (multicore included) are not Turing machines. I see a lot of bugs that are a result of people in the imperative mindset not understanding this.
[1] http://scholar.google.com/scholar?cluster=4892527405117123487&hl=en&as_sdt=0,5&sciodt=0,5 http://scholar.google.com/scholar?cluster=489252740511712348...
[2] http://pdos.csail.mit.edu/commuter/ http://pdos.csail.mit.edu/commuter/
- discreteevent 13y agoI think you are talking about a distributed calculation. This is not really representative of all distributed systems. Telecom software or engine managment systems don't make much use of map-reduce for example
- seanmcdirmid 13y agoThe MapReduce use case is quite narrow, we also have replicated storage systems that require, as you said, paxos; many HPC loads are using CUDA these days, which, even though heavily based on SIMD, is very imperative. As soon as you want iterative computation or long running incremental computation on streams...you need to worry about fault tolerance again, and state becomes very useful. You can leverage commutativity without going functional...heck, that's how most physics engines work and they are very far away from FP.
- chubot 13y agoI'm not only talking about MapReduce; I'm saying there are ideas from functional programming that have more affinity for the underlying hardware than state machines. This is only going to get more important in the future. Right now we're straddling the two worlds kind of awkwardly. State is useful; however exactly as in "traditional" functional programming, it should be used sparingly in distributed systems. State requires very special handling and thought. Statelessness or purity is the default. Clusters have a tiny bit of state and lots of stateless workers, just like a Haskell program has a tiny bit of state and lots of pure functions. To be clear, I don't think Haskell makes sense at the single machine level. (I personally don't want to program in it.) I'm saying that the ideas are more appropriate at the level of architecture and not code.