4 ms·
While cool,but really, Is there such a great clamoring for scaling programs over multiple cores Absolutely. Queues, databases, caches, machine learning, graph
by aphyr 13y ago
While cool,but really, Is there such a great clamoring for scaling programs over multiple cores
Absolutely. Queues, databases, caches, machine learning, graph algorithms, CRDTs, image, audio, and video processing, stream algorithms, analytics, geospatial work, even HTTP APIs: all demand compute performance. Since CPU flops have been essentially stagnant for ten years, the only way to improve performance on commodity hardware is with parallelism. Almost every program I've worked on in the last five years has demanded multicore programming, or at least in-process concurrency.
Clojure is not the fastest language out there, but it does make reasoning about concurrent code dramatically simpler than other languages. My Clojure programs tend to parallelize trivially over 16 or 32 cores, and only rarely contain race conditions, deadlocks, or other threading bugs. In contrast, I routinely struggle to write correct multithreaded code in Java, and when I do, I usually adopt a slower locking mechanism because I'm just not smart enough to reason about concurrent mutable state.
Why is Clojure better for writing concurrent programs? For one, it's multi-paradigm. You can serialize transformations over state using normal JVM locks, agents, atoms, refs, or promises. Immutable shared-structure objects make concurrency safety trivial--but Clojure's identity types make reasoning about state easy, instead of having to express everything through monadic composition. For mutable state, you've got java.util.concurrent and Cliff Click's high_scale_lib available, plus great libraries like Netty. It's a deeply pragmatic language where the defaults are safe, but controlled speed is available where you need it.
Or you could run 64 Unicorn/V8 workers per node, and push all the hard problems into the database. That's a totally valid strategy--but I think there's a little more to life than that. ;-)