6 ms·

Clojure 1.3 First Impression (It's Fast)

102 points by johnaspden 16y ago

johnaspden 16y ago
Last time I tried Scala, it blew Clojure out of the water for speed on the program I translated. I assume that Java did too. I think that's probably not true any more. Anyway, pretty fractal tree program in lisp!
- edifice 16y ago
  You're still going to find Scala to be significantly faster on arbitrary code. It's great that there are ways to speed up key functions in Clojure now though.
  - swannodette 16y ago
    Backed up by what evidence? And by arbitrary code do you mean code that embraces mutability?
    - edifice 16y ago
      Scala effectively has these kinds of type declarations on every function. This article itself proves the point. Take out the :static and type decls and see.
      - swannodette 16y ago
        Idiomatic Clojure tends to use Clojure's persistent data structures - you don't often create custom types, all of the core functions on those data structures are already pretty much as fast as possible. No need for ^:static or type decls. The article also doesn't get into deftype/defrecord/protocols. In those cases you always get the fastest path of the platform. Again you can build things that have the same perf of the core data structures w/o resorting to ^:static or type decls. So what exactly do you mean again by arbitrary code? Perhaps you meant numeric code - there ^:static and type decls help plenty. Perhaps you mean Java interop? Again sure. Personally I think Clojure is really taking the dynamic, generic, and fast thing to a whole new level, w/o type-hinting of any kind. EDIT: This isn't too say Clojure performance can't continue to improve. Scala's ability to use Java arrays of primitives in higher order operations is something I really, really want to see (and the the above is a big chunk of the work in that direction)
        jules 16y ago
        It's strange that you mention Clojure's persistent data structures as an advantage for performance. They have many advantages but performance isn't one of them.
        swannodette 16y ago
        I'm curious how you come to this conclusion? Programs in any language will often be designed around immutability in order to improve code maintainability. Without fast persistent data structures this will most certainly decrease the performance of your otherwise elegant design due to considerable amounts of copying.
        lukev 16y ago
        If you're going for raw speed you should use mutable data structures. Sadly, that's just the way it is - you can update a mutable data structure in a couple of instructions by swapping out a pointer, while "updating" an immutable/persistent object is inherently more work at the machine level. Clojure is awesome because it provides immutable persistent structures that are well onto the good side of "good enough", not because it's the fastest thing possible. Java itself is far from the fastest thing possible, and the speed of Java is itself a strict upper bound on Clojure's speed.
        swannodette 16y ago
        Sure, but of course if you want raw speed you would write assembly. Most people move up the food chain to C to get some basic level of correctness. I think the history of production oriented PLs has revolved around balancing correctness/performance. And it's always a question of diminishing return (I've written enough C/C++/Objective-C++ to know this). Every sizeable application I've written in a lower level language eventually approaches some kind of wacky manual memory management scheme to preserve performance w/o sacrificing correctness. I think Clojure's solution is a step in the right direction. But no, if I was writing a real-time system with extremely limited memory constraints I would not write in the Clojure. But I don't think that's what this thread is about.
      - nickik 16y ago
        Of corse scala is fast with stuff like that because its typed. But should we talk about all the stuff you can't do because you have types all the time? I think clojure is getting really close to a dynamic language that can get down to the speed of a static language if you need it.
        johnaspden 16y ago
        Yes, let's. You first!
- swah 16y ago
  And Ocaml is faster than Ruby!
  - swah 16y ago
    I was pointing out that the comparison between Scala and Clojure is between a static language and a dynamic one, the static one will almost always perform better.
whakojacko 16y ago
As someone familiar with Java/Scala but not Clojure, what exactly do those type hints do? Just prevent it from autoboxing the primitives? Avoid excessive reflection? Can this be done for other types or just int/long/float/etc?
- apgwoz 16y ago
  It avoids reflection, and instanceof checks.
- swannodette 16y ago
  type-hinting Java objects to avoid reflection has been possible for some time now. What wasn't possible was for fns to take or return primitives w/o boxing. ^:static lets you do that. ^:static also allows the JVM to apply the most aggressive optimizations to your code - ^:static tells the JVM the code won't change - so callers of ^:static fns won't get the latest version if you redef them. The benefit being that with numeric code you'll often see an order of magnitude performance jump. So a general strategy is to write your code as you normally would - def'ing redef'ing fns at will. Then when it works - declare the critical paths ^:static, and adding primitive type hints ^long, ^double if the code is numeric.
  - _delirium 16y ago
    CMUCL, dealing with a similar infinitely-dynamic problem in Lisp, had a similar approach but dealt with it on a block-of-code level instead of per-function. You could declare a block to be compiled together, and any calls between functions in the block would be treated as having a promise that you'd never redef them without recompiling the whole block, so they could be statically optimized with respect to each other. Not sure if that's a better or worse approach overall. It probably depends on the structure of your program. One nice feature was that the same function could be static from some perspectives and not from others--- to functions in the same compilation block it was static, but functions from outside that block that called it treated it as non-static, and would immediately get the new version if, say, that whole block were recompiled. That let you make the core code optimized by sticking it in one big compilation block, while not changing the normal dynamic Lisp semantics of the block when viewed from the perspective of any outside function.

102 points by johnaspden 16y ago

Last time I tried Scala, it blew Clojure out of the water for speed on the program I translated. I assume that Java did too. I think that's probably not true any more. Anyway, pretty fractal tree program in lisp!

edifice 16y ago
You're still going to find Scala to be significantly faster on arbitrary code. It's great that there are ways to speed up key functions in Clojure now though.
- swannodette 16y ago
  Backed up by what evidence? And by arbitrary code do you mean code that embraces mutability?
  - edifice 16y ago
    Scala effectively has these kinds of type declarations on every function. This article itself proves the point. Take out the :static and type decls and see.
    - swannodette 16y ago
      Idiomatic Clojure tends to use Clojure's persistent data structures - you don't often create custom types, all of the core functions on those data structures are already pretty much as fast as possible. No need for ^:static or type decls. The article also doesn't get into deftype/defrecord/protocols. In those cases you always get the fastest path of the platform. Again you can build things that have the same perf of the core data structures w/o resorting to ^:static or type decls. So what exactly do you mean again by arbitrary code? Perhaps you meant numeric code - there ^:static and type decls help plenty. Perhaps you mean Java interop? Again sure. Personally I think Clojure is really taking the dynamic, generic, and fast thing to a whole new level, w/o type-hinting of any kind. EDIT: This isn't too say Clojure performance can't continue to improve. Scala's ability to use Java arrays of primitives in higher order operations is something I really, really want to see (and the the above is a big chunk of the work in that direction)
      - jules 16y ago
        It's strange that you mention Clojure's persistent data structures as an advantage for performance. They have many advantages but performance isn't one of them.
        swannodette 16y ago
        I'm curious how you come to this conclusion? Programs in any language will often be designed around immutability in order to improve code maintainability. Without fast persistent data structures this will most certainly decrease the performance of your otherwise elegant design due to considerable amounts of copying.
        lukev 16y ago
        If you're going for raw speed you should use mutable data structures. Sadly, that's just the way it is - you can update a mutable data structure in a couple of instructions by swapping out a pointer, while "updating" an immutable/persistent object is inherently more work at the machine level. Clojure is awesome because it provides immutable persistent structures that are well onto the good side of "good enough", not because it's the fastest thing possible. Java itself is far from the fastest thing possible, and the speed of Java is itself a strict upper bound on Clojure's speed.
        swannodette 16y ago
        Sure, but of course if you want raw speed you would write assembly. Most people move up the food chain to C to get some basic level of correctness. I think the history of production oriented PLs has revolved around balancing correctness/performance. And it's always a question of diminishing return (I've written enough C/C++/Objective-C++ to know this). Every sizeable application I've written in a lower level language eventually approaches some kind of wacky manual memory management scheme to preserve performance w/o sacrificing correctness. I think Clojure's solution is a step in the right direction. But no, if I was writing a real-time system with extremely limited memory constraints I would not write in the Clojure. But I don't think that's what this thread is about.
    - nickik 16y ago
      Of corse scala is fast with stuff like that because its typed. But should we talk about all the stuff you can't do because you have types all the time? I think clojure is getting really close to a dynamic language that can get down to the speed of a static language if you need it.
      - johnaspden 16y ago
        Yes, let's. You first!
swah 16y ago
And Ocaml is faster than Ruby!
- swah 16y ago
  I was pointing out that the comparison between Scala and Clojure is between a static language and a dynamic one, the static one will almost always perform better.

whakojacko 16y ago

As someone familiar with Java/Scala but not Clojure, what exactly do those type hints do? Just prevent it from autoboxing the primitives? Avoid excessive reflection? Can this be done for other types or just int/long/float/etc?

apgwoz 16y ago
It avoids reflection, and instanceof checks.
swannodette 16y ago
type-hinting Java objects to avoid reflection has been possible for some time now. What wasn't possible was for fns to take or return primitives w/o boxing. ^:static lets you do that. ^:static also allows the JVM to apply the most aggressive optimizations to your code - ^:static tells the JVM the code won't change - so callers of ^:static fns won't get the latest version if you redef them. The benefit being that with numeric code you'll often see an order of magnitude performance jump. So a general strategy is to write your code as you normally would - def'ing redef'ing fns at will. Then when it works - declare the critical paths ^:static, and adding primitive type hints ^long, ^double if the code is numeric.
- _delirium 16y ago
  CMUCL, dealing with a similar infinitely-dynamic problem in Lisp, had a similar approach but dealt with it on a block-of-code level instead of per-function. You could declare a block to be compiled together, and any calls between functions in the block would be treated as having a promise that you'd never redef them without recompiling the whole block, so they could be statically optimized with respect to each other. Not sure if that's a better or worse approach overall. It probably depends on the structure of your program. One nice feature was that the same function could be static from some perspectives and not from others--- to functions in the same compilation block it was static, but functions from outside that block that called it treated it as non-static, and would immediately get the new version if, say, that whole block were recompiled. That let you make the core code optimized by sticking it in one big compilation block, while not changing the normal dynamic Lisp semantics of the block when viewed from the perspective of any outside function.