4 ms·

Does Clojure still have a non-trivial impact on performance? And if it doesn't, does it still look as pretty? I only ask because the whole idea of using it for

Does Clojure still have a non-trivial impact on performance? And if it doesn't, does it still look as pretty? I only ask because the whole idea of using it for "Big Data" flies right out the window if it's significantly slower than plain old Java. Especially if it's slower using naive or expected Clojure methodology. And if there's not a speed advantage, than companies can find a lot more people who know and can work Java code than Clojure.

willismichael 13y ago
Performance and scalability are two different things. My favorite example is to think of performance as being like the speed limit, and scalability as the ability to easily add more lanes to the road. Once there are enough vehicles on the road, it doesn't matter how high you set the speed limit, they'll still slow to a crawl. Now imagine if there was a trivial way to add more lanes to the road. Clojure has very nice concurrency mechanisms baked right into the language.
- optimiz3 13y ago
  This doesn't answer the author's question. We know the difference between performance and scalability. How does it perform compared to Java?
  - saosebastiao 13y ago
    It is slower than java. It can approach Java with type annotations which aren't that hard to do. It is faster than any other dynamic language I have used, including python, ruby, javascript(v8), and lua(jit).
    - graue 13y ago
      Faster than LuaJIT? At what? Did you run benchmarks to back that up? That's a very strong claim, and I've never heard anyone make it before. LuaJIT is unbelievably fast. I'd be extremely curious to see the proof. (As a side note, on some level it's hard for me to accept that Clojure is fast because using it feels gratingly slow — since all the tools take several seconds to do anything thanks to the JVM's startup time. `lein repl`: 4 seconds wait. `luajit`, or `python` for that matter: utterly instant.)
      - saosebastiao 13y ago
        No benchmarks, just a few small algorithm implementations (kmeans, random forest, ff neural net). Clojure was faster, although the difference was pretty trivial. I have been frustrated with Clojure lately because of startup time (I do a lot of CLI tools), and would love to use Luajit, but my coworkers hate when I write in languages other than Ruby or Clojure. For my own pet projects it is great.
        graue 13y ago
        Well that's pretty darn cool to hear that Clojure can achieve LuaJIT's level of speed. But I agree, I feel Clojure just isn't suited for CLI tools at all, unfortunately. I know there are solutions that keep a background JVM ready and reuse it but I've never managed to get that to work smoothly.
  - Mikeb85 13y ago
    How does Java compare to C or Fortran? And how much does it matter? Ruby and Python are incredibly popular despite being incredibly slow. Clojure has decent enough performance, plus easy parallelism/concurrency...
- jdmichal 13y ago
  Performance and language prettiness are also two different things. Again, when we're talking about "Big Data", the implication is that there is a very large number of rows being processed. That's nice that "adding lanes" looks pretty in Clojure, but it's not like it's impossible to do in Java. And if I can double or quadruple my throughput by switching to uglier Java code? And on top of that I drastically widen up the pool of potential employees? Well, this is starting to look like a pretty shut case from a business perspective. And that's also why I asked about whether naive Clojure script is performant, or whether you have to take so many non-obvious routes to eek out the performance that it loses "conciseness" as a point in its favor. (Disclaimer: I don't have recent performance numbers on Java vs Clojure; that's why I was asking for them. The double/quadruple numbers above were pulled from thin air.)
  - tensor 13y ago
    In my experience it's always possible to get java level performance out of clojure when you need it, even if that ends up meaning that you essentially write java with sexpressions and macros. However, even for "medium data", when I find clojure too slow I fall back to C/C++. I've never found myself thinking "maybe I should write this in java." Unfortunately, I can't find good examples of modern performant clojure with benchmarks at the moment. As far as I understand, the alioth benchmarks are still out of date.
moomin 13y ago
Funny you mention big data, because it's something of Clojure's sweet spot in my opinion. Take a look at Cascalog and Storm. Both are in use at Twitter. Then add pallet for dynamic provisioning.
columbo 13y ago
It's a good question, and it really depends. I have one system where clojure performs better than java. It's not an apples-to-apples comparison as the java source is very rigid and stateful. There's a ton of recalculations being done and there's almost no way to thread it out because the current implementation relies heavily on mutation. The clojure equivalent is much smaller and just overpowers the java implementation due to the ease of threading (pmap, memoize). In that specific example I was looking at maybe 20x performance increase? I have another test where I cannot even get close to java performance. Even after adding in type specific checks. In that example java wins because it's a native/primitive java implementation. Even without threading the java side wins because it's been reduced to such a simplistic operation (500 lines of java code using only primitive types) that the overhead of clojure can't beat it. In this test java is about 5x fastar than the clojure equivalent. It really depends on what you are doing I tend to think of it like raw java > clojure > over engineered java... In some instances it can be very, very powerful.

willismichael 13y ago

Performance and scalability are two different things. My favorite example is to think of performance as being like the speed limit, and scalability as the ability to easily add more lanes to the road. Once there are enough vehicles on the road, it doesn't matter how high you set the speed limit, they'll still slow to a crawl. Now imagine if there was a trivial way to add more lanes to the road. Clojure has very nice concurrency mechanisms baked right into the language.

optimiz3 13y ago
This doesn't answer the author's question. We know the difference between performance and scalability. How does it perform compared to Java?
- saosebastiao 13y ago
  It is slower than java. It can approach Java with type annotations which aren't that hard to do. It is faster than any other dynamic language I have used, including python, ruby, javascript(v8), and lua(jit).
  - graue 13y ago
    Faster than LuaJIT? At what? Did you run benchmarks to back that up? That's a very strong claim, and I've never heard anyone make it before. LuaJIT is unbelievably fast. I'd be extremely curious to see the proof. (As a side note, on some level it's hard for me to accept that Clojure is fast because using it feels gratingly slow — since all the tools take several seconds to do anything thanks to the JVM's startup time. `lein repl`: 4 seconds wait. `luajit`, or `python` for that matter: utterly instant.)
    - saosebastiao 13y ago
      No benchmarks, just a few small algorithm implementations (kmeans, random forest, ff neural net). Clojure was faster, although the difference was pretty trivial. I have been frustrated with Clojure lately because of startup time (I do a lot of CLI tools), and would love to use Luajit, but my coworkers hate when I write in languages other than Ruby or Clojure. For my own pet projects it is great.
      - graue 13y ago
        Well that's pretty darn cool to hear that Clojure can achieve LuaJIT's level of speed. But I agree, I feel Clojure just isn't suited for CLI tools at all, unfortunately. I know there are solutions that keep a background JVM ready and reuse it but I've never managed to get that to work smoothly.
- Mikeb85 13y ago
  How does Java compare to C or Fortran? And how much does it matter? Ruby and Python are incredibly popular despite being incredibly slow. Clojure has decent enough performance, plus easy parallelism/concurrency...
jdmichal 13y ago
Performance and language prettiness are also two different things. Again, when we're talking about "Big Data", the implication is that there is a very large number of rows being processed. That's nice that "adding lanes" looks pretty in Clojure, but it's not like it's impossible to do in Java. And if I can double or quadruple my throughput by switching to uglier Java code? And on top of that I drastically widen up the pool of potential employees? Well, this is starting to look like a pretty shut case from a business perspective. And that's also why I asked about whether naive Clojure script is performant, or whether you have to take so many non-obvious routes to eek out the performance that it loses "conciseness" as a point in its favor. (Disclaimer: I don't have recent performance numbers on Java vs Clojure; that's why I was asking for them. The double/quadruple numbers above were pulled from thin air.)
- tensor 13y ago
  In my experience it's always possible to get java level performance out of clojure when you need it, even if that ends up meaning that you essentially write java with sexpressions and macros. However, even for "medium data", when I find clojure too slow I fall back to C/C++. I've never found myself thinking "maybe I should write this in java." Unfortunately, I can't find good examples of modern performant clojure with benchmarks at the moment. As far as I understand, the alioth benchmarks are still out of date.

moomin 13y ago

Funny you mention big data, because it's something of Clojure's sweet spot in my opinion. Take a look at Cascalog and Storm. Both are in use at Twitter. Then add pallet for dynamic provisioning.

columbo 13y ago

It's a good question, and it really depends. I have one system where clojure performs better than java. It's not an apples-to-apples comparison as the java source is very rigid and stateful. There's a ton of recalculations being done and there's almost no way to thread it out because the current implementation relies heavily on mutation. The clojure equivalent is much smaller and just overpowers the java implementation due to the ease of threading (pmap, memoize). In that specific example I was looking at maybe 20x performance increase? I have another test where I cannot even get close to java performance. Even after adding in type specific checks. In that example java wins because it's a native/primitive java implementation. Even without threading the java side wins because it's been reduced to such a simplistic operation (500 lines of java code using only primitive types) that the overhead of clojure can't beat it. In this test java is about 5x fastar than the clojure equivalent. It really depends on what you are doing I tend to think of it like raw java > clojure > over engineered java... In some instances it can be very, very powerful.