12 ms·

If the author reads this: Can you elaborate more on what you mean by "lackluster performance"? What is the use case? If you're looking for top speed in terms o

If the author reads this: Can you elaborate more on what you mean by "lackluster performance"? What is the use case? If you're looking for top speed in terms of C/assembly performance - I'd say yes, probably the JVM will get in your way. However, I spent months building a database/key-value store in Clojure and it's quite doable to write very high performance code in Clojure as long as you put the right type hints everywhere. Tools like YourKit can help you identify the bottlenecks in your program. Again, depends on the use case, but Clojure makes it very, very idiomatic to write code that makes full use of a multicore system - something possible in the like of Java, C, et al but definitely not idiomatic at all.

pron 10y ago
(not the author) > probably the JVM will get in your way Not so much the JVM, but Clojure and how it uses the JVM (although, as you say, it's possible to get not too far from top JVM performance with Clojure). It's fairly easy to get C performance (and even beat it in concurrent code) for the same amount of effort on the JVM. Currently, the main handicap the JVM has is the lack of arrays-of-structs which may cause lots of cache-misses, and requires less-than-elegant code to overcome. This, thankfully, is being addressed by the addition of value types.
- raspasov 10y ago
  Yup, the JVM is something that people (esp. startups & new-co's) are, all things being equal, vastly underusing or over-complaining about :). Before I started doing Clojure the JVM ecosystem was this "scary" thing but 4 years later I've learned to appreciate it a lot - stable AND sound libraries, high performance out-of-the box, and very,very high performance if you really need it.
  - hellofunk 10y ago
    > very,very high performance if you really need it But not as high performance as Java directly. And typically not as high performance as native languages, which the author addresses.
    - pron 10y ago
      There's absolutely nothing about native languages that makes them a-priori faster than the JVM. On the contrary, the JVMs compiler is likely to emit betteroptimized machine code than their compilers. Some low-level languages (like C/C++/Rust) can (and do in many cases) offer better performance, but that's not because they're "native" but because they allow (and require) finer control over every operation. This level of control comes at a significant cost, though.
      - hellofunk 10y ago
        Notice I said "typically" because, typically it is true that a native language runs faster than Clojure, which is what my comment was referring to.
  - agibsonccc 10y ago
    What clojure and even scala do with the JVM scares me. The most concrete example I know of is using objects for everything. 1 of the fundamental problems I have with many of the higher level JVM languages is how much they hide from you. That being said, most people won't notice. I do numerical software in java so might be a bit more sensitive to some of this stuff.
    - raspasov 10y ago
      If you ever have the need, checkout this blog post about optimizing math operations in Clojure http://insideclojure.org/2014/12/15/warn-on-boxed/ http://insideclojure.org/2014/12/15/warn-on-boxed/
- WildUtah 10y ago
  It's fairly easy to get C performance (and even beat it in concurrent code) for the same amount of effort on the JVM. Ha. That's what we were promised back in the mid-1990s. It's no more true today than it was then. Well written Java will always be about 2-4x slower than similar C code. JVM applications have many compensating advantages, including the aforementioned ease of exploiting concurrency. That doesn't erase the reality of single thread performance where Java has never, in twenty years of strong expert effort, caught up.
  - fnordsensei 10y ago
    Agreed, in that the performance benefits you can get out of a higher level language will often depend on the stuff where the limits of human cognition is the performance boundary. As computing gets faster in general, the practical considerations should shift more towards the performance you can get out of an entire system than unit-for-unit performance. On that level, human comprehension often seems to be what needs to be accounted for. I think of it as problem akin to how you can make traffic run better in an entire city rather than optimising the performance of an engine in a single car. It doesn't invalidate making more performant engines, but it's a different level of consideration. When saying, "JVM is as performant as C", it's easy to run the numbers and see whether it's true or not, objectively. The caveat that GP throws in is, for the same amount of effort. Then you need to specify which human and under what circumstances, and the entire thing gets hairier.
    - WildUtah 10y ago
      As computing gets faster in general, the practical considerations should shift more towards the performance you can get out of an entire system than unit-for-unit performance. Those days are over, unfortunately. Computing stopped getting faster in general around 2010. And that also changes the calculus around single thread performance. You can no longer count on hardware eventually to solve performance problems.
      - mtrycz 10y ago
        And this is why parallel/concurrent computing/applications are necessary to ease speedups. And that's what the new languages promise to let you get right. For the same effort.
- agibsonccc 10y ago
  It can though. We are a java shop that does HPC and deep learning. Our work is open source. We've been around 2.5 going on 3 years now. Over the years we've surpassed bottleneck after bottleneck in the JVM. It's a faster wrapper around c code for us. We maintain and created: https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp which wraps a lot of c++ components. This is also how we do GPUs and the like with our own memory management for our "numpy for java": https://github.com/deeplearning4j/nd4j/tree/master/nd4j-backends/nd4j-backend-impls/nd4j-cuda https://github.com/deeplearning4j/nd4j/tree/master/nd4j-back... Short of it "off heap and NO GC" matters a lot. We have 1 c codebase we use that's meant to be controlled via JNI here: https://github.com/deeplearning4j/libnd4j https://github.com/deeplearning4j/libnd4j The speed gains we've seen are massive. Java can't compete with good ole simd and the like for numerical computing. We are a big advocate of the JVM as a platform but let's be clear about its weaknesses. You need to use unsafe (see: netty,aeron, and any other low level database written in java or using java as the core language with some c++)and other tricks out of the box for real performance.
  - pjmlp 10y ago
    The problem is that you are using it in areas, HPC and deep learning, where using plain C++ would be a better option. I think. There is a reason why Java 10 aims to improve Java's story in regard to mechanical sympathy.
    - agibsonccc 10y ago
      We wrote a lot of that stack ourselves. And our compute code is in c++. There is a lot of stuff going on in big data land (including us) wrapping a lot of the c++ in java and enabling people to use the JVM for what it's good at (data access). We are pushing this with nvidia: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-deep-learning/ https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-... I also work pretty closely with a lot of the spark/gpu folks at IBM. While I do largely agree with you, we have our own JNI compiler called javacpp that alleviates a lot of those concerns already: https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp Having our own pointer class and doing our own memory management has helped a lot. What you're talking about is using java for the compute. There's no reason you can't wrap that in a runtime that most people know how to use. A lot of python folks do that now..but then you have to deal with python's limitations at which point the majority of your code (way more than needs be) will end up being in c anyways vs java where you can write a significant part of your app in java and have it be fast out of the box.
- virtualwhys 10y ago
  > This, thankfully, is being addressed by the addition of value types. Last I heard (JLS 2016 keynote by Brian Goetz) they have yet to commit/guarantee when value types will land. Java 10 is planned; if they're completed by then we're looking at probably 2020 given the delays with Java 9 release. That's 3 years for other languages/platforms to evolve while the JVM unboxes itself.
  - pron 10y ago
    Large organizations switch programming languages no more than once a decade, and Java is way ahead of the competition as it stands.
krat0sprakhar 10y ago
> as long as you put the right type hints everywhere. I'm a novice Clojure programmer and haven't previously heard of this. Can you please elaborate?
- ajss 10y ago
  (defn [o n] (.write o n)) will have to look up the .write method by doing reflection on o & n at runtime, which is really slow. (defn [^WriteInterface o ^long n] (.write o n)) can be much faster because the compiler knows the type of o & n.
  - abc_lisper 10y ago
    I think you meant to say, (defn func[o n] (.write ^WriteInterface o ^long n)) This is needed only when calling Java methods, because Java allows overloading. Clojure only does arity-overloading, not type overloading, and hence doesn't need the type info.
- moomin 10y ago
  Short version: put :warn-on-reflection in your leiningen file, And then add type hints (google them) to reduce the warnings to zero. There's no reason the average Clojure project ever need be using reflection for dispatch.
  - raspasov 10y ago
    Correct, a lot of the average application-level code you see in the wild does not have to worry about this. If we're talking about things like load-balancers or databases or endpoints/functions that will be hit 1000s of times per second - then it's a diff. conversation.
- hkjgkjy 10y ago
  You don't need to specify what type a function takes in Clojure, which is often nice. If you do though, the compiler can sometimes optimize, and make things much faster. Basically, the compiler does not have to * look up the type of the thing * run the action on it but can instead skip straight to step 2. If you somewhere in your Clojure program (or repl) run (set! warn-on-reflection true) it will print to the console when you are doing reflection. I think it's a good idea to put in the beginning of your core namespace - so that it is always on. Why not always wear the seatbelt when driving, you know?
fsloth 10y ago
"something possible in the like of Java, C, et al but definitely not idiomatic at all." What on earth is non-idiomatic multicore c code? You have to try really hard with Macro tomfoolery before c becomes non-idiomatic :) .
- lvh 10y ago
  Isn't this just a convoluted way of arguing that there's no such thing as idiomatic multicore C? If everything's idiomatic, then surely nothing is.
  - fsloth 10y ago
    No, it was a convoluted way to say that the attempt to measure C programs by their quality as "idiomatic" will unlikely bear any fruits of value. That said, if there is any non-idiomatic C code then the attempts of 'experts' in other languages to mangle C to look more like their language surely fall into this category (There are lots of examples where the author starts 'We can make C look more like fortran by defining our macros to replace these keywords..).
klibertp 10y ago
I'm not the author, but looking through the sibling comments I feel I should point out the obvious: startup time. It can take long seconds for Clojure REPL to start up. No other dynamic language I had contact with was that bad on that front. Even Erlang and Elixir boot up faster! Of course, I know about various ways of setting up a daemon process in the background, but then there is a need for reloading/hot-swapping code, which Clojure also fails to do well. Both Erlang and Common Lisp (also Emacs... even Emacs!) are better in this regard, both for different reasons. Anyway, Clojure overall performance impression - how fast it feels for users - is bad and will remain bad unless the startup problem is fixed.

pron 10y ago

(not the author) > probably the JVM will get in your way Not so much the JVM, but Clojure and how it uses the JVM (although, as you say, it's possible to get not too far from top JVM performance with Clojure). It's fairly easy to get C performance (and even beat it in concurrent code) for the same amount of effort on the JVM. Currently, the main handicap the JVM has is the lack of arrays-of-structs which may cause lots of cache-misses, and requires less-than-elegant code to overcome. This, thankfully, is being addressed by the addition of value types.

raspasov 10y ago
Yup, the JVM is something that people (esp. startups & new-co's) are, all things being equal, vastly underusing or over-complaining about :). Before I started doing Clojure the JVM ecosystem was this "scary" thing but 4 years later I've learned to appreciate it a lot - stable AND sound libraries, high performance out-of-the box, and very,very high performance if you really need it.
- hellofunk 10y ago
  > very,very high performance if you really need it But not as high performance as Java directly. And typically not as high performance as native languages, which the author addresses.
  - pron 10y ago
    There's absolutely nothing about native languages that makes them a-priori faster than the JVM. On the contrary, the JVMs compiler is likely to emit betteroptimized machine code than their compilers. Some low-level languages (like C/C++/Rust) can (and do in many cases) offer better performance, but that's not because they're "native" but because they allow (and require) finer control over every operation. This level of control comes at a significant cost, though.
    - hellofunk 10y ago
      Notice I said "typically" because, typically it is true that a native language runs faster than Clojure, which is what my comment was referring to.
- agibsonccc 10y ago
  What clojure and even scala do with the JVM scares me. The most concrete example I know of is using objects for everything. 1 of the fundamental problems I have with many of the higher level JVM languages is how much they hide from you. That being said, most people won't notice. I do numerical software in java so might be a bit more sensitive to some of this stuff.
  - raspasov 10y ago
    If you ever have the need, checkout this blog post about optimizing math operations in Clojure http://insideclojure.org/2014/12/15/warn-on-boxed/ http://insideclojure.org/2014/12/15/warn-on-boxed/
WildUtah 10y ago
It's fairly easy to get C performance (and even beat it in concurrent code) for the same amount of effort on the JVM. Ha. That's what we were promised back in the mid-1990s. It's no more true today than it was then. Well written Java will always be about 2-4x slower than similar C code. JVM applications have many compensating advantages, including the aforementioned ease of exploiting concurrency. That doesn't erase the reality of single thread performance where Java has never, in twenty years of strong expert effort, caught up.
- fnordsensei 10y ago
  Agreed, in that the performance benefits you can get out of a higher level language will often depend on the stuff where the limits of human cognition is the performance boundary. As computing gets faster in general, the practical considerations should shift more towards the performance you can get out of an entire system than unit-for-unit performance. On that level, human comprehension often seems to be what needs to be accounted for. I think of it as problem akin to how you can make traffic run better in an entire city rather than optimising the performance of an engine in a single car. It doesn't invalidate making more performant engines, but it's a different level of consideration. When saying, "JVM is as performant as C", it's easy to run the numbers and see whether it's true or not, objectively. The caveat that GP throws in is, for the same amount of effort. Then you need to specify which human and under what circumstances, and the entire thing gets hairier.
  - WildUtah 10y ago
    As computing gets faster in general, the practical considerations should shift more towards the performance you can get out of an entire system than unit-for-unit performance. Those days are over, unfortunately. Computing stopped getting faster in general around 2010. And that also changes the calculus around single thread performance. You can no longer count on hardware eventually to solve performance problems.
    - mtrycz 10y ago
      And this is why parallel/concurrent computing/applications are necessary to ease speedups. And that's what the new languages promise to let you get right. For the same effort.
agibsonccc 10y ago
It can though. We are a java shop that does HPC and deep learning. Our work is open source. We've been around 2.5 going on 3 years now. Over the years we've surpassed bottleneck after bottleneck in the JVM. It's a faster wrapper around c code for us. We maintain and created: https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp which wraps a lot of c++ components. This is also how we do GPUs and the like with our own memory management for our "numpy for java": https://github.com/deeplearning4j/nd4j/tree/master/nd4j-backends/nd4j-backend-impls/nd4j-cuda https://github.com/deeplearning4j/nd4j/tree/master/nd4j-back... Short of it "off heap and NO GC" matters a lot. We have 1 c codebase we use that's meant to be controlled via JNI here: https://github.com/deeplearning4j/libnd4j https://github.com/deeplearning4j/libnd4j The speed gains we've seen are massive. Java can't compete with good ole simd and the like for numerical computing. We are a big advocate of the JVM as a platform but let's be clear about its weaknesses. You need to use unsafe (see: netty,aeron, and any other low level database written in java or using java as the core language with some c++)and other tricks out of the box for real performance.
- pjmlp 10y ago
  The problem is that you are using it in areas, HPC and deep learning, where using plain C++ would be a better option. I think. There is a reason why Java 10 aims to improve Java's story in regard to mechanical sympathy.
  - agibsonccc 10y ago
    We wrote a lot of that stack ourselves. And our compute code is in c++. There is a lot of stuff going on in big data land (including us) wrapping a lot of the c++ in java and enabling people to use the JVM for what it's good at (data access). We are pushing this with nvidia: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-deep-learning/ https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-... I also work pretty closely with a lot of the spark/gpu folks at IBM. While I do largely agree with you, we have our own JNI compiler called javacpp that alleviates a lot of those concerns already: https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp Having our own pointer class and doing our own memory management has helped a lot. What you're talking about is using java for the compute. There's no reason you can't wrap that in a runtime that most people know how to use. A lot of python folks do that now..but then you have to deal with python's limitations at which point the majority of your code (way more than needs be) will end up being in c anyways vs java where you can write a significant part of your app in java and have it be fast out of the box.
virtualwhys 10y ago
> This, thankfully, is being addressed by the addition of value types. Last I heard (JLS 2016 keynote by Brian Goetz) they have yet to commit/guarantee when value types will land. Java 10 is planned; if they're completed by then we're looking at probably 2020 given the delays with Java 9 release. That's 3 years for other languages/platforms to evolve while the JVM unboxes itself.
- pron 10y ago
  Large organizations switch programming languages no more than once a decade, and Java is way ahead of the competition as it stands.

krat0sprakhar 10y ago

> as long as you put the right type hints everywhere. I'm a novice Clojure programmer and haven't previously heard of this. Can you please elaborate?

ajss 10y ago
(defn [o n] (.write o n)) will have to look up the .write method by doing reflection on o & n at runtime, which is really slow. (defn [^WriteInterface o ^long n] (.write o n)) can be much faster because the compiler knows the type of o & n.
- abc_lisper 10y ago
  I think you meant to say, (defn func[o n] (.write ^WriteInterface o ^long n)) This is needed only when calling Java methods, because Java allows overloading. Clojure only does arity-overloading, not type overloading, and hence doesn't need the type info.
moomin 10y ago
Short version: put :warn-on-reflection in your leiningen file, And then add type hints (google them) to reduce the warnings to zero. There's no reason the average Clojure project ever need be using reflection for dispatch.
- raspasov 10y ago
  Correct, a lot of the average application-level code you see in the wild does not have to worry about this. If we're talking about things like load-balancers or databases or endpoints/functions that will be hit 1000s of times per second - then it's a diff. conversation.
hkjgkjy 10y ago
You don't need to specify what type a function takes in Clojure, which is often nice. If you do though, the compiler can sometimes optimize, and make things much faster. Basically, the compiler does not have to * look up the type of the thing * run the action on it but can instead skip straight to step 2. If you somewhere in your Clojure program (or repl) run (set! warn-on-reflection true) it will print to the console when you are doing reflection. I think it's a good idea to put in the beginning of your core namespace - so that it is always on. Why not always wear the seatbelt when driving, you know?

fsloth 10y ago

"something possible in the like of Java, C, et al but definitely not idiomatic at all." What on earth is non-idiomatic multicore c code? You have to try really hard with Macro tomfoolery before c becomes non-idiomatic :) .

lvh 10y ago
Isn't this just a convoluted way of arguing that there's no such thing as idiomatic multicore C? If everything's idiomatic, then surely nothing is.
- fsloth 10y ago
  No, it was a convoluted way to say that the attempt to measure C programs by their quality as "idiomatic" will unlikely bear any fruits of value. That said, if there is any non-idiomatic C code then the attempts of 'experts' in other languages to mangle C to look more like their language surely fall into this category (There are lots of examples where the author starts 'We can make C look more like fortran by defining our macros to replace these keywords..).

klibertp 10y ago

I'm not the author, but looking through the sibling comments I feel I should point out the obvious: startup time. It can take long seconds for Clojure REPL to start up. No other dynamic language I had contact with was that bad on that front. Even Erlang and Elixir boot up faster! Of course, I know about various ways of setting up a daemon process in the background, but then there is a need for reloading/hot-swapping code, which Clojure also fails to do well. Both Erlang and Common Lisp (also Emacs... even Emacs!) are better in this regard, both for different reasons. Anyway, Clojure overall performance impression - how fast it feels for users - is bad and will remain bad unless the startup problem is fixed.