3 ms·

It can though. We are a java shop that does HPC and deep learning. Our work is open source. We've been around 2.5 going on 3 years now. Over the years we've sur

It can though. We are a java shop that does HPC and deep learning. Our work is open source. We've been around 2.5 going on 3 years now. Over the years we've surpassed bottleneck after bottleneck in the JVM. It's a faster wrapper around c code for us. We maintain and created: https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp which wraps a lot of c++ components. This is also how we do GPUs and the like with our own memory management for our "numpy for java": https://github.com/deeplearning4j/nd4j/tree/master/nd4j-backends/nd4j-backend-impls/nd4j-cuda https://github.com/deeplearning4j/nd4j/tree/master/nd4j-back... Short of it "off heap and NO GC" matters a lot. We have 1 c codebase we use that's meant to be controlled via JNI here: https://github.com/deeplearning4j/libnd4j https://github.com/deeplearning4j/libnd4j The speed gains we've seen are massive. Java can't compete with good ole simd and the like for numerical computing. We are a big advocate of the JVM as a platform but let's be clear about its weaknesses. You need to use unsafe (see: netty,aeron, and any other low level database written in java or using java as the core language with some c++)and other tricks out of the box for real performance.

pjmlp 10y ago
The problem is that you are using it in areas, HPC and deep learning, where using plain C++ would be a better option. I think. There is a reason why Java 10 aims to improve Java's story in regard to mechanical sympathy.
- agibsonccc 10y ago
  We wrote a lot of that stack ourselves. And our compute code is in c++. There is a lot of stuff going on in big data land (including us) wrapping a lot of the c++ in java and enabling people to use the JVM for what it's good at (data access). We are pushing this with nvidia: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-deep-learning/ https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-... I also work pretty closely with a lot of the spark/gpu folks at IBM. While I do largely agree with you, we have our own JNI compiler called javacpp that alleviates a lot of those concerns already: https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp Having our own pointer class and doing our own memory management has helped a lot. What you're talking about is using java for the compute. There's no reason you can't wrap that in a runtime that most people know how to use. A lot of python folks do that now..but then you have to deal with python's limitations at which point the majority of your code (way more than needs be) will end up being in c anyways vs java where you can write a significant part of your app in java and have it be fast out of the box.
  - pjmlp 10y ago
    Thanks for the clarification. I am mostly a line-of-business developer doing enterprise consultancy and that is how we use C++, just as infrastructure language when either JVM or .NET stacks need a bit of outside help. Just wondering if sometimes having a 100% C++ solution would be a better approach than the added integration effort it requires, on the other hand, without people like you guys we wouldn't have access to nice tooling in Java for similar work, so congratulations on the work thus far and all the best for the project.
pron 10y ago
I'm willing to bet that 99% of the performance difference will be solved by value types. Anyway, Cliff Click, who wrote H2O[1], a large machine learning platform, reports achieving Fortran speeds (i.e. maximum throughput) with pure Java[2]. This means that a C application won't even be 1% faster. > Java can't compete with good ole simd and the like for numerical computing. Oh, sure, for some specialized use cases of course that's true, but you could use OpenCL in Java, too. With 10+ years with C/C++ and 10+ years with Java, I'd bet on Java when it comes to performance bang-for-the-buck almost in every case (given that it's a large app), and even more than that the more concurrent, complex and unpredictable the app is (but this requires carefully looking at the design). Value types will make Java competitive in absolute terms in more and more domains. Also, with the new JIT (Graal) you can control machine-code generation at whatever level of detail you want. > You need to use unsafe (see: netty,aeron, and any other low level database written in java or using java as the core language with some c++)and other tricks out of the box for real performance. They're not using unsafe for throughput but (mostly) for latency. That's a whole other matter. Also, some of those use "mechanical sympathy" as a driver of performance instead of algorithms that the JVM makes easier. I've built a concurrent DB in pure Java that relies on synchronization that would require at least double the effort if I were to write it in C (hazard pointers, etc.), and may not even have better performance. [1]: https://github.com/h2oai https://github.com/h2oai [2]: https://vimeo.com/105743312 https://vimeo.com/105743312
- agibsonccc 10y ago
  >> I'm willing to bet that 99% of the performance difference will be solved by value types. Anyway, Cliff Click, who wrote H2O[1], a large machine learning platform, reports achieving Fortran speeds (i.e. maximum throughput) with pure Java[2]. This means that a C application won't even be 1% faster. I've had a personal conversation with Cliff himself. Java no matter what you do can't deal with hardware acceleration and gpus. We agreed on that. Numerical software is a different beast. I also kept mentioning "simd instructions" as well as things like openmp. You are talking about systems software. Unfortunately that matters a lot for machine learning. The axis along which you can get equivalent speeds should be specified here. "values types" != "runs on faster chips" You aren't likely to beat intel or nvidia's compilers at their own game here . Java will always be playing catch up to last gen's tech there. Disclosure: I'm more than aware of what's going on in the space. We compete with them for customers and have a very clear understanding of their offerings. H20 has a great k/v store based on the exact mechanics you're talking about. That's about it though. Also of note: Cliff doesn't work on h20 anymore: https://twitter.com/cliff_click/status/700817408110399492 https://twitter.com/cliff_click/status/700817408110399492 >> Oh, sure, for some specialized use cases of course that's true, but you could use OpenCL in Java, too.... OpenCL isn't exactly the industry standard for this stuff. You always end up using cuda, and you always end up dropping down to c. There's just no way to avoid that if you want the fastest out there. Another disclosure, we work closely with nvidia and I may be biased: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-deep-learning/ https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-... I agree with you on the last part, but I keep mentioning "numerical software" for a reason. There are certain things the JVM is good at, writing a database and systems software is one of those things. There are still bits of HDFS in c++ though. I don't think you'll be able to get around having bits of your code in c which is what I emphasize here.

pjmlp 10y ago

The problem is that you are using it in areas, HPC and deep learning, where using plain C++ would be a better option. I think. There is a reason why Java 10 aims to improve Java's story in regard to mechanical sympathy.

agibsonccc 10y ago
We wrote a lot of that stack ourselves. And our compute code is in c++. There is a lot of stuff going on in big data land (including us) wrapping a lot of the c++ in java and enabling people to use the JVM for what it's good at (data access). We are pushing this with nvidia: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-deep-learning/ https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-... I also work pretty closely with a lot of the spark/gpu folks at IBM. While I do largely agree with you, we have our own JNI compiler called javacpp that alleviates a lot of those concerns already: https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp Having our own pointer class and doing our own memory management has helped a lot. What you're talking about is using java for the compute. There's no reason you can't wrap that in a runtime that most people know how to use. A lot of python folks do that now..but then you have to deal with python's limitations at which point the majority of your code (way more than needs be) will end up being in c anyways vs java where you can write a significant part of your app in java and have it be fast out of the box.
- pjmlp 10y ago
  Thanks for the clarification. I am mostly a line-of-business developer doing enterprise consultancy and that is how we use C++, just as infrastructure language when either JVM or .NET stacks need a bit of outside help. Just wondering if sometimes having a 100% C++ solution would be a better approach than the added integration effort it requires, on the other hand, without people like you guys we wouldn't have access to nice tooling in Java for similar work, so congratulations on the work thus far and all the best for the project.

pron 10y ago

I'm willing to bet that 99% of the performance difference will be solved by value types. Anyway, Cliff Click, who wrote H2O[1], a large machine learning platform, reports achieving Fortran speeds (i.e. maximum throughput) with pure Java[2]. This means that a C application won't even be 1% faster. > Java can't compete with good ole simd and the like for numerical computing. Oh, sure, for some specialized use cases of course that's true, but you could use OpenCL in Java, too. With 10+ years with C/C++ and 10+ years with Java, I'd bet on Java when it comes to performance bang-for-the-buck almost in every case (given that it's a large app), and even more than that the more concurrent, complex and unpredictable the app is (but this requires carefully looking at the design). Value types will make Java competitive in absolute terms in more and more domains. Also, with the new JIT (Graal) you can control machine-code generation at whatever level of detail you want. > You need to use unsafe (see: netty,aeron, and any other low level database written in java or using java as the core language with some c++)and other tricks out of the box for real performance. They're not using unsafe for throughput but (mostly) for latency. That's a whole other matter. Also, some of those use "mechanical sympathy" as a driver of performance instead of algorithms that the JVM makes easier. I've built a concurrent DB in pure Java that relies on synchronization that would require at least double the effort if I were to write it in C (hazard pointers, etc.), and may not even have better performance. [1]: https://github.com/h2oai https://github.com/h2oai [2]: https://vimeo.com/105743312 https://vimeo.com/105743312

agibsonccc 10y ago
>> I'm willing to bet that 99% of the performance difference will be solved by value types. Anyway, Cliff Click, who wrote H2O[1], a large machine learning platform, reports achieving Fortran speeds (i.e. maximum throughput) with pure Java[2]. This means that a C application won't even be 1% faster. I've had a personal conversation with Cliff himself. Java no matter what you do can't deal with hardware acceleration and gpus. We agreed on that. Numerical software is a different beast. I also kept mentioning "simd instructions" as well as things like openmp. You are talking about systems software. Unfortunately that matters a lot for machine learning. The axis along which you can get equivalent speeds should be specified here. "values types" != "runs on faster chips" You aren't likely to beat intel or nvidia's compilers at their own game here . Java will always be playing catch up to last gen's tech there. Disclosure: I'm more than aware of what's going on in the space. We compete with them for customers and have a very clear understanding of their offerings. H20 has a great k/v store based on the exact mechanics you're talking about. That's about it though. Also of note: Cliff doesn't work on h20 anymore: https://twitter.com/cliff_click/status/700817408110399492 https://twitter.com/cliff_click/status/700817408110399492 >> Oh, sure, for some specialized use cases of course that's true, but you could use OpenCL in Java, too.... OpenCL isn't exactly the industry standard for this stuff. You always end up using cuda, and you always end up dropping down to c. There's just no way to avoid that if you want the fastest out there. Another disclosure, we work closely with nvidia and I may be biased: https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-deep-learning/ https://blogs.nvidia.com/blog/2016/10/06/how-skymind-nvidia-... I agree with you on the last part, but I keep mentioning "numerical software" for a reason. There are certain things the JVM is good at, writing a database and systems software is one of those things. There are still bits of HDFS in c++ though. I don't think you'll be able to get around having bits of your code in c which is what I emphasize here.