5 ms·

Java is great for a lot of things. That's why you see large distributed systems projects that power large companies being written in java. See presto at faceboo

Java is great for a lot of things. That's why you see large distributed systems projects that power large companies being written in java. See presto at facebook or kafka at linkedin for example. It's not as good at for loops though. If you need it for hpc you should still be using c++ underneath somewhere for simd instructions and other neat tricks only available in lower level languages. I look at java as a better python or: "frontend to c++ code"

_asummers 10y ago
Even with HPC (the Azul Systems folks would object to your claim), you just have to code in a foreign style of not allocating memory. Once you get that style in your tight loops, Java can run very very fast.
- agibsonccc 10y ago
  I've debated with cliff click about this before actually. There was a consensus that the JVM can support certain kinds of operations for that stuff (eg: basic SIMD) with that coding style. It can't do everything though. It's a lot better to go JNI for 99% of this stuff though. I'd also love to know when java has first class support for gpus (you know where hpc actually runs?). I'll continue relying on c++ for that stuff. It's also what nvidia supports first class. If you have any first hand experience with this stuff I'd also love to hear your thoughts on this as well. Better yet: Here's a mailing list where these people (including me) sit. Go ahead and peruse it. I've learned a lot being on here. I'm STILL of the opinion the JVM can't do most of these things well after having run the numbers: https://groups.google.com/forum/#!forum/mechanical-sympathy https://groups.google.com/forum/#!forum/mechanical-sympathy For cpu I'll be sticking with openmp for the long term. Don't even get me started with cpu specific optimizations (eg: different intel generations)
  - deleted 10y ago
    [deleted]
  - _asummers 10y ago
    Fair enough, I personally don't have first hand experience with the super tight loop stuff on the JVM (though that seems like it'd be fun!). The JVM has always been fast enough for MY needs, but I realize that's not true for everyone. Thanks for the link, that seems like a good mailing list.
    - agibsonccc 10y ago
      Nah I mean this stuff IS hard. That's why I offered that mailing list. Like I said a lot of people from azul and a lot of the hardcore jvm heads sit on there. If there's a way to make java faster for that great. You are are right that the jvm DOES support some of the tight loop stuff. It's harder to do and leads to messy code though. In C++ you can write maintainable code and add pragmas. It's a lot cleaner and with javacpp (https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp) we just generate the jni bindings. We found that to be easier in practice. I'm admittedly not THAT familiar with the jvm internals. I have just tried to get it to work and have did enough reading to know jni will just be cleaner for that stuff. The JVM is certainly fast enough, just mainly for distributed systems code (eg: databases, message queues) Honestly that's what makes it appealing for me. I can write fast production systems and tweak it where necessary for the HPC apps. Java provides a fast and safe baseline for 90% of the stuff I'd want it to do. We tried for about a year to get it to work for lin alg and finally said screw it let's just do c++.
im_down_w_otp 10y ago
I completely disagree that Java is well suited for distributed systems. Several such systems are written in Java solely because Java is very popular generally and is the lingua franca of "enterprise" software development, not because Java is particularly well suited to the distributed systems problem space intrinsically. Also, Kafka is written in Scala, not Java. Though I don't think Scala is particularly any better suited to the problem either.
- agibsonccc 10y ago
  Tell that to facebook,twitter,linkedin, most big banks? The JVM (whether you like it or not) powers most of the bigger database systems in the world. New code is still being written for the jvm. Lightbend and pivotal are also companies setting up large systems on the jvm. Seems to work fine for them. What are the alternatives? go? Your point re: scala. Scala is STILL based on java. There isn't much of a difference for speed here. Akka and its ilk still rely on netty for the underlying transport mechanism (written in java) I'm very much well aware of what's written in scala. That includes spark and kafka among other things which is great. Those STILL rely on java libraries though. Go is still slower yet: https://www.techempower.com/blog/2016/02/25/framework-benchmarks-round-12/ https://www.techempower.com/blog/2016/02/25/framework-benchm... The other might be what? erlang? Good luck finding developers. The JVM is still the only platform that has not only big data mind share but things like microservices frameworks like lagom and spring boot while ALSO having things like message queues being written for it.
  - im_down_w_otp 10y ago
    Overwhelmingly databases are not written in Java. Most of them are written in C and C++. Secondly, I'm not denying that lot's of systems are built on Java. But they're built on Java because those places already have super heavy investment in Java ecosystem libraries, tools, and engineering resources. Not because Java or the JVM is somehow intrinsically awesome for building distributed databases or distributed systems things. In fact it's quite the contrary. It's an epic pain in the ass to keep heap growth and GC latency under control in most of these systems. The former being critical for node stability and the latter being critical for dead-node detection consistency (among other things). I used to be a distributed database engineer, and I count among my friends many people who still work at Elastic.co, Confluent, MapR, Cloudera, Twitter, and Couchbase. I currently work for a big FinTech company building core distributed transaction processing infrastructure, and I am 100% certain that compelling the use of Java for certain systems development tasks has nothing to do with it as a piece of technology and 100% to due with politics.

_asummers 10y ago

Even with HPC (the Azul Systems folks would object to your claim), you just have to code in a foreign style of not allocating memory. Once you get that style in your tight loops, Java can run very very fast.

agibsonccc 10y ago
I've debated with cliff click about this before actually. There was a consensus that the JVM can support certain kinds of operations for that stuff (eg: basic SIMD) with that coding style. It can't do everything though. It's a lot better to go JNI for 99% of this stuff though. I'd also love to know when java has first class support for gpus (you know where hpc actually runs?). I'll continue relying on c++ for that stuff. It's also what nvidia supports first class. If you have any first hand experience with this stuff I'd also love to hear your thoughts on this as well. Better yet: Here's a mailing list where these people (including me) sit. Go ahead and peruse it. I've learned a lot being on here. I'm STILL of the opinion the JVM can't do most of these things well after having run the numbers: https://groups.google.com/forum/#!forum/mechanical-sympathy https://groups.google.com/forum/#!forum/mechanical-sympathy For cpu I'll be sticking with openmp for the long term. Don't even get me started with cpu specific optimizations (eg: different intel generations)
- deleted 10y ago
  [deleted]
- _asummers 10y ago
  Fair enough, I personally don't have first hand experience with the super tight loop stuff on the JVM (though that seems like it'd be fun!). The JVM has always been fast enough for MY needs, but I realize that's not true for everyone. Thanks for the link, that seems like a good mailing list.
  - agibsonccc 10y ago
    Nah I mean this stuff IS hard. That's why I offered that mailing list. Like I said a lot of people from azul and a lot of the hardcore jvm heads sit on there. If there's a way to make java faster for that great. You are are right that the jvm DOES support some of the tight loop stuff. It's harder to do and leads to messy code though. In C++ you can write maintainable code and add pragmas. It's a lot cleaner and with javacpp (https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp) we just generate the jni bindings. We found that to be easier in practice. I'm admittedly not THAT familiar with the jvm internals. I have just tried to get it to work and have did enough reading to know jni will just be cleaner for that stuff. The JVM is certainly fast enough, just mainly for distributed systems code (eg: databases, message queues) Honestly that's what makes it appealing for me. I can write fast production systems and tweak it where necessary for the HPC apps. Java provides a fast and safe baseline for 90% of the stuff I'd want it to do. We tried for about a year to get it to work for lin alg and finally said screw it let's just do c++.

im_down_w_otp 10y ago

I completely disagree that Java is well suited for distributed systems. Several such systems are written in Java solely because Java is very popular generally and is the lingua franca of "enterprise" software development, not because Java is particularly well suited to the distributed systems problem space intrinsically. Also, Kafka is written in Scala, not Java. Though I don't think Scala is particularly any better suited to the problem either.

agibsonccc 10y ago
Tell that to facebook,twitter,linkedin, most big banks? The JVM (whether you like it or not) powers most of the bigger database systems in the world. New code is still being written for the jvm. Lightbend and pivotal are also companies setting up large systems on the jvm. Seems to work fine for them. What are the alternatives? go? Your point re: scala. Scala is STILL based on java. There isn't much of a difference for speed here. Akka and its ilk still rely on netty for the underlying transport mechanism (written in java) I'm very much well aware of what's written in scala. That includes spark and kafka among other things which is great. Those STILL rely on java libraries though. Go is still slower yet: https://www.techempower.com/blog/2016/02/25/framework-benchmarks-round-12/ https://www.techempower.com/blog/2016/02/25/framework-benchm... The other might be what? erlang? Good luck finding developers. The JVM is still the only platform that has not only big data mind share but things like microservices frameworks like lagom and spring boot while ALSO having things like message queues being written for it.
- im_down_w_otp 10y ago
  Overwhelmingly databases are not written in Java. Most of them are written in C and C++. Secondly, I'm not denying that lot's of systems are built on Java. But they're built on Java because those places already have super heavy investment in Java ecosystem libraries, tools, and engineering resources. Not because Java or the JVM is somehow intrinsically awesome for building distributed databases or distributed systems things. In fact it's quite the contrary. It's an epic pain in the ass to keep heap growth and GC latency under control in most of these systems. The former being critical for node stability and the latter being critical for dead-node detection consistency (among other things). I used to be a distributed database engineer, and I count among my friends many people who still work at Elastic.co, Confluent, MapR, Cloudera, Twitter, and Couchbase. I currently work for a big FinTech company building core distributed transaction processing infrastructure, and I am 100% certain that compelling the use of Java for certain systems development tasks has nothing to do with it as a piece of technology and 100% to due with politics.