5 ms·
Serious question, if performance is the lynchpin, why write it in Java? Especially considering they use unsafe "heavily", for big joins they could easily just
by pixelpoet 3y ago
Serious question, if performance is the lynchpin, why write it in Java?
Especially considering they use unsafe "heavily", for big joins they could easily just call out to some native code if the surrounding code reaaaaally must be Java (again, why?). It's the worst of both worlds using unsafe Java: you don't get native speed, there's loads of memory overhead from everything being an Object (besides the rest of the VM stuff), and get to "enjoy" GC pauses in the middle of your hot loops, and with fewer safety guarantees than something like Rust.
- bluestreak 3y agothis is THE number one question being asked about QuestDB :) There is a thread that might help https://news.ycombinator.com/item?id=37557880 https://news.ycombinator.com/item?id=37557880
- pixelpoet 3y agoI see... but that seems a little weak considering it's a funded product, the first adjective they use to describe it is "fast", and good old C++ would totally slay it. The author has a C++ background, maybe he could spend an afternoon trying that? I couldn't even try to count the number of great posts I've read about fast hash tables from e.g. Paul Khuong alone...
- gavinray 3y agoIn the most non-inflammatory way possible: I am not sure I'm convinced of the performance benefits of crossing a JNI boundary to invoke a method in another language, versus letting the JVM JIT and optimize native JVM code. Would be interesting to see benchmarks here, do you know if there are any public ones available, specific to QuestDB's interop code?
- bluestreak 3y agoIn certain situations, crossing the JNI boundary can be advantageous. When data resides in "native" memory, outside the Java heap, the coordination of concurrent execution logic can be handled in Java, while the actual execution occurs in C++ or Rust. In this context, the negligible penalty for crossing the JNI boundary once per 10 million rows pales in comparison to the substantial benefits achievable through C++ optimization or CPU-specific assembly.
- lukax 3y agoThey started with Java and tried to use as little GC as possible and started writing hot loops in C, C++ and Rust. Logic is in Java, other stuff is mostly native. https://questdb.io/blog/leveraging-rust-in-our-high-performance-java-database/ https://questdb.io/blog/leveraging-rust-in-our-high-performa...
- toasted-subs 3y agoYeah, I genuinely don't understand why so many people reach for Java for everything.
- marginalia_nu 3y agoIt's a very well rounded language is why, and whatever papercuts exist are either on the way out in newer/future java versions, or made up for by the tooling ecosystem.
- deleted 3y ago[deleted]
- dimgl 3y agoInterestingly I recently saw an MQ broker written in Java and it seemed to have some pretty impressive performance. I'm not a huge fan of Java's DX but I have to admit, that's some serious performance. Can't remember the name but it was on the homepage this week I believe.
- winrid 3y agoIsn't javas DX pretty great? I mean not the huge XML heavy frameworks but Java itself and the IDEs work well...