5 ms·

and the costs they are willing to pay. Go/Rust just kill everything else (except maybe C++) for performance and resource needs. JVM requires so many resources j

and the costs they are willing to pay. Go/Rust just kill everything else (except maybe C++) for performance and resource needs. JVM requires so many resources just to run small apps.

pron 12d ago
Quite the opposite, and the reason is that you can't extrapolate from small programs to large ones. Low-level languages (like C++) incur some significant overheads as they grow large (because of essential constraints of low-level languages that prevent them from doing certain optimisations that matter mostly in large programs), and these are exactly the overheads the JVM is designed to reduce. In small or short-lived programs, the situation is different, because Java does have some warmup costs and some fixed memory overheads that matter when you're small or short-lived. Go's compiler and GC are pretty basic, and are certainly good enough for smaller things, but don't scale as well to high workloads. Just the other day a colleague tested Caffeine, an old and well-established Java caching library, and Moka, a Rust caching library with the same workload. Caffeine had the same latency as Moka across all percentiles at twice the throughput.
- p2detar 12d ago
  I use Java every day but just to point out that your info about Go‘s GC seems out of date. They switched to Green Tea in 1.25 (I think?) - new GC that even has AVX-512 optimizations. Not sure what you mean by basic about the compiler but it‘s very fast and supports a large set of platforms. That‘s not basic to me. We are using JDK25 and are considering rewriting parts of our product to Go because of lower memory pressure and faster startup time, i.e., cloud friendly. I actually love both languages.
  - pron 12d ago
    > I use Java every day but just to point out that your info about Go‘s GC seems out of date. I'm well aware that Go's GC has improved, but the moving algorithm was designed not just to be fast for a GC, but to be faster than no GC. So Go's new GC is good - for a mark and sweep collector. But it can't compete with a moving collector (the only thing that can is arenas, which are user-friendly only in Zig). > We are using JDK25 and are considering rewriting parts of our product to Go because of lower memory pressure and faster startup time, i.e., cloud friendly. Java probably will never have perfect warmup, but it's getting very good - https://openjdk.org/jeps/544 https://openjdk.org/jeps/544 - probably in JDK 28. As for memory, I think Java's memory strategy is generally misunderstood and I've given a talk about it: https://youtu.be/xr73mR7ii9M https://youtu.be/xr73mR7ii9M The footprint overhead exists to compensate for CPU utilisation when the CPU utilisation is more disruptive than memory usage. The problem is that many Java developers - and I'm not blaming them - don't understand this tradeoff and how to configure the JVM for optimal resource usage, but the great news is that a solution is coming soon, too - https://openjdk.org/jeps/8377305 https://openjdk.org/jeps/8377305 - also possibly in JDK 28. So it's very likely that both of these issues will be resolved six months from today, and you'd still get to enjoy better performance and telemetry than all alternatives.
    - nixon_why69 11d ago
      A couple data points, I like Java but I've seen metrics of container fleets at multiple companies that were memory constrained with low CPU usage sitting around underutilized. The reason in both cases was a bunch of memory-heavy yet CPU-efficient Java processes.
      - pron 11d ago
        When CPU utilisation is low, the heap can be set much smaller. Many don't know that, so in the next year we'll have the VM do it automatically: https://openjdk.org/jeps/8377305 https://openjdk.org/jeps/8377305 The amount of memory a Java program uses is whatever the setting is, not how much it "needs", because the need depends on the preference of the CPU/RAM tradeoff. But again, not many understand that, so we're making that automatic.
        nixon_why69 11d ago
        I'm sure both of the cases I'm thinking of could have been tuned better. Just saying that it's a default case that I've seen 2 places land, both of which had a lot of smart engineers following best practices. Maybe its food for thought for you in your position
        pron 11d ago
        Yes, this is why we're doing automatic heap sizing :)
    - zozbot234 11d ago
      CPU utilization is a red herring. Unless you're doing heavy number crunching (which these days heavily favors GPUs) the practical bottleneck on CPU utilization for large general purpose programs (especially when spanning multiple cores) is memory bandwidth. And moving GC is terrible for memory bandwidth compared to both Go-style concurrent GC (which doesn't have to do bulk moves) and manual memory management.
  - the-smug-one 12d ago
    Go's compiler is fast because it doesn't do as many advanced (read: computationally expensive) optimizations as other compilers do. No clue about Green Tea and how awesome it is :-). Lower memory pressure is certainly a difficult thing to beat Go at, Java (OpenJDK) is probably never gonna get there. You get a lot of other stuff, like better peak performance, instead. Btw, have you tried Leyden/AOT for better startup times? Curious about your experiences with that.
    - gf000 11d ago
      Actually, javac itself is plenty fast, pretty similar to Go's (it also barely does any optimization) It's usually the build systems that add quite some overhead.
    - p2detar 11d ago
      > Btw, have you tried Leyden/AOT for better startup times? Curious about your experiences with that. Nope, not yet. It's a good question given that up to now we used to deliver our product only on-premises and Windows Server-only, but this year we are now finally going with the Cloud, which means Docker containers and Linux. If I remember correctly Leyden required some sort of warm-up and training data collection before being able to effectively execute AOT, right? I need to freshen up my info on that. I did try GraalVM-compiled Java executables a couple of years ago and they were not bad, but the binaries were quite big (not a showstopper though) and the class-loading issues were kind of a PITA.
      - the-smug-one 11d ago
        A simplified way (and it is simplified) is that it takes your warmed up ordinary Java JIT JVM and dumps all of the "warmed up" stuff to an archive that's super quick to start. Then you skip a lot of interpretation, etc. You need to run your regular app while recording, in order to get something out of it.
- someone_19 12d ago
  > because of essential constraints of low-level languages that prevent them from doing certain optimisations that matter mostly in large programs Which specific optimizations are you referring to? In my experience, this is largely a myth; compared to Rust, you actually get even faster code right away. JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null).
  - pron 12d ago
    > Which specific optimizations are you referring to? A JIT with speculative optimisation and a moving GC. There are two constraints in low-level languages that trump any of their performance goals, one technical and one a matter of preference. The technical limitation is that they must use stable pointers (because they need to be low-level and so having an FFI layer that separates "hardware pointers" from "language pointers", as we have in Java defeats their main purpose). This means that you need to translate data storage or code storage to hardware addresses, and that interferes with both moving collection and with JIT compilation. The other constraint is that low-level languages value worst-case performance over the average-case and even amortised performance. These languages prefer an operation (e.g. dynamic dispatch) to be slow as long as it's never too slow. With a JIT (and I describe more later), virtual dispatch can be super-fast almost all the time, but occassionally, you'll hit a trap because the speculation was wrong, and then you need to deoptimise and recompile. > In my experience, this is largely a myth; compared to Rust, you actually get even faster code right away. We wouldn't be doing it in the first place if it was a myth. In a low-level language, you can get very fast code if you do some manual optimisations, but they don't easily scale as the program grows and evolves, because they're viral. The two most basic examples are dynamic dispatch (which is the most general mechanism, which scales the best in terms of program evolution) and shared heap objects (again, the most general mechanism). These become more common and less easily avoided over time, and they're slow in low-level languages because of the constraints I mentioned. That low-level languages make it harder and harder to preserve good performance over time as they evolve and grow is a problem familiar to those who've worked for years on large software written in a low level language (as I have). The JVM was designed, among other things, to solve this performance problem in large programs. > JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null). A JIT can make such languages decently fast, but that's not how it's used in Java. In Java it is used for speculative optimisation, which allows far more aggressive optimisation than an AOT compiler can do. E.g. by default, Java inlines and specialises virtual calls 15 levels deep. An AOT compiler can't do that or its code will explode. We get around it with selective use of templates in C++ (or comptime in Zig), but it has to be selective, and it's viral.
- NovaX 12d ago
  > Caffeine had the same latency as Moka across all percentiles at twice the throughput. Caffeine's next release has roughly 25% higher read throughput, with unchanged write throughput, thanks to fixing a false sharing mistake. That won't be visible in real workloads, but is fun nonetheless (500M reads/s on 8 cores).

and the costs they are willing to pay. Go/Rust just kill everything else (except maybe C++) for performance and resource needs. JVM requires so many resources just to run small apps.

pron 12d ago

Quite the opposite, and the reason is that you can't extrapolate from small programs to large ones. Low-level languages (like C++) incur some significant overheads as they grow large (because of essential constraints of low-level languages that prevent them from doing certain optimisations that matter mostly in large programs), and these are exactly the overheads the JVM is designed to reduce. In small or short-lived programs, the situation is different, because Java does have some warmup costs and some fixed memory overheads that matter when you're small or short-lived. Go's compiler and GC are pretty basic, and are certainly good enough for smaller things, but don't scale as well to high workloads. Just the other day a colleague tested Caffeine, an old and well-established Java caching library, and Moka, a Rust caching library with the same workload. Caffeine had the same latency as Moka across all percentiles at twice the throughput.

p2detar 12d ago
I use Java every day but just to point out that your info about Go‘s GC seems out of date. They switched to Green Tea in 1.25 (I think?) - new GC that even has AVX-512 optimizations. Not sure what you mean by basic about the compiler but it‘s very fast and supports a large set of platforms. That‘s not basic to me. We are using JDK25 and are considering rewriting parts of our product to Go because of lower memory pressure and faster startup time, i.e., cloud friendly. I actually love both languages.
- pron 12d ago
  > I use Java every day but just to point out that your info about Go‘s GC seems out of date. I'm well aware that Go's GC has improved, but the moving algorithm was designed not just to be fast for a GC, but to be faster than no GC. So Go's new GC is good - for a mark and sweep collector. But it can't compete with a moving collector (the only thing that can is arenas, which are user-friendly only in Zig). > We are using JDK25 and are considering rewriting parts of our product to Go because of lower memory pressure and faster startup time, i.e., cloud friendly. Java probably will never have perfect warmup, but it's getting very good - https://openjdk.org/jeps/544 https://openjdk.org/jeps/544 - probably in JDK 28. As for memory, I think Java's memory strategy is generally misunderstood and I've given a talk about it: https://youtu.be/xr73mR7ii9M https://youtu.be/xr73mR7ii9M The footprint overhead exists to compensate for CPU utilisation when the CPU utilisation is more disruptive than memory usage. The problem is that many Java developers - and I'm not blaming them - don't understand this tradeoff and how to configure the JVM for optimal resource usage, but the great news is that a solution is coming soon, too - https://openjdk.org/jeps/8377305 https://openjdk.org/jeps/8377305 - also possibly in JDK 28. So it's very likely that both of these issues will be resolved six months from today, and you'd still get to enjoy better performance and telemetry than all alternatives.
  - nixon_why69 11d ago
    A couple data points, I like Java but I've seen metrics of container fleets at multiple companies that were memory constrained with low CPU usage sitting around underutilized. The reason in both cases was a bunch of memory-heavy yet CPU-efficient Java processes.
    - pron 11d ago
      When CPU utilisation is low, the heap can be set much smaller. Many don't know that, so in the next year we'll have the VM do it automatically: https://openjdk.org/jeps/8377305 https://openjdk.org/jeps/8377305 The amount of memory a Java program uses is whatever the setting is, not how much it "needs", because the need depends on the preference of the CPU/RAM tradeoff. But again, not many understand that, so we're making that automatic.
      - nixon_why69 11d ago
        I'm sure both of the cases I'm thinking of could have been tuned better. Just saying that it's a default case that I've seen 2 places land, both of which had a lot of smart engineers following best practices. Maybe its food for thought for you in your position
        pron 11d ago
        Yes, this is why we're doing automatic heap sizing :)
  - zozbot234 11d ago
    CPU utilization is a red herring. Unless you're doing heavy number crunching (which these days heavily favors GPUs) the practical bottleneck on CPU utilization for large general purpose programs (especially when spanning multiple cores) is memory bandwidth. And moving GC is terrible for memory bandwidth compared to both Go-style concurrent GC (which doesn't have to do bulk moves) and manual memory management.
- the-smug-one 12d ago
  Go's compiler is fast because it doesn't do as many advanced (read: computationally expensive) optimizations as other compilers do. No clue about Green Tea and how awesome it is :-). Lower memory pressure is certainly a difficult thing to beat Go at, Java (OpenJDK) is probably never gonna get there. You get a lot of other stuff, like better peak performance, instead. Btw, have you tried Leyden/AOT for better startup times? Curious about your experiences with that.
  - gf000 11d ago
    Actually, javac itself is plenty fast, pretty similar to Go's (it also barely does any optimization) It's usually the build systems that add quite some overhead.
  - p2detar 11d ago
    > Btw, have you tried Leyden/AOT for better startup times? Curious about your experiences with that. Nope, not yet. It's a good question given that up to now we used to deliver our product only on-premises and Windows Server-only, but this year we are now finally going with the Cloud, which means Docker containers and Linux. If I remember correctly Leyden required some sort of warm-up and training data collection before being able to effectively execute AOT, right? I need to freshen up my info on that. I did try GraalVM-compiled Java executables a couple of years ago and they were not bad, but the binaries were quite big (not a showstopper though) and the class-loading issues were kind of a PITA.
    - the-smug-one 11d ago
      A simplified way (and it is simplified) is that it takes your warmed up ordinary Java JIT JVM and dumps all of the "warmed up" stuff to an archive that's super quick to start. Then you skip a lot of interpretation, etc. You need to run your regular app while recording, in order to get something out of it.
someone_19 12d ago
> because of essential constraints of low-level languages that prevent them from doing certain optimisations that matter mostly in large programs Which specific optimizations are you referring to? In my experience, this is largely a myth; compared to Rust, you actually get even faster code right away. JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null).
- pron 12d ago
  > Which specific optimizations are you referring to? A JIT with speculative optimisation and a moving GC. There are two constraints in low-level languages that trump any of their performance goals, one technical and one a matter of preference. The technical limitation is that they must use stable pointers (because they need to be low-level and so having an FFI layer that separates "hardware pointers" from "language pointers", as we have in Java defeats their main purpose). This means that you need to translate data storage or code storage to hardware addresses, and that interferes with both moving collection and with JIT compilation. The other constraint is that low-level languages value worst-case performance over the average-case and even amortised performance. These languages prefer an operation (e.g. dynamic dispatch) to be slow as long as it's never too slow. With a JIT (and I describe more later), virtual dispatch can be super-fast almost all the time, but occassionally, you'll hit a trap because the speculation was wrong, and then you need to deoptimise and recompile. > In my experience, this is largely a myth; compared to Rust, you actually get even faster code right away. We wouldn't be doing it in the first place if it was a myth. In a low-level language, you can get very fast code if you do some manual optimisations, but they don't easily scale as the program grows and evolves, because they're viral. The two most basic examples are dynamic dispatch (which is the most general mechanism, which scales the best in terms of program evolution) and shared heap objects (again, the most general mechanism). These become more common and less easily avoided over time, and they're slow in low-level languages because of the constraints I mentioned. That low-level languages make it harder and harder to preserve good performance over time as they evolve and grow is a problem familiar to those who've worked for years on large software written in a low level language (as I have). The JVM was designed, among other things, to solve this performance problem in large programs. > JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null). A JIT can make such languages decently fast, but that's not how it's used in Java. In Java it is used for speculative optimisation, which allows far more aggressive optimisation than an AOT compiler can do. E.g. by default, Java inlines and specialises virtual calls 15 levels deep. An AOT compiler can't do that or its code will explode. We get around it with selective use of templates in C++ (or comptime in Zig), but it has to be selective, and it's viral.
NovaX 12d ago
> Caffeine had the same latency as Moka across all percentiles at twice the throughput. Caffeine's next release has roughly 25% higher read throughput, with unchanged write throughput, thanks to fixing a false sharing mistake. That won't be visible in real workloads, but is fun nonetheless (500M reads/s on 8 cores).