5 ms·

Java is only slow to start the VM. If your process is long-lived it's exceedingly quick because you only pay the startup penalty once. "GC pauses" are greatly

by nvy 3y ago

Java is only slow to start the VM. If your process is long-lived it's exceedingly quick because you only pay the startup penalty once. "GC pauses" are greatly exaggerated in terms of impact and frankly for the vast majority of uses cases GC simply doesn't become an issue. The JVM is really really good because at one point or another they had basically every luminary in the field working on it.

tetha 3y ago
The fear of GC pauses is mostly outdated. It was right in the past, but it is becoming less and less of an issue with the new GC work coming online. And for small heaps in web application servers, the pauses were a blip on the high percentiles of response times and you usually had a dozen other issues before that. It's similar to SWAP in Linux. It was implemented somewhat meh in older kernels, and usually if swap got hit, the system died anyhow. So there was no real difference between random processes being OOM-killed or the system grinding to a halt swapping. Modern kernels in the 4+ line have received quite a bit of work on the swap handling and swap is used a lot and very cleverly to eek out just a bit more available memory more quickly. Old habits die hard though and it takes time for old knowledge to change.
- pkolaczk 3y ago
  The problem of GC pauses is solved only if you can afford to waste 5x-20x more memory than the app is really using. Otherwise those low-pause modern collectors can burn a significant number of CPU cycles or even fall into emergency STW path (when they can't keep up with garbage production rate).
  - xxs 3y ago
    If the allocations die trivially (the extreme/vast majority do) in the young gen, STW doesn't occur. Indeed being aware, how the GC operates is important to write a decent Java application. So is understanding how the hardware works in general.
    - pkolaczk 3y ago
      > If the allocations die trivially (the extreme/vast majority do) in the young gen Low pause GCs (ZGC, Shenadoah) for Java are not generational yet. Also, even if they were, a very high temporary object allocation rate increases young gen GC frequency and thus increases the number of objects pushed to old gen eventually. And it is not like those young gen GCs are free either. They burn quite some CPU time and they cause micro-pauses - the GC has to scan parts of the heap to learn which objects are reachable and then it has to copy the survivors. In practice, high allocation rate requires a decent amount of overhead RAM to make that process efficient. It doesn't matter if pauses are only 10 ms short if you do 50 of them per second. ;) In non-GCed languages those trivial short-term objects are typically allocated on stack and their allocation/deallocation is trivial and doesn't pause at all.
      - xxs 3y ago
        >the GC has to scan parts of the heap Hmm, for generation GCs, it should use Card marking, the pointers in the tenured gen should have means to be trivially determined where they belong to. With 64bit pointers, there is space for quite a lot of metadata, incl. the Class (or most commonly allocated/used classes).
        pkolaczk 3y ago
        Card marking does not identify pointers directly, but only identifies parts of the heap that may contain pointers to the new gen. So those parts still need to be scanned. Surely, it is way less work than scanning the whole heap, but still non-zero.
      - the-smug-one 3y ago
        >Low pause GCs (ZGC, Shenadoah) for Java are not generational yet. They are. GenZGC is merged into mainline and got into the latest JDK, GenShenandoah didn't manage but will be in next release. >The problem of GC pauses is solved only if you can afford to waste 5x-20x more memory than the app is really using I don't think that is true. ZGC uses multi-mapping in order to dereference its colored pointers, this causes some tools to report excessive memory usage but nothing actually hits RAM.
- kalleboo 3y ago
  > it is becoming less and less of an issue with the new GC work coming online I feel like I've been hearing this same line for 20 years now
Shorel 3y ago
Java has a ridiculously fast new operator. Much faster than the one in C++. It works by allocating all or most needed memory at program start, instead of asking the operating system for it every time. But, as soon as you don't use heap memory, and use the stack, C++ is again much, much faster than Java. It all depends on memory management.
- seanhunter 3y ago
  C++'s new operator can also do this using "placement new" semantics[1], which usually means making (and using) a custom allocator. You're still using the normal new operator, just passing extra parameters to do the placement you want. To your second point about raw speed, one of the extraordinary things we found in long-running processes performing computation using real-time marketdata at Goldman was that the HotSpot JVM was able to optimize java programs through the day due to their usage, so if you started them each day they would actually end up faster than the C++ versions at the end of the day even though they would start off slower. That's not due to memory allocation it's due to things like inlining of functions. The implication of that is that if you very carefully inlined all the functions appropriately in the C++ version based on profiling actual usage you would be able to achieve the same result, but for the JVM it just happens automatically without you doing anything. [1] Search https://en.cppreference.com/w/cpp/language/new https://en.cppreference.com/w/cpp/language/new for "Placement New"
- pkolaczk 3y ago
  > Java has a ridiculously fast new operator. Much faster than the one in C++. Can you point us to some benchmarks? I don't doubt it is faster, but I doubt the "ridiculously" part. Last time I measured it was only a tad faster than jemalloc (~20%) if you allowed the benchmark to run long enough for GC to start cleaning up. Unfortunately I haven't saved it (it was a very informal benchmark). I'm curios to see that esp. on modern low-pause GCs.
  - the-smug-one 3y ago
    It's just pointer bumping (still). If you're continously using new memory then soft page faults and finding physical backing via kernel will dominate. mallocsen want to avoid this scenario, so you often see malloc using more memory than necessary. Anyway, don't use new in C++ is my opinion.
TacticalCoder 3y ago
> Java is only slow to start the VM And it's not even that slow anymore compared to, say, starting the JVM in 2010. Some numbers for those curious... On modern hardware launching the JVM to run a program immediately exiting takes less than 100 ms (so does starting Emacs complete with its GUI and running some elisp code exiting Emacs: 80 ms on my Ryzen 7000 series including reading from the M.2 NVMe PCIe 4.0 x4 SSD). It's once you start loading lots of classes that JVM startup time can be slow. One example would be a Clojure program doing nothing besides exiting: thousands of Java classes being used and you get into 1.2 seconds territory to do nothing. 12x slower than a Java program doing nothing. As a sidenote for both Java and Clojure there are now ways to reduce startup time, like using GraalVM (which, for example, Babashka, a natively compiled Clojure interpreter, making Clojure startup so fast it can be used for scripts). > The JVM is really really good because at one point or another they had basically every luminary in the field working on it. I agree. The JVM is an impressive piece of machinery and it'll even give you, say, an AIOOBE (ArrayIndexOutOfBoungsException) instead of an exploit if you fuck up.
- nvy 3y ago
  >so does starting Emacs complete with its GUI I'm with you all the way and am a daily emacs user but I'm not sure I'd point to emacs as a good performance comparison, haha. Emacs has always been a bit of a dog in my experience.

by nvy 3y ago

tetha 3y ago

The fear of GC pauses is mostly outdated. It was right in the past, but it is becoming less and less of an issue with the new GC work coming online. And for small heaps in web application servers, the pauses were a blip on the high percentiles of response times and you usually had a dozen other issues before that. It's similar to SWAP in Linux. It was implemented somewhat meh in older kernels, and usually if swap got hit, the system died anyhow. So there was no real difference between random processes being OOM-killed or the system grinding to a halt swapping. Modern kernels in the 4+ line have received quite a bit of work on the swap handling and swap is used a lot and very cleverly to eek out just a bit more available memory more quickly. Old habits die hard though and it takes time for old knowledge to change.

pkolaczk 3y ago
The problem of GC pauses is solved only if you can afford to waste 5x-20x more memory than the app is really using. Otherwise those low-pause modern collectors can burn a significant number of CPU cycles or even fall into emergency STW path (when they can't keep up with garbage production rate).
- xxs 3y ago
  If the allocations die trivially (the extreme/vast majority do) in the young gen, STW doesn't occur. Indeed being aware, how the GC operates is important to write a decent Java application. So is understanding how the hardware works in general.
  - pkolaczk 3y ago
    > If the allocations die trivially (the extreme/vast majority do) in the young gen Low pause GCs (ZGC, Shenadoah) for Java are not generational yet. Also, even if they were, a very high temporary object allocation rate increases young gen GC frequency and thus increases the number of objects pushed to old gen eventually. And it is not like those young gen GCs are free either. They burn quite some CPU time and they cause micro-pauses - the GC has to scan parts of the heap to learn which objects are reachable and then it has to copy the survivors. In practice, high allocation rate requires a decent amount of overhead RAM to make that process efficient. It doesn't matter if pauses are only 10 ms short if you do 50 of them per second. ;) In non-GCed languages those trivial short-term objects are typically allocated on stack and their allocation/deallocation is trivial and doesn't pause at all.
    - xxs 3y ago
      >the GC has to scan parts of the heap Hmm, for generation GCs, it should use Card marking, the pointers in the tenured gen should have means to be trivially determined where they belong to. With 64bit pointers, there is space for quite a lot of metadata, incl. the Class (or most commonly allocated/used classes).
      - pkolaczk 3y ago
        Card marking does not identify pointers directly, but only identifies parts of the heap that may contain pointers to the new gen. So those parts still need to be scanned. Surely, it is way less work than scanning the whole heap, but still non-zero.
    - the-smug-one 3y ago
      >Low pause GCs (ZGC, Shenadoah) for Java are not generational yet. They are. GenZGC is merged into mainline and got into the latest JDK, GenShenandoah didn't manage but will be in next release. >The problem of GC pauses is solved only if you can afford to waste 5x-20x more memory than the app is really using I don't think that is true. ZGC uses multi-mapping in order to dereference its colored pointers, this causes some tools to report excessive memory usage but nothing actually hits RAM.
kalleboo 3y ago
> it is becoming less and less of an issue with the new GC work coming online I feel like I've been hearing this same line for 20 years now

Shorel 3y ago

Java has a ridiculously fast new operator. Much faster than the one in C++. It works by allocating all or most needed memory at program start, instead of asking the operating system for it every time. But, as soon as you don't use heap memory, and use the stack, C++ is again much, much faster than Java. It all depends on memory management.

seanhunter 3y ago
C++'s new operator can also do this using "placement new" semantics[1], which usually means making (and using) a custom allocator. You're still using the normal new operator, just passing extra parameters to do the placement you want. To your second point about raw speed, one of the extraordinary things we found in long-running processes performing computation using real-time marketdata at Goldman was that the HotSpot JVM was able to optimize java programs through the day due to their usage, so if you started them each day they would actually end up faster than the C++ versions at the end of the day even though they would start off slower. That's not due to memory allocation it's due to things like inlining of functions. The implication of that is that if you very carefully inlined all the functions appropriately in the C++ version based on profiling actual usage you would be able to achieve the same result, but for the JVM it just happens automatically without you doing anything. [1] Search https://en.cppreference.com/w/cpp/language/new https://en.cppreference.com/w/cpp/language/new for "Placement New"
pkolaczk 3y ago
> Java has a ridiculously fast new operator. Much faster than the one in C++. Can you point us to some benchmarks? I don't doubt it is faster, but I doubt the "ridiculously" part. Last time I measured it was only a tad faster than jemalloc (~20%) if you allowed the benchmark to run long enough for GC to start cleaning up. Unfortunately I haven't saved it (it was a very informal benchmark). I'm curios to see that esp. on modern low-pause GCs.
- the-smug-one 3y ago
  It's just pointer bumping (still). If you're continously using new memory then soft page faults and finding physical backing via kernel will dominate. mallocsen want to avoid this scenario, so you often see malloc using more memory than necessary. Anyway, don't use new in C++ is my opinion.

TacticalCoder 3y ago

> Java is only slow to start the VM And it's not even that slow anymore compared to, say, starting the JVM in 2010. Some numbers for those curious... On modern hardware launching the JVM to run a program immediately exiting takes less than 100 ms (so does starting Emacs complete with its GUI and running some elisp code exiting Emacs: 80 ms on my Ryzen 7000 series including reading from the M.2 NVMe PCIe 4.0 x4 SSD). It's once you start loading lots of classes that JVM startup time can be slow. One example would be a Clojure program doing nothing besides exiting: thousands of Java classes being used and you get into 1.2 seconds territory to do nothing. 12x slower than a Java program doing nothing. As a sidenote for both Java and Clojure there are now ways to reduce startup time, like using GraalVM (which, for example, Babashka, a natively compiled Clojure interpreter, making Clojure startup so fast it can be used for scripts). > The JVM is really really good because at one point or another they had basically every luminary in the field working on it. I agree. The JVM is an impressive piece of machinery and it'll even give you, say, an AIOOBE (ArrayIndexOutOfBoungsException) instead of an exploit if you fuck up.

nvy 3y ago
>so does starting Emacs complete with its GUI I'm with you all the way and am a daily emacs user but I'm not sure I'd point to emacs as a good performance comparison, haha. Emacs has always been a bit of a dog in my experience.