4 ms·

Serial GC Changes

68 points by 0x54MUR41 2mo ago

kasperni 2mo ago
Are people still dealing with GC issues? I find that it basically just more or less works out of the box on modern JVMs.
- hylaride 2mo ago
  There are edge cases where GC issues can crop up, in particular specific "serverless" models (eg AWS lambdas) where the JVM can get "paused" between executions and GC doesn't cleanly run, causing memory to trend upwards until the next cold-start happens (especially if you're running it within a docker container yourself). Limited CPU situations that can exist in these kinds of runtime environments also limit GC in several ways, too.
  - skullone 2mo ago
    What's it like running JVM inside serverless? Python gets interesting enough when it pauses waiting for another call sometimes, the JVM seems like it'd introduce its own interesting things :o
    - hylaride 2mo ago
      Other than some GC issues, meaning we had to give it more memory than we'd otherwise like thereby making them a bit more expensive, it's mostly been fine. The key to ephemeral java is keeping scope limited and not using it when cold starts matter (which can cause a bit of jitter in execution lag). Our use case was asynchronous, so it worked. We wanted to investigate using the native AWS java lambdas with snapstart and/or using parallel GC, but the place I used them at a year ago was under the tyranny of product having complete control of our backlog, so we didn't get to go that far with it or even more serious tech debt for that matter. More memory may or may not even be much of an issue depending on your workload. If you can handle the odd OOM error it would have been fine, but for our use case lambdas were so cheap that it didn't really matter, outside of us techies preferring to do it "right". Having the max execution time be less than 15m could also minimize the heap bloat at the cost of more cold starts.
- kelseyfrog 2mo ago
  It's selection bias. There's a huge number of GC language users, but those who experience problems tend to be the ones who comment. You're right; the vast majority of people use GC just fine and go about their day. We should update little to none when we see evidence of GC hardship.
  - stmw 2mo ago
    It depends a lot on the scale of the system, which often means that those fewer users are actually doing things that are more complex and more valuable - either at the lower end or at the high end.
  - 2001zhaozhao 2mo ago
    I sometimes run into browser JavaScript GC issues for browser games specifically but on the JVM side i have not run into any pain points for a long time. (I run a first-person shooter Minecraft server, and for this and other fast-paced gaming in general pause times under a millisecond or so is generally good.)
- thangalin 2mo ago
  > Are people still dealing with GC issues? Have you tried real-time audio processing for digital radio communications on a JVM that requires sub-millisecond latency on older, temperature-hardened CPUs?
  - pron 2mo ago
    Use ZGC.
    - hedora 2mo ago
      Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware? I moved to c++/rust years ago because those languages do, and tens of milliseconds matter for network services. At the time Java could pause for 10’s of seconds, which was 1000x worse than waiting for a spinning disk to seek. These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max. This article is from last year, but suggests they’re still off by an order of magnitude: https://www.morling.dev/blog/lower-java-tail-latencies-with-zgc/ https://www.morling.dev/blog/lower-java-tail-latencies-with-... Also, that’s measuring a 30 second window. If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
      - pron 2mo ago
        > Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware? Yes and yes (although we need to be more precise when we talk about latencies; see next paragraph). > These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max. 1. You don't need to tune it. The algorithm simply doesn't collect garbage in stop-the-world pauses. 2. Hiccups are sporadic. They should not be compared to the average latency of normal operation. The relevant question is, is ZGC introducing longer hiccups than those a non-realtime kernel would, and the answer is no. > This article is from last year, but suggests they’re still off by an order of magnitude The article doesn't measure GC pauses when it shows latencies (it says: "With ZGC on the other hand, the longest GC pause time observed is ~50 microseconds"). It measures the response latencies of some service. Note that allocation stalls also occur with malloc, it just isn't reported conveniently. Of course, one of the greatest advantages of moving collectors still applies: Under high allocation rates, moving collectors (but not malloc/free!) allow you to compensate for increased CPU spent on memory management by increasing the heap (i.e. if your allocation rate doubles, you can increase the heap and keep the CPU cost of memory management the same). In the past, this advantage translated to higher throughputs compared to malloc/free, but suffered from GC pauses. Those pauses are gone today. > If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load? No, it does not. You could, of course, construct some pathological cases where you'd have a high allocation rate for long-lived objects which would result in high CPU utilisation by the GC, but it's easier to get into pathological malloc/free cases in C++ (or Rust) than with ZGC. Let me put it another way: no matter your memory management strategy, it's possible to overwhelm it, but the likelihood that a real, "naive" program would overwhelm a malloc/free allocator is higher than it would the JDK's GCs.
- geodel 2mo ago
  Perhaps. That's why actual JVM GC developers are talking about it.
smallnix 2mo ago
> selecting something else depending on arcane environmental conditions was more of a burden than an advantage What is being referred to here? Does "else" refer to ZGC, Shenandoah? What does arcane env conditions mean?
- aatos 2mo ago
  It's in the JEP (https://openjdk.org/jeps/523 https://openjdk.org/jeps/523): > We made G1 the default collector for server environments in JDK 9 (JEP 248). At that time, testing showed that Serial had significant advantages in throughput and footprint in constrained environments with a single CPU or less than 1792 MB of physical memory. We therefore adjusted the JVM's GC selection algorithm to choose Serial in such environments.
stmw 2mo ago
GC's haven't freed us from manual memory management, you just do all that manual work with environment variables, or making sure to "pick the right collector for the job", or debugging performance or heap size issues, or chasing down weak references or confused finalizers.
- pron 2mo ago
  > you just do all that manual work with environment variables You really don't anymore. For the past several years, Java's GCs mostly pick the right settings automatically, except for heap size, which will be taken care of soon (https://openjdk.org/jeps/8377305 https://openjdk.org/jeps/8377305). The reason heap size isn't automatic is that with moving collectors it determines the CPU/RAM tradeoff, and doing that in a more natural way isn't trivial, but we have the algorithm now and will merge it soon. > or making sure to "pick the right collector for the job" There are really only four options, most of which are easy to choose among: Parallel for batch jobs where only throughput matters, ZGC for interactive applications where latency matters a lot, and then consider either G1 or Serial if there's a problem with those choices. As someone who's worked for a long, long time solving manual memory management issues, the amount of effort required isn't just in a different ballpark, but in a different city. Sure, spending a few hours a year to reconsider your settings isn't nothing, but it isn't even remotely in the same category of pain with manual memory management (or even automatic memory management, but with malloc/free underneath).
  - hedora 2mo ago
    Concretely, what are current tail latencies, worst case? Ten years ago, “rewrite in C++” was definitely easier than getting the Java GC to stay up under server load. Most servers I work with run on big machines and are the only process, so figure a 100-250GB heap that lives for months, all async, small requests, so insane amounts of Future and String allocation spam. Optimizing that stuff away in Java is harder than writing Rust, so assume idiomatic Java. Also, is there any work on statically enforcing data race freedom in Java? That’s a bigger rust selling point than memory safety for me. I think swift has done some interesting work in that space. It would be nice to get those sorts of safety properties without manually writing borrow checker annotations.
Traubenfuchs 2mo ago
If any JDK wizard is present: In my opinion, the thing the JRE is REALLY missing is a single process level memory limit setting. Nowadays you still have to consider the off heap memory when limiting a JRE processes max memory. To clarify: -XX:MaxRAMPercentage should not exist. Instead the process should be told: „You can use x mb/gb of memory. Use of that what you need for offheap and use the rest for heap.“ That setting is embarassing. Figuring out its value is a mixture of voodoo, vibe-driven guessing and playing the game of „how much wasted memory do you want to risk to prevent a crash?“. Give us -XX:MaxProcessMemory=4g please. Why doesn‘t this exist?
exabrial 2mo ago
Huge thanks to the OpenJDK Team! I still can't believe we are finding incredible improvements. The JVM is an engineering marvel

68 points by 0x54MUR41 2mo ago

Are people still dealing with GC issues? I find that it basically just more or less works out of the box on modern JVMs.

hylaride 2mo ago
There are edge cases where GC issues can crop up, in particular specific "serverless" models (eg AWS lambdas) where the JVM can get "paused" between executions and GC doesn't cleanly run, causing memory to trend upwards until the next cold-start happens (especially if you're running it within a docker container yourself). Limited CPU situations that can exist in these kinds of runtime environments also limit GC in several ways, too.
- skullone 2mo ago
  What's it like running JVM inside serverless? Python gets interesting enough when it pauses waiting for another call sometimes, the JVM seems like it'd introduce its own interesting things :o
  - hylaride 2mo ago
    Other than some GC issues, meaning we had to give it more memory than we'd otherwise like thereby making them a bit more expensive, it's mostly been fine. The key to ephemeral java is keeping scope limited and not using it when cold starts matter (which can cause a bit of jitter in execution lag). Our use case was asynchronous, so it worked. We wanted to investigate using the native AWS java lambdas with snapstart and/or using parallel GC, but the place I used them at a year ago was under the tyranny of product having complete control of our backlog, so we didn't get to go that far with it or even more serious tech debt for that matter. More memory may or may not even be much of an issue depending on your workload. If you can handle the odd OOM error it would have been fine, but for our use case lambdas were so cheap that it didn't really matter, outside of us techies preferring to do it "right". Having the max execution time be less than 15m could also minimize the heap bloat at the cost of more cold starts.
kelseyfrog 2mo ago
It's selection bias. There's a huge number of GC language users, but those who experience problems tend to be the ones who comment. You're right; the vast majority of people use GC just fine and go about their day. We should update little to none when we see evidence of GC hardship.
- stmw 2mo ago
  It depends a lot on the scale of the system, which often means that those fewer users are actually doing things that are more complex and more valuable - either at the lower end or at the high end.
- 2001zhaozhao 2mo ago
  I sometimes run into browser JavaScript GC issues for browser games specifically but on the JVM side i have not run into any pain points for a long time. (I run a first-person shooter Minecraft server, and for this and other fast-paced gaming in general pause times under a millisecond or so is generally good.)
thangalin 2mo ago
> Are people still dealing with GC issues? Have you tried real-time audio processing for digital radio communications on a JVM that requires sub-millisecond latency on older, temperature-hardened CPUs?
- pron 2mo ago
  Use ZGC.
  - hedora 2mo ago
    Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware? I moved to c++/rust years ago because those languages do, and tens of milliseconds matter for network services. At the time Java could pause for 10’s of seconds, which was 1000x worse than waiting for a spinning disk to seek. These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max. This article is from last year, but suggests they’re still off by an order of magnitude: https://www.morling.dev/blog/lower-java-tail-latencies-with-zgc/ https://www.morling.dev/blog/lower-java-tail-latencies-with-... Also, that’s measuring a 30 second window. If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
    - pron 2mo ago
      > Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware? Yes and yes (although we need to be more precise when we talk about latencies; see next paragraph). > These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max. 1. You don't need to tune it. The algorithm simply doesn't collect garbage in stop-the-world pauses. 2. Hiccups are sporadic. They should not be compared to the average latency of normal operation. The relevant question is, is ZGC introducing longer hiccups than those a non-realtime kernel would, and the answer is no. > This article is from last year, but suggests they’re still off by an order of magnitude The article doesn't measure GC pauses when it shows latencies (it says: "With ZGC on the other hand, the longest GC pause time observed is ~50 microseconds"). It measures the response latencies of some service. Note that allocation stalls also occur with malloc, it just isn't reported conveniently. Of course, one of the greatest advantages of moving collectors still applies: Under high allocation rates, moving collectors (but not malloc/free!) allow you to compensate for increased CPU spent on memory management by increasing the heap (i.e. if your allocation rate doubles, you can increase the heap and keep the CPU cost of memory management the same). In the past, this advantage translated to higher throughputs compared to malloc/free, but suffered from GC pauses. Those pauses are gone today. > If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load? No, it does not. You could, of course, construct some pathological cases where you'd have a high allocation rate for long-lived objects which would result in high CPU utilisation by the GC, but it's easier to get into pathological malloc/free cases in C++ (or Rust) than with ZGC. Let me put it another way: no matter your memory management strategy, it's possible to overwhelm it, but the likelihood that a real, "naive" program would overwhelm a malloc/free allocator is higher than it would the JDK's GCs.
geodel 2mo ago
Perhaps. That's why actual JVM GC developers are talking about it.

smallnix 2mo ago

> selecting something else depending on arcane environmental conditions was more of a burden than an advantage What is being referred to here? Does "else" refer to ZGC, Shenandoah? What does arcane env conditions mean?

aatos 2mo ago
It's in the JEP (https://openjdk.org/jeps/523 https://openjdk.org/jeps/523): > We made G1 the default collector for server environments in JDK 9 (JEP 248). At that time, testing showed that Serial had significant advantages in throughput and footprint in constrained environments with a single CPU or less than 1792 MB of physical memory. We therefore adjusted the JVM's GC selection algorithm to choose Serial in such environments.

stmw 2mo ago

GC's haven't freed us from manual memory management, you just do all that manual work with environment variables, or making sure to "pick the right collector for the job", or debugging performance or heap size issues, or chasing down weak references or confused finalizers.

pron 2mo ago
> you just do all that manual work with environment variables You really don't anymore. For the past several years, Java's GCs mostly pick the right settings automatically, except for heap size, which will be taken care of soon (https://openjdk.org/jeps/8377305 https://openjdk.org/jeps/8377305). The reason heap size isn't automatic is that with moving collectors it determines the CPU/RAM tradeoff, and doing that in a more natural way isn't trivial, but we have the algorithm now and will merge it soon. > or making sure to "pick the right collector for the job" There are really only four options, most of which are easy to choose among: Parallel for batch jobs where only throughput matters, ZGC for interactive applications where latency matters a lot, and then consider either G1 or Serial if there's a problem with those choices. As someone who's worked for a long, long time solving manual memory management issues, the amount of effort required isn't just in a different ballpark, but in a different city. Sure, spending a few hours a year to reconsider your settings isn't nothing, but it isn't even remotely in the same category of pain with manual memory management (or even automatic memory management, but with malloc/free underneath).
- hedora 2mo ago
  Concretely, what are current tail latencies, worst case? Ten years ago, “rewrite in C++” was definitely easier than getting the Java GC to stay up under server load. Most servers I work with run on big machines and are the only process, so figure a 100-250GB heap that lives for months, all async, small requests, so insane amounts of Future and String allocation spam. Optimizing that stuff away in Java is harder than writing Rust, so assume idiomatic Java. Also, is there any work on statically enforcing data race freedom in Java? That’s a bigger rust selling point than memory safety for me. I think swift has done some interesting work in that space. It would be nice to get those sorts of safety properties without manually writing borrow checker annotations.

Traubenfuchs 2mo ago

If any JDK wizard is present: In my opinion, the thing the JRE is REALLY missing is a single process level memory limit setting. Nowadays you still have to consider the off heap memory when limiting a JRE processes max memory. To clarify: -XX:MaxRAMPercentage should not exist. Instead the process should be told: „You can use x mb/gb of memory. Use of that what you need for offheap and use the rest for heap.“ That setting is embarassing. Figuring out its value is a mixture of voodoo, vibe-driven guessing and playing the game of „how much wasted memory do you want to risk to prevent a crash?“. Give us -XX:MaxProcessMemory=4g please. Why doesn‘t this exist?

exabrial 2mo ago

Huge thanks to the OpenJDK Team! I still can't believe we are finding incredible improvements. The JVM is an engineering marvel