4 ms·

Nice to see ZGC marked as production-ready. With Shenandoah, ZGC, and G1 the options for garbage collection in Java are so much better than they were just a cou

by mjb 6y ago

Nice to see ZGC marked as production-ready. With Shenandoah, ZGC, and G1 the options for garbage collection in Java are so much better than they were just a couple of years ago. Even better, the need to do a bunch of brittle tuning and tweaking has mostly gone away. Java still makes the GC work harder than other languages. In some of my testing, idiomatic "microservice" Java code makes ~10x as much garbage per request as equivalent idiomatic Go code. Some of that is the language, and some is just that libraries and frameworks are super allocation-heavy. If the ecosystem can tighten that up, then these great GCs will have even more impact.

jillesvangurp 6y ago
Depends, if you look at projects like Elasticsearch, you see a shift towards using a lot of off heap memory and less reliance on heap over time. Mostly the JVM and languages on the JVM are evolving towards things like having immutable data structures, async processing and IO, using data classes/structs, etc. A lot of my recent projects have actually been fairly modest in terms of memory requirements compared to a few years ago where I'd have thought nothing of using 4GB heap on a simple spring app.
darksaints 6y ago
Fully agreed about java libraries and frameworks being more allocation-heavy than they need to be. The problem is that the libraries and frameworks are built using the abstraction capabilities it has, and in Java, those abstraction capabilities are very allocation heavy. I'd love to see a jvm or graal implementation of ASAP memory management. The idea generally being having extremely aggressive analysis on your code, reducing heap allocations where possible, statically managing objects where they can provably infer object lifetimes, reference counting everything left over that provably can't result in cycles, and then garbage collecting everything else.
- specialist 6y ago
  "ASAP memory management" Do you mean this? ASAP: As Static As Possible memory management https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf
  - higerordermap 6y ago
    Guess so. That paper is pretty well known.
  - darksaints 6y ago
    Yes. To be clear though, the idea is already hard enough to implement, but it would be incomprehensibly hard in the JVM ecosystem. The JVM directly implements allocation decisions, but the analysis necessary for ASAP is done mostly in higher level reprensentations of code, and there is no direct API in java byte code to tell the JVM to statically allocate or reference count a specific object.
    - thu2111 6y ago
      Hardly incomprehensibly hard. It's been done already, it just never got out of the lab into the product. http://codrutstancu.com/p81-stancu.pdf http://codrutstancu.com/p81-stancu.pdf SVM has support for snapshots/isolates, where you can ~instantly instantiate a new JVM with a fresh heap. If you set your GC params such that it never really collects, you get something close to that but without the complexity.
- MrBuddyCasino 6y ago
  > Fully agreed about java libraries and frameworks being more allocation-heavy than they need to be. Much to the chagrin of Martin Thompson, even the standard library is affected. Avoiding allocations is something most Java programmers don't even think about, because they are not aware it might be an issue and because there is no alternative - in Go and Rust you can stack-allocate instead, in Java you can't (not really).
- nicktelford 6y ago
  GraalVM has Partial Escape Analysis, which improves on HotSpot's Escape Analysis by finding even more allocations that are suitable for Scalar Replacement. It's supposed to be especially well suited for highly functional code e.g. anything that makes heavy use of closures.
- tom_mellior 6y ago
  > reference counting everything left over that provably can't result in cycles, and then garbage collecting everything else This is interesting. I skimmed some introductory parts of https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf as well as the definition of the SCAN function and searched the PDF for "reference count" to see how it does what you propose. At the moment I don't have the impression that ASAP dynamically reference counts anything. Let alone selectively reference count certain objects but use a tracing GC for others. Could you tell me where exactly to look for details? Wouldn't references from RC objects to GC objects, and vice versa, be an absolute pain to deal with?
  - darksaints 6y ago
    Sorry, I seem to have partly confused this with another approach, but can't find the paper at the moment. That being said, it does read like it uses similar approaches to automated reference counting: code is inserted by the compiler to manage reference counts and eventually dispose of objects when no longer referenced. It also appears that there are approaches toward using regions and linear types where possible. I think the takeaway though is that ASAP makes as many decisions as it possibly can to statically enforce memory safety and reclamation, but building a finely grained program-specific approach toward garbage collection for anything else. Literally every distinct program would result in a distinct garbage collector tailored to that program that is inlined into the code.
ragnese 6y ago
I thought the rule of thumb was that only live objects matter for GC performance (mark-and-sweep only counts what's alive and just purges everything else). Is that not true in modern era?
- nicktelford 6y ago
  The more allocations you have, the more frequently your young-gen GC has to run (or the larger your heap size needs to be to keep GC frequency constant). Allocations are cheap, but they aren't free, and you definitely pay a cost in GC.
nicktelford 6y ago
I believe the main reason for this is that Golang is heavily "value-oriented". Combined with escape analysis it causes most of the short-lived objects to actually be allocated directly on the stack. While the JVM also has escape analysis, because everything is a pointer the chances of an object escaping is considerably higher. I'm hoping that when Project Valhalla eventually lands, it'll help greatly with allocations and reduce memory pressure even further.
vbezhenar 6y ago
What is idiomatic in Java? Netty does little-to-no allocations. One can implement very thin layer on top of Netty to provide API similar to Go standard HTTP server.
- tomlu 6y ago
  You have to work extra hard and give up language niceties to reduce allocations, more so than certain other languages that are more value oriented. You want an List<T> of objects? It's now a list of pointers to individual allocations. You want a Path class instead of just passing strings around, for type safety? One extra allocation per object.

by mjb 6y ago

jillesvangurp 6y ago

Depends, if you look at projects like Elasticsearch, you see a shift towards using a lot of off heap memory and less reliance on heap over time. Mostly the JVM and languages on the JVM are evolving towards things like having immutable data structures, async processing and IO, using data classes/structs, etc. A lot of my recent projects have actually been fairly modest in terms of memory requirements compared to a few years ago where I'd have thought nothing of using 4GB heap on a simple spring app.

darksaints 6y ago

Fully agreed about java libraries and frameworks being more allocation-heavy than they need to be. The problem is that the libraries and frameworks are built using the abstraction capabilities it has, and in Java, those abstraction capabilities are very allocation heavy. I'd love to see a jvm or graal implementation of ASAP memory management. The idea generally being having extremely aggressive analysis on your code, reducing heap allocations where possible, statically managing objects where they can provably infer object lifetimes, reference counting everything left over that provably can't result in cycles, and then garbage collecting everything else.

specialist 6y ago
"ASAP memory management" Do you mean this? ASAP: As Static As Possible memory management https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf
- higerordermap 6y ago
  Guess so. That paper is pretty well known.
- darksaints 6y ago
  Yes. To be clear though, the idea is already hard enough to implement, but it would be incomprehensibly hard in the JVM ecosystem. The JVM directly implements allocation decisions, but the analysis necessary for ASAP is done mostly in higher level reprensentations of code, and there is no direct API in java byte code to tell the JVM to statically allocate or reference count a specific object.
  - thu2111 6y ago
    Hardly incomprehensibly hard. It's been done already, it just never got out of the lab into the product. http://codrutstancu.com/p81-stancu.pdf http://codrutstancu.com/p81-stancu.pdf SVM has support for snapshots/isolates, where you can ~instantly instantiate a new JVM with a fresh heap. If you set your GC params such that it never really collects, you get something close to that but without the complexity.
MrBuddyCasino 6y ago
> Fully agreed about java libraries and frameworks being more allocation-heavy than they need to be. Much to the chagrin of Martin Thompson, even the standard library is affected. Avoiding allocations is something most Java programmers don't even think about, because they are not aware it might be an issue and because there is no alternative - in Go and Rust you can stack-allocate instead, in Java you can't (not really).
nicktelford 6y ago
GraalVM has Partial Escape Analysis, which improves on HotSpot's Escape Analysis by finding even more allocations that are suitable for Scalar Replacement. It's supposed to be especially well suited for highly functional code e.g. anything that makes heavy use of closures.
tom_mellior 6y ago
> reference counting everything left over that provably can't result in cycles, and then garbage collecting everything else This is interesting. I skimmed some introductory parts of https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf as well as the definition of the SCAN function and searched the PDF for "reference count" to see how it does what you propose. At the moment I don't have the impression that ASAP dynamically reference counts anything. Let alone selectively reference count certain objects but use a tracing GC for others. Could you tell me where exactly to look for details? Wouldn't references from RC objects to GC objects, and vice versa, be an absolute pain to deal with?
- darksaints 6y ago
  Sorry, I seem to have partly confused this with another approach, but can't find the paper at the moment. That being said, it does read like it uses similar approaches to automated reference counting: code is inserted by the compiler to manage reference counts and eventually dispose of objects when no longer referenced. It also appears that there are approaches toward using regions and linear types where possible. I think the takeaway though is that ASAP makes as many decisions as it possibly can to statically enforce memory safety and reclamation, but building a finely grained program-specific approach toward garbage collection for anything else. Literally every distinct program would result in a distinct garbage collector tailored to that program that is inlined into the code.

ragnese 6y ago

I thought the rule of thumb was that only live objects matter for GC performance (mark-and-sweep only counts what's alive and just purges everything else). Is that not true in modern era?

nicktelford 6y ago
The more allocations you have, the more frequently your young-gen GC has to run (or the larger your heap size needs to be to keep GC frequency constant). Allocations are cheap, but they aren't free, and you definitely pay a cost in GC.

nicktelford 6y ago

I believe the main reason for this is that Golang is heavily "value-oriented". Combined with escape analysis it causes most of the short-lived objects to actually be allocated directly on the stack. While the JVM also has escape analysis, because everything is a pointer the chances of an object escaping is considerably higher. I'm hoping that when Project Valhalla eventually lands, it'll help greatly with allocations and reduce memory pressure even further.

vbezhenar 6y ago

What is idiomatic in Java? Netty does little-to-no allocations. One can implement very thin layer on top of Netty to provide API similar to Go standard HTTP server.

tomlu 6y ago
You have to work extra hard and give up language niceties to reduce allocations, more so than certain other languages that are more value oriented. You want an List<T> of objects? It's now a list of pointers to individual allocations. You want a Path class instead of just passing strings around, for type safety? One extra allocation per object.