3 ms·

> We seldom use the new keyword and objects are designed to be pooled and reused. Just note that depending on the selection of the GC, this kind of usage may m

by pron 3y ago

> We seldom use the new keyword and objects are designed to be pooled and reused. Just note that depending on the selection of the GC, this kind of usage may make the GC work more than when allocating new objects, not less. In particular, with the newer GCs -- G1 and ZGC -- mutating existing objects may be more costly than allocating new ones depending on circumstances. In general, the new GCs are optimised to work the least and give the best performance when the allocation rate is neither too high nor too low; the new GCs also reuse memory better than object pools. Reusing objects also precludes scalarization optimisations, i.e. not every `new Foo` actually results in a heap allocation, and can be optimised to work directly in registers. So while on very old JVMs (such as Java 8) a "zero allocation" strategy may result in better performance and in "zero GC", on newer JVMs it may result in worse performance and more GC work (in fact, it will almost surely not yield zero GC). While it depends on many variables, I would advise against a zero allocation strategy on newer JVMs as the default path toward better performance or even better latency. It's an approach that seems to be very strongly coupled to the way the JVM was designed over a decade ago, but a lot has changed since then. Additionally, Java now offers manual memory management and efficient FFI that are significantly better than what JNI offered: https://openjdk.org/jeps/442 https://openjdk.org/jeps/442

coldbrewed 3y ago
> In general, the new GCs are optimised to work the least and give the best performance when the allocation rate is neither too high nor too low; the new GCs also reuse memory better than object pools. This a little bit surprising to me that low object creation can degrade GC performance; what's the failure mode for G1/ZGC in this scenario?
- Groxx 3y ago
  In many cases, because it defeats generational collection. It pushes everything into longer generations because they hang around longer. Doing more young generation collection is sometimes cheaper in aggregate (more frequent but far smaller and usually much more efficient) than adding more data to the older generations (less frequent and more costly, longer pauses, for object pools it happens on all of it even when none of it is currently used, etc). But as doctorpangloss said: so many caveats. There's ample evidence that it is both better and worse, it depends on lots of details. The main thing you can confidently claim is that it is not the majority of code, so most language optimizations will choose to improve straightforward and common stuff at the cost of this niche. Not always, but there is definitely more energy in improving the 90%+ cases and that adds up over time. Squeezing out the last bits of performance requires constant upkeep.
- pron 3y ago
  > what's the failure mode for G1/ZGC in this scenario? GC barriers (special code that gets triggered on some operations by some GCs, such as when mutating a reference field during a GC cycle -- that's a "write barrier"). Concurrent GCs have special rules for newly allocated objects (which are really just a pointer bump) because no one else has seen them yet; young objects require no GC barriers. But once an object is old, a concurrent GC needs to do some work to learn about references in the object changing. So while allocating a new object is usually a pointer bump (and sometimes not even that when the object is scalarized), mutating an old object triggers a GC slow-path that has to mark some data in a shared data structure (so we're talking memory ordering fences) to make sure that the mutated pointer is not overlooked by the GC. OpenJDK's new GCs are really, very, very good. The new generational ZGC in JDK 21 (with sub-millisecond worst case pause) is just amazing. But these GCs are optimised for "reasonable" Java code and against "unreasonable" object pooling. Things are different from where they were a decade ago in Java 8.
doctorpangloss 3y ago
Finance and video games both have really stubborn engineering traditions. > https://openjdk.org/jeps/442 https://openjdk.org/jeps/442 This succinctly is the answer. > not every `new Foo` actually results in a heap allocation, and can be optimised to work directly in registers. While I know in my heart of hearts that you are right, this category of advice has so many caveats. It's like when Graal markets itself as a potential substitute for NodeJs, where its performance is abjectly terrible. I have never personally succeeded in convincing anyone with these deep programming traditions to try something new that may be Better in Every Way. It always has to be this long, trickle down journey of adoption. People still talk about John Carmack and video game engineering, decades after any of his innovations have been verbatim relevant and decades after an individual could possibly have a hope and prayer of authoring a first person shooter with an audience from scratch. But see, I need to know a lot of stuff to understand that.
- latchkey 3y ago
  > I have never personally succeeded in convincing anyone with these deep programming traditions to try something new that may be Better in Every Way. This is true of almost anything really, not just programming.
  - mhss 3y ago
    Agreed. It's culture, and culture is very hard and slow to change.
- vips7L 3y ago
  Yeah scalar replacement is very conservative and things that a human can easily tell don’t escape the jit cannot. This is 2ish years old but has a lot of detail on what escape analysis can and cannot see: https://gist.github.com/JohnTortugo/c2607821202634a6509ec3c321ebf370 https://gist.github.com/JohnTortugo/c2607821202634a6509ec3c3...
cmrdporcupine 3y ago
Yeah, it kinda messes with the "most objects die young" philosophy behind generational collectors. Here's another (related) caveat with taking the "minimal object creation", pooled memory, and/or using lots of non-GC native heap memory path: now you've got blocks of memory sitting there in Process RSS that the GC either can't do anything about, or (worse) knows nothing about. These types of runtimes are on the whole designed with the philosophy that they Own All The Things, so it's like an invasive body driving the "immune system" nuts... Never had this problem in Java (haven't worked in it for 10 years) but at a previous job (in Julia) I worked on a database buffer pool / pager where the memory was explicitly managed/allocated (through syscalls to anonymous mmap) for performance reasons. Those pages belong to the same PID as the broader Julia process, but were not managed by Julia runtime, and so the runtime can run into all sorts of issues with OOM kills or huge pauses as the system allocates aggressively without collecting often, thinking it has plenty of headroom... but doesn't. I know for a fact the JVM GC is smarter than this, and can be tuned more expertly to manage these type of situations, but it's still a big giant caveat...

by pron 3y ago

coldbrewed 3y ago

> In general, the new GCs are optimised to work the least and give the best performance when the allocation rate is neither too high nor too low; the new GCs also reuse memory better than object pools. This a little bit surprising to me that low object creation can degrade GC performance; what's the failure mode for G1/ZGC in this scenario?

Groxx 3y ago
In many cases, because it defeats generational collection. It pushes everything into longer generations because they hang around longer. Doing more young generation collection is sometimes cheaper in aggregate (more frequent but far smaller and usually much more efficient) than adding more data to the older generations (less frequent and more costly, longer pauses, for object pools it happens on all of it even when none of it is currently used, etc). But as doctorpangloss said: so many caveats. There's ample evidence that it is both better and worse, it depends on lots of details. The main thing you can confidently claim is that it is not the majority of code, so most language optimizations will choose to improve straightforward and common stuff at the cost of this niche. Not always, but there is definitely more energy in improving the 90%+ cases and that adds up over time. Squeezing out the last bits of performance requires constant upkeep.
pron 3y ago
> what's the failure mode for G1/ZGC in this scenario? GC barriers (special code that gets triggered on some operations by some GCs, such as when mutating a reference field during a GC cycle -- that's a "write barrier"). Concurrent GCs have special rules for newly allocated objects (which are really just a pointer bump) because no one else has seen them yet; young objects require no GC barriers. But once an object is old, a concurrent GC needs to do some work to learn about references in the object changing. So while allocating a new object is usually a pointer bump (and sometimes not even that when the object is scalarized), mutating an old object triggers a GC slow-path that has to mark some data in a shared data structure (so we're talking memory ordering fences) to make sure that the mutated pointer is not overlooked by the GC. OpenJDK's new GCs are really, very, very good. The new generational ZGC in JDK 21 (with sub-millisecond worst case pause) is just amazing. But these GCs are optimised for "reasonable" Java code and against "unreasonable" object pooling. Things are different from where they were a decade ago in Java 8.

doctorpangloss 3y ago

Finance and video games both have really stubborn engineering traditions. > https://openjdk.org/jeps/442 https://openjdk.org/jeps/442 This succinctly is the answer. > not every `new Foo` actually results in a heap allocation, and can be optimised to work directly in registers. While I know in my heart of hearts that you are right, this category of advice has so many caveats. It's like when Graal markets itself as a potential substitute for NodeJs, where its performance is abjectly terrible. I have never personally succeeded in convincing anyone with these deep programming traditions to try something new that may be Better in Every Way. It always has to be this long, trickle down journey of adoption. People still talk about John Carmack and video game engineering, decades after any of his innovations have been verbatim relevant and decades after an individual could possibly have a hope and prayer of authoring a first person shooter with an audience from scratch. But see, I need to know a lot of stuff to understand that.

latchkey 3y ago
> I have never personally succeeded in convincing anyone with these deep programming traditions to try something new that may be Better in Every Way. This is true of almost anything really, not just programming.
- mhss 3y ago
  Agreed. It's culture, and culture is very hard and slow to change.
vips7L 3y ago
Yeah scalar replacement is very conservative and things that a human can easily tell don’t escape the jit cannot. This is 2ish years old but has a lot of detail on what escape analysis can and cannot see: https://gist.github.com/JohnTortugo/c2607821202634a6509ec3c321ebf370 https://gist.github.com/JohnTortugo/c2607821202634a6509ec3c3...

cmrdporcupine 3y ago

Yeah, it kinda messes with the "most objects die young" philosophy behind generational collectors. Here's another (related) caveat with taking the "minimal object creation", pooled memory, and/or using lots of non-GC native heap memory path: now you've got blocks of memory sitting there in Process RSS that the GC either can't do anything about, or (worse) knows nothing about. These types of runtimes are on the whole designed with the philosophy that they Own All The Things, so it's like an invasive body driving the "immune system" nuts... Never had this problem in Java (haven't worked in it for 10 years) but at a previous job (in Julia) I worked on a database buffer pool / pager where the memory was explicitly managed/allocated (through syscalls to anonymous mmap) for performance reasons. Those pages belong to the same PID as the broader Julia process, but were not managed by Julia runtime, and so the runtime can run into all sorts of issues with OOM kills or huge pauses as the system allocates aggressively without collecting often, thinking it has plenty of headroom... but doesn't. I know for a fact the JVM GC is smarter than this, and can be tuned more expertly to manage these type of situations, but it's still a big giant caveat...