6 ms·

I wonder how much the perceived advantages of generational GC are only due to Java's tendency to generate a lot of short-lived garbage.

by cesarb 2y ago

mananaysiempre 2y ago
“A lot” is relative—I seem to remember a discussion long ago mentioning that Haskell/Scala-style programs were a problem for the JVM because the JVM is not designed for so many objects dying so quickly.
- kgeist 2y ago
  I remember long time ago JBullet, a physics simulation library, had to maintain an error-prone pool of vectors and matrices because all the calculations involving the creation of temporary vectors and matrices put significant stress on the GC. Today, Minecraft allocates around 600 MB/s for a similar reason (Position3D objects & friends); however, on my PC, I don't notice any pauses.
  - gf000 2y ago
    Is that really error-prone for a physics simulation engine? That's pretty common to only refer to data by indices in other places - sure, there are languages that can deal with it a bit better (e.g. operator overloading, manual memory management and/or value types), but as far as I know this is even often done in Rust, which has the previous properties (it's done to avoid the borrow checker there). Also, Minecraft was famously written famously inefficiently.
    - kgeist 2y ago
      By "error-prone," I mean that if you accidentally stored a reference to a pooled vector and used it after it was returned to the pool, all sorts of unexpected things could happen since everything is by reference. You had to be very careful.
  - hinkley 2y ago
    I had a friend/former mentor who proved that object pools are slower in Java once GenGC was introduced because they create pointers from the old generation to the new. He also authored the fasted XML parser of that era and previously helped me land a project everyone but us and our boss said was impossible because Java Is Too Slow, so I take him at his word. So unless those pools are full of primitive data types, you’re going to cause more frequent full GC pauses, and longer due to all the back pointers.
    - fiddlerwoaroof 2y ago
      If you could add some sort of hint to the object pool instance that it should always be in the new generation, then you wouldn’t have this problem. This could probably be provided generically as a class named something like NurseryCollection. However, I thought the people that really were serious about performance would allocate the pool in memory the GC doesn’t control (back in the day using internal APIs, but I think there are new official APIs for this) such that the object pool and every object in it would just be ignored by the GC.
      - hinkley 2y ago
        Looks like an arena allocator is available in JDK 20. But I’m fairly sure it’s been in the RT spec for a lot longer.
    - ickyforce 2y ago
      > So unless those pools are full of primitive data types, you’re going to cause more frequent full GC pauses, and longer due to all the back pointers. If you have enough objects pooled to get low allocation rate then you never trigger full GC. That's what "low latency Java" aims for. I think that the main downside of object pools in Java is that it's very easy to accidentally forget to return an object to a pool or return it twice. It turns out that it's hard to keep track of what "owns" the object and should return it if the code is not a simple try{}finally{}. This was a significant source of bugs at my previous job (algo trading).
      - 2y ago
- gf000 2y ago
  The JVM is more than happy with short-lived objects, they are barely more expensive than a normal stack allocation (mostly by the header only). The JVM has a so-called Thread-Local Allocation Buffer, which is basically a simple buffer with a pointer pointing to the next free slot. It can be simply bumped up on a new object's allocation and later on the still alive objects will be evacuated to another generation and the whole thing can be cleared. This is faster than any malloc implementation CPP/rust/whatever might use, besides they also doing arena allocations. Haskell has a few tricks up its sleeve that can help with its kind of allocation patterns, but speaking specifically about Haskell, it's mostly the laziness that makes it quite different.
  - lowbloodsugar 2y ago
    But it’s not as fast as just allocating a structure on the stack instead of checks graph 20 objects on the heap, all pointing to the next with 64bit pointers. I’ve got Rust struct that are 80 bytes and the same structure in Java is 800 bytes.
    - gf000 2y ago
      What's the difference between the stack and a thread-local, constantly hot/in-cache other region of the same memory? Especially that in many cases the objects are allocated right next to each other, so fetching happens from cache again. Not saying that expertly written Rust code can't beat naive Java one, and especially with the object headers Java will have a performance disadvantage in many cases - but that's a tiny disadvantage in most real-world use cases, multiple times offset by its advantages. (Besides, Java has on-stack replacement which can do the exact same thing as rust does, but the JIT-compiler Gods have to be in the correct alignment for that)
      - lowbloodsugar 2y ago
        What’s the difference…? 1. The size. 2. I don’t see that happening and I look at the output of the C2 compiler when I’m dealing with hotspots. In aware of the cool shit the JVM can do but done right, areas of code can be as fast as C or Rust. It in general, My experience is that Rust is much faster. Sometimes 10x faster where it matters to me (sustained server load).
pfdietz 2y ago
Generational GC was perceived as useful even before Java existed, for example in Lisp.
- actuallyalys 2y ago
  Funnily enough, my impression is that Clojure, the most popular Lisp on the JVM, creates a lot of garbage—probably more than Java—but I believe that’s due to it using immutable collections and not due to it being a Lisp.
  - fiddlerwoaroof 2y ago
    Yeah, immutable collections have to use structure-sharing to be implemented efficiently and this frequently results in common operations allocating lots of small objects rather than updating some object in-place.
    - zozbot234 2y ago
      You can use a borrow checker to replace immutable objects with update-in-place ("ephemeral") ones where feasible.
      - fiddlerwoaroof 2y ago
        And, escape analysis and "dynamic extent" hints can have similar effects by stack-allocating intermediate values. Also, you can design your transformations to compose before handing off to a result builder, which is how modern Clojure reduces this problem.
davidgay 2y ago
You've clearly never observed the allocation behaviour of a continuation-based compiler (every function call causes a short-lived heap allocation - the heap is the "stack")...
- wbl 2y ago
  And the stack the heap with Cheney on the MTA.
steveklabnik 2y ago
I have long suspected that there is some sort of deeper truth here. It’s not really about Java. If you squint hard enough, a C program has the stack as its nursery, and the heap as a singular old generation. The stack is even arena allocated, of sorts! This is of course a bit hand-wavy, I haven’t had the time to truly try and investigate this in a more rigorous way.
- jerf 2y ago
  In general, memory management needs to be understood as a continuum and not a binary "uses GC or uses manual memory management". Per the very discussion we're having, GC is not an atomic thing, and on the other end, what is commonly called "manual memory management" is itself not. malloc still makes non-trivial decisions, as evidenced by the fact that some programs can be greatly benefited or harmed by switching out the malloc it uses. Beyond what most people call "manual memory management" lies "arena allocation", and in the really extreme case, fully static memory layout with no allocation at all. The latter is how NES games were generally written and sufficiently small embedded projects still get written that way today. Further pushing this into a "continuum" rather than a binary flag is that programs can and do freely mix and match. Nothing stops a program from using two mallocs, some arenas, GC for certain values, and static allocations for yet others. The Memory Management Police do not come along and arrest you for having Impure Designs or anything. There are languages like (I believe) Zig where the ability to cleanly do this is more-or-less a first-class language feature.

I wonder how much the perceived advantages of generational GC are only due to Java's tendency to generate a lot of short-lived garbage.

by cesarb 2y ago

I wonder how much the perceived advantages of generational GC are only due to Java's tendency to generate a lot of short-lived garbage.

mananaysiempre 2y ago

“A lot” is relative—I seem to remember a discussion long ago mentioning that Haskell/Scala-style programs were a problem for the JVM because the JVM is not designed for so many objects dying so quickly.

kgeist 2y ago
I remember long time ago JBullet, a physics simulation library, had to maintain an error-prone pool of vectors and matrices because all the calculations involving the creation of temporary vectors and matrices put significant stress on the GC. Today, Minecraft allocates around 600 MB/s for a similar reason (Position3D objects & friends); however, on my PC, I don't notice any pauses.
- gf000 2y ago
  Is that really error-prone for a physics simulation engine? That's pretty common to only refer to data by indices in other places - sure, there are languages that can deal with it a bit better (e.g. operator overloading, manual memory management and/or value types), but as far as I know this is even often done in Rust, which has the previous properties (it's done to avoid the borrow checker there). Also, Minecraft was famously written famously inefficiently.
  - kgeist 2y ago
    By "error-prone," I mean that if you accidentally stored a reference to a pooled vector and used it after it was returned to the pool, all sorts of unexpected things could happen since everything is by reference. You had to be very careful.
- hinkley 2y ago
  I had a friend/former mentor who proved that object pools are slower in Java once GenGC was introduced because they create pointers from the old generation to the new. He also authored the fasted XML parser of that era and previously helped me land a project everyone but us and our boss said was impossible because Java Is Too Slow, so I take him at his word. So unless those pools are full of primitive data types, you’re going to cause more frequent full GC pauses, and longer due to all the back pointers.
  - fiddlerwoaroof 2y ago
    If you could add some sort of hint to the object pool instance that it should always be in the new generation, then you wouldn’t have this problem. This could probably be provided generically as a class named something like NurseryCollection. However, I thought the people that really were serious about performance would allocate the pool in memory the GC doesn’t control (back in the day using internal APIs, but I think there are new official APIs for this) such that the object pool and every object in it would just be ignored by the GC.
    - hinkley 2y ago
      Looks like an arena allocator is available in JDK 20. But I’m fairly sure it’s been in the RT spec for a lot longer.
  - ickyforce 2y ago
    > So unless those pools are full of primitive data types, you’re going to cause more frequent full GC pauses, and longer due to all the back pointers. If you have enough objects pooled to get low allocation rate then you never trigger full GC. That's what "low latency Java" aims for. I think that the main downside of object pools in Java is that it's very easy to accidentally forget to return an object to a pool or return it twice. It turns out that it's hard to keep track of what "owns" the object and should return it if the code is not a simple try{}finally{}. This was a significant source of bugs at my previous job (algo trading).
    - 2y ago
gf000 2y ago
The JVM is more than happy with short-lived objects, they are barely more expensive than a normal stack allocation (mostly by the header only). The JVM has a so-called Thread-Local Allocation Buffer, which is basically a simple buffer with a pointer pointing to the next free slot. It can be simply bumped up on a new object's allocation and later on the still alive objects will be evacuated to another generation and the whole thing can be cleared. This is faster than any malloc implementation CPP/rust/whatever might use, besides they also doing arena allocations. Haskell has a few tricks up its sleeve that can help with its kind of allocation patterns, but speaking specifically about Haskell, it's mostly the laziness that makes it quite different.
- lowbloodsugar 2y ago
  But it’s not as fast as just allocating a structure on the stack instead of checks graph 20 objects on the heap, all pointing to the next with 64bit pointers. I’ve got Rust struct that are 80 bytes and the same structure in Java is 800 bytes.
  - gf000 2y ago
    What's the difference between the stack and a thread-local, constantly hot/in-cache other region of the same memory? Especially that in many cases the objects are allocated right next to each other, so fetching happens from cache again. Not saying that expertly written Rust code can't beat naive Java one, and especially with the object headers Java will have a performance disadvantage in many cases - but that's a tiny disadvantage in most real-world use cases, multiple times offset by its advantages. (Besides, Java has on-stack replacement which can do the exact same thing as rust does, but the JIT-compiler Gods have to be in the correct alignment for that)
    - lowbloodsugar 2y ago
      What’s the difference…? 1. The size. 2. I don’t see that happening and I look at the output of the C2 compiler when I’m dealing with hotspots. In aware of the cool shit the JVM can do but done right, areas of code can be as fast as C or Rust. It in general, My experience is that Rust is much faster. Sometimes 10x faster where it matters to me (sustained server load).

pfdietz 2y ago

Generational GC was perceived as useful even before Java existed, for example in Lisp.

actuallyalys 2y ago
Funnily enough, my impression is that Clojure, the most popular Lisp on the JVM, creates a lot of garbage—probably more than Java—but I believe that’s due to it using immutable collections and not due to it being a Lisp.
- fiddlerwoaroof 2y ago
  Yeah, immutable collections have to use structure-sharing to be implemented efficiently and this frequently results in common operations allocating lots of small objects rather than updating some object in-place.
  - zozbot234 2y ago
    You can use a borrow checker to replace immutable objects with update-in-place ("ephemeral") ones where feasible.
    - fiddlerwoaroof 2y ago
      And, escape analysis and "dynamic extent" hints can have similar effects by stack-allocating intermediate values. Also, you can design your transformations to compose before handing off to a result builder, which is how modern Clojure reduces this problem.

davidgay 2y ago

You've clearly never observed the allocation behaviour of a continuation-based compiler (every function call causes a short-lived heap allocation - the heap is the "stack")...

wbl 2y ago
And the stack the heap with Cheney on the MTA.

steveklabnik 2y ago

I have long suspected that there is some sort of deeper truth here. It’s not really about Java. If you squint hard enough, a C program has the stack as its nursery, and the heap as a singular old generation. The stack is even arena allocated, of sorts! This is of course a bit hand-wavy, I haven’t had the time to truly try and investigate this in a more rigorous way.

jerf 2y ago
In general, memory management needs to be understood as a continuum and not a binary "uses GC or uses manual memory management". Per the very discussion we're having, GC is not an atomic thing, and on the other end, what is commonly called "manual memory management" is itself not. malloc still makes non-trivial decisions, as evidenced by the fact that some programs can be greatly benefited or harmed by switching out the malloc it uses. Beyond what most people call "manual memory management" lies "arena allocation", and in the really extreme case, fully static memory layout with no allocation at all. The latter is how NES games were generally written and sufficiently small embedded projects still get written that way today. Further pushing this into a "continuum" rather than a binary flag is that programs can and do freely mix and match. Nothing stops a program from using two mallocs, some arenas, GC for certain values, and static allocations for yet others. The Memory Management Police do not come along and arrest you for having Impure Designs or anything. There are languages like (I believe) Zig where the ability to cleanly do this is more-or-less a first-class language feature.