6 ms·

>Be ever so slightly wary when someone sells you the non-moving GC or no-GC solutions And you should also be wary whenever someone uses the JVM to make a gene

>Be ever so slightly wary when someone sells you the non-moving GC or no-GC solutions And you should also be wary whenever someone uses the JVM to make a general point about GC strategies. The importance of a moving/compacting GC varies hugely depending on whether or not a language/runtime lets developers define the memory layout of objects. In a language like Java (and other JVM based languages) that doesn't let you do that relying on the GC is the only way to fix locality. But that is not the case in languages that have structured value types (C#, Go, Swift, ...).

pizlonator 9y ago
Totally! Java is in a class of languages that encourages a high allocation rate. I think that each language has a kind of "average" allocation rate that is an indirect outcome of the various design decisions in the language. I think you're right that C# probably has a lower average allocation rate because of structs. It's true that languages with lower average allocation rates benefit less from compaction, because they benefit less from any GC optimization.
geodel 9y ago
You might have checked but here is IanT's reply on "Why golang garbage-collector not implement Generational and Compact gc? There is also interesting discussion in thread. https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBUH1mHCAAJ https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBU... "However, modern memory allocation algorithms, like the tcmalloc-based approach used by the Go runtime, have essentially no fragmentation issues. And while a bump allocator can be simple and efficient for a single-threaded program, in a multi-threaded program like Go it requires locks. In general it's likely to be more efficient to allocate memory using a set of per-thread caches, and at that point you've lost the advantages of a bump allocator. So I would assert that, in general, with many caveats, there is no real advantage .."
- pizlonator 9y ago
  Bump allocators don't require locks. Bump allocators are especially effective in the presence of thread-local caches. Whoever wrote that must not be very familiar with how memory management works.
- pcwalton 9y ago
  > And while a bump allocator can be simple and efficient for a single-threaded program, in a multi-threaded program like Go it requires locks. In general it's likely to be more efficient to allocate memory using a set of per-thread caches, and at that point you've lost the advantages of a bump allocator. Ian is extremely confused here. Every production multithreaded generational GC uses thread-local allocation buffers (edit: thanks to Filip for correcting my terminology).
  - pizlonator 9y ago
    Very few production multithreaded generational GCs use per-thread nurseries. But probably all of them use thread-local caches, or what you might call per-thread caches. I think that's what you might have meant. Per-thread nurseries is a shit idea unless your language has extra-super-high levels of infant mortality. Remember that in a per-thread nursery genGC, the write barrier has to infect the pointed-to object with the global heap so that future stores to that object also trigger the barrier. In a normal genGC, the pointed-to object is unaffected; the stored-to object is merely shaded (or card-marked, or logged, or whatever) for revisiting. Humorously for parent's post, to my knowledge the first instance of a thread-local cache was the idea that a multithreaded compacting GC can chop off a slice of memory from the global bump allocator and then use it to service a thread-local allocator. Not 100% sure though because maybe Boehm and friends beat everyone to it.
    - pcwalton 9y ago
      > Very few production multithreaded generational GCs use per-thread nurseries. > Per-thread nurseries is a shit idea unless your language has extra-super-high levels of infant mortality. Isn't that what TLABs are? HotSpot uses TLABs by default, unless my info is out of date… For example, [1] shows the generated assembler for object allocation, which uses nonatomic pointer bumps. I think we might be talking about different things by "per-thread nurseries". All I mean by a "per-thread nursery" is a TLAB. [1]: https://shipilev.net/jvm-anatomy-park/4-tlab-allocation/ https://shipilev.net/jvm-anatomy-park/4-tlab-allocation/
      - 9y ago
- jerven 9y ago
  I think that shipilev wrote this blog post after seeing that post. He and Gil Tene, of Azul Zing/C4 pauseless GC fame, don't seem to be impressed by the GO team GC claims. Mostly because the GO gc does fragment, which can lead to crashes when it shouldn't. The GO team seems to believe escape analysis means no need for generational hypothesis, while Gil and Shiplev think that escape analysis means less allocation (which does help collection, but not enough). Shiplev, these days works with the RedHat Shenadoah GC team. Which is low pause time (in the order of the Go Team promise) while compacting and fully parallel. So he knows a thing or two about modern GCs and JIT and compiler tech ;) Not to say that the Go team doesn't, it just that they might assume somethings which are not true for some apps.
  - geodel 9y ago
    Don't know about him but Gil seems happy by Go's choices GC area. Here are Gil's view: https://groups.google.com/forum/#!searchin/mechanical-sympathy/Go$20gc%7Csort:relevance/mechanical-sympathy/JJTUs_jXw5A/55cRmOUoCwAJ https://groups.google.com/forum/#!searchin/mechanical-sympat... About Go: "Go's GC story is evolving. And it's doing so at a fairly quick pace. Judging it based on a current implementation and current choices is premature, I think. I think that the Go team's implementation priority focus on latency is a good one. And unlike others, I don't actually think there is a strong latency vs. throughput tradeoff in the long run." And Kind words about Hotspot: "The overall approach with HotSpot can be thought of as "start with a STW generational collector and increment your way to lower average/typical pauses in the old generations". E.g. all practical collectors in HotSpot use a monolithic stop-the-world copying newgen collector (not concurrent or incremental, monolithic, run-to-completion STW, always). And all collectors in HotSpot have a fallback to a full STW oldgen pause. The various non-monolithic-STW "cool modes" (Partially Concurrent non-compacting Oldgen in CMS, Incremental STW compaction in G1) are filters in the oldgen that occur between full STW newgen pauses and full STW oldgen pauses. They are "happy" when the filters work well enough to keep the STW newgen pauses in the tens of msec and delay the full-STW pauses for hours or days. And they consider it to be "ok and still working" even when those levels are not met." You can check Shenandoah pause times (20-45ms) : https://www.slideshare.net/RedHatDevelopers/shenandoah-gc-java-without-the-garbage-collection-hiccups-christine-flood https://www.slideshare.net/RedHatDevelopers/shenandoah-gc-ja... Decent for Java world but not really competing with Go. To add they seem very critical of Generational GC: "Generational GC pay a steep penalty for copying Data."
pcwalton 9y ago
> The importance of a moving/compacting GC varies hugely depending on whether or not a language/runtime lets developers define the memory layout of objects. Far from varying "hugely", the programmer-specified memory layout of objects empirically barely has any effect at all on locality. This meme is cited a lot but doesn't actually hold up in reality. We've probably done one of the few actual studies of this in Rust, because we started reordering struct fields in the compiler instead of using the field order the developer wrote (though this can be turned off), and it barely affected locality at all. Memory usage was slightly improved (which argues against your point that developers, left to their own devices, specify optimal layouts).
- pizlonator 9y ago
  I thought he was talking about the existence (or lack) of struct types, not the ability to lay things out in a struct. The presence of struct types reduces allocation churn, so all memory management things become less important.
  - pcwalton 9y ago
    Yeah, if "letting developers define the memory layout of objects" means "value types" then that's obviously true. But C# has value types, and the generational hypothesis certainly holds for C#. So while good memory management becomes less important, it doesn't change what the optimal MM approach is.
    - pizlonator 9y ago
      Of course it changes the MM approach if allocation rate goes down. The higher the allocation rate, the larger the throughput advantage of compacting collectors. But compacting collectors have all sorts of other costs - complexity, hard to make them concurrent, potential for high barrier overheads. So, if your allocation rate is low enough, then the throughput advantage of the compacting collector might be low enough that you might think to use some other kind of collector. JSC is a great example. I should say that the allocation rate that matters is really the allocation rate scaled against the GC's speed; in classic terms we're talking about the cons/mark ratio. JS has a low cons/mark ratio (low relative allocation rate) because the JS collector is fundamentally no slower than any other one but the JS mutator is significantly slower than something like Java. So JS's allocation rate relative to the GC is lower than Java's allocation rate relative to the GC. This means that all GC throughput effects are dampened, which is one of the big reasons why JSC can get away with a collector that doesn't compact. Also, the generational hypothesis has nothing to do with whether compaction is profitable. These are orthogonal things. (EDIT: replaced "fundamentally now slower" with "fundamentally no slower", above)

pizlonator 9y ago

Totally! Java is in a class of languages that encourages a high allocation rate. I think that each language has a kind of "average" allocation rate that is an indirect outcome of the various design decisions in the language. I think you're right that C# probably has a lower average allocation rate because of structs. It's true that languages with lower average allocation rates benefit less from compaction, because they benefit less from any GC optimization.

geodel 9y ago

You might have checked but here is IanT's reply on "Why golang garbage-collector not implement Generational and Compact gc? There is also interesting discussion in thread. https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBUH1mHCAAJ https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBU... "However, modern memory allocation algorithms, like the tcmalloc-based approach used by the Go runtime, have essentially no fragmentation issues. And while a bump allocator can be simple and efficient for a single-threaded program, in a multi-threaded program like Go it requires locks. In general it's likely to be more efficient to allocate memory using a set of per-thread caches, and at that point you've lost the advantages of a bump allocator. So I would assert that, in general, with many caveats, there is no real advantage .."

pizlonator 9y ago
Bump allocators don't require locks. Bump allocators are especially effective in the presence of thread-local caches. Whoever wrote that must not be very familiar with how memory management works.
pcwalton 9y ago
> And while a bump allocator can be simple and efficient for a single-threaded program, in a multi-threaded program like Go it requires locks. In general it's likely to be more efficient to allocate memory using a set of per-thread caches, and at that point you've lost the advantages of a bump allocator. Ian is extremely confused here. Every production multithreaded generational GC uses thread-local allocation buffers (edit: thanks to Filip for correcting my terminology).
- pizlonator 9y ago
  Very few production multithreaded generational GCs use per-thread nurseries. But probably all of them use thread-local caches, or what you might call per-thread caches. I think that's what you might have meant. Per-thread nurseries is a shit idea unless your language has extra-super-high levels of infant mortality. Remember that in a per-thread nursery genGC, the write barrier has to infect the pointed-to object with the global heap so that future stores to that object also trigger the barrier. In a normal genGC, the pointed-to object is unaffected; the stored-to object is merely shaded (or card-marked, or logged, or whatever) for revisiting. Humorously for parent's post, to my knowledge the first instance of a thread-local cache was the idea that a multithreaded compacting GC can chop off a slice of memory from the global bump allocator and then use it to service a thread-local allocator. Not 100% sure though because maybe Boehm and friends beat everyone to it.
  - pcwalton 9y ago
    > Very few production multithreaded generational GCs use per-thread nurseries. > Per-thread nurseries is a shit idea unless your language has extra-super-high levels of infant mortality. Isn't that what TLABs are? HotSpot uses TLABs by default, unless my info is out of date… For example, [1] shows the generated assembler for object allocation, which uses nonatomic pointer bumps. I think we might be talking about different things by "per-thread nurseries". All I mean by a "per-thread nursery" is a TLAB. [1]: https://shipilev.net/jvm-anatomy-park/4-tlab-allocation/ https://shipilev.net/jvm-anatomy-park/4-tlab-allocation/
    - 9y ago
jerven 9y ago
I think that shipilev wrote this blog post after seeing that post. He and Gil Tene, of Azul Zing/C4 pauseless GC fame, don't seem to be impressed by the GO team GC claims. Mostly because the GO gc does fragment, which can lead to crashes when it shouldn't. The GO team seems to believe escape analysis means no need for generational hypothesis, while Gil and Shiplev think that escape analysis means less allocation (which does help collection, but not enough). Shiplev, these days works with the RedHat Shenadoah GC team. Which is low pause time (in the order of the Go Team promise) while compacting and fully parallel. So he knows a thing or two about modern GCs and JIT and compiler tech ;) Not to say that the Go team doesn't, it just that they might assume somethings which are not true for some apps.
- geodel 9y ago
  Don't know about him but Gil seems happy by Go's choices GC area. Here are Gil's view: https://groups.google.com/forum/#!searchin/mechanical-sympathy/Go$20gc%7Csort:relevance/mechanical-sympathy/JJTUs_jXw5A/55cRmOUoCwAJ https://groups.google.com/forum/#!searchin/mechanical-sympat... About Go: "Go's GC story is evolving. And it's doing so at a fairly quick pace. Judging it based on a current implementation and current choices is premature, I think. I think that the Go team's implementation priority focus on latency is a good one. And unlike others, I don't actually think there is a strong latency vs. throughput tradeoff in the long run." And Kind words about Hotspot: "The overall approach with HotSpot can be thought of as "start with a STW generational collector and increment your way to lower average/typical pauses in the old generations". E.g. all practical collectors in HotSpot use a monolithic stop-the-world copying newgen collector (not concurrent or incremental, monolithic, run-to-completion STW, always). And all collectors in HotSpot have a fallback to a full STW oldgen pause. The various non-monolithic-STW "cool modes" (Partially Concurrent non-compacting Oldgen in CMS, Incremental STW compaction in G1) are filters in the oldgen that occur between full STW newgen pauses and full STW oldgen pauses. They are "happy" when the filters work well enough to keep the STW newgen pauses in the tens of msec and delay the full-STW pauses for hours or days. And they consider it to be "ok and still working" even when those levels are not met." You can check Shenandoah pause times (20-45ms) : https://www.slideshare.net/RedHatDevelopers/shenandoah-gc-java-without-the-garbage-collection-hiccups-christine-flood https://www.slideshare.net/RedHatDevelopers/shenandoah-gc-ja... Decent for Java world but not really competing with Go. To add they seem very critical of Generational GC: "Generational GC pay a steep penalty for copying Data."

pcwalton 9y ago

> The importance of a moving/compacting GC varies hugely depending on whether or not a language/runtime lets developers define the memory layout of objects. Far from varying "hugely", the programmer-specified memory layout of objects empirically barely has any effect at all on locality. This meme is cited a lot but doesn't actually hold up in reality. We've probably done one of the few actual studies of this in Rust, because we started reordering struct fields in the compiler instead of using the field order the developer wrote (though this can be turned off), and it barely affected locality at all. Memory usage was slightly improved (which argues against your point that developers, left to their own devices, specify optimal layouts).

pizlonator 9y ago
I thought he was talking about the existence (or lack) of struct types, not the ability to lay things out in a struct. The presence of struct types reduces allocation churn, so all memory management things become less important.
- pcwalton 9y ago
  Yeah, if "letting developers define the memory layout of objects" means "value types" then that's obviously true. But C# has value types, and the generational hypothesis certainly holds for C#. So while good memory management becomes less important, it doesn't change what the optimal MM approach is.
  - pizlonator 9y ago
    Of course it changes the MM approach if allocation rate goes down. The higher the allocation rate, the larger the throughput advantage of compacting collectors. But compacting collectors have all sorts of other costs - complexity, hard to make them concurrent, potential for high barrier overheads. So, if your allocation rate is low enough, then the throughput advantage of the compacting collector might be low enough that you might think to use some other kind of collector. JSC is a great example. I should say that the allocation rate that matters is really the allocation rate scaled against the GC's speed; in classic terms we're talking about the cons/mark ratio. JS has a low cons/mark ratio (low relative allocation rate) because the JS collector is fundamentally no slower than any other one but the JS mutator is significantly slower than something like Java. So JS's allocation rate relative to the GC is lower than Java's allocation rate relative to the GC. This means that all GC throughput effects are dampened, which is one of the big reasons why JSC can get away with a collector that doesn't compact. Also, the generational hypothesis has nothing to do with whether compaction is profitable. These are orthogonal things. (EDIT: replaced "fundamentally now slower" with "fundamentally no slower", above)