6 ms·

I think that shipilev wrote this blog post after seeing that post. He and Gil Tene, of Azul Zing/C4 pauseless GC fame, don't seem to be impressed by the GO team

by jerven 9y ago

I think that shipilev wrote this blog post after seeing that post. He and Gil Tene, of Azul Zing/C4 pauseless GC fame, don't seem to be impressed by the GO team GC claims. Mostly because the GO gc does fragment, which can lead to crashes when it shouldn't. The GO team seems to believe escape analysis means no need for generational hypothesis, while Gil and Shiplev think that escape analysis means less allocation (which does help collection, but not enough). Shiplev, these days works with the RedHat Shenadoah GC team. Which is low pause time (in the order of the Go Team promise) while compacting and fully parallel. So he knows a thing or two about modern GCs and JIT and compiler tech ;) Not to say that the Go team doesn't, it just that they might assume somethings which are not true for some apps.

geodel 9y ago
Don't know about him but Gil seems happy by Go's choices GC area. Here are Gil's view: https://groups.google.com/forum/#!searchin/mechanical-sympathy/Go$20gc%7Csort:relevance/mechanical-sympathy/JJTUs_jXw5A/55cRmOUoCwAJ https://groups.google.com/forum/#!searchin/mechanical-sympat... About Go: "Go's GC story is evolving. And it's doing so at a fairly quick pace. Judging it based on a current implementation and current choices is premature, I think. I think that the Go team's implementation priority focus on latency is a good one. And unlike others, I don't actually think there is a strong latency vs. throughput tradeoff in the long run." And Kind words about Hotspot: "The overall approach with HotSpot can be thought of as "start with a STW generational collector and increment your way to lower average/typical pauses in the old generations". E.g. all practical collectors in HotSpot use a monolithic stop-the-world copying newgen collector (not concurrent or incremental, monolithic, run-to-completion STW, always). And all collectors in HotSpot have a fallback to a full STW oldgen pause. The various non-monolithic-STW "cool modes" (Partially Concurrent non-compacting Oldgen in CMS, Incremental STW compaction in G1) are filters in the oldgen that occur between full STW newgen pauses and full STW oldgen pauses. They are "happy" when the filters work well enough to keep the STW newgen pauses in the tens of msec and delay the full-STW pauses for hours or days. And they consider it to be "ok and still working" even when those levels are not met." You can check Shenandoah pause times (20-45ms) : https://www.slideshare.net/RedHatDevelopers/shenandoah-gc-java-without-the-garbage-collection-hiccups-christine-flood https://www.slideshare.net/RedHatDevelopers/shenandoah-gc-ja... Decent for Java world but not really competing with Go. To add they seem very critical of Generational GC: "Generational GC pay a steep penalty for copying Data."
- pcwalton 9y ago
  > About Go: You neglected to add the rest of the comment, which refutes the rest of your post: > I would predict that as Go's use application cases expand it's GC implementations will evolve to include generational moving collectors. (Emphasis mine.) > To add they seem very critical of Generational GC: That's their LRU cache benchmark, which is an artificial benchmark testing the worst case for generational GC, as they point out later in the slide deck. The takeaway from this should not be that generational GC is bad in general.
  - fmstephe 9y ago
    I would like to add another choice quotation from that discussion. "It is common to see early GC-based runtime implementations that do not move objects around succumb to an architectural "hole" that creates a reliance on that fact early in the maturity cycles. This usually happens by allowing native code extensions to assume object addresses are fixed (i.e. that objects will not move during their lifetime) and getting stuck with that "assumed quality" in the long run due to practical compatibility needs. From what I can tell, it is not too late for Go to avoid this problem." Previously I was not concerned about the long-term outlook for Go's GC. It's low pause (with some pathological cases) and currently very inefficient. The long term plan had previously mentioned moving to a generational/moving collector in the future. Gil's endorsement was cheering. But, Ian's comments on non-moving collector's on the Golang-nuts mailing list were alarming (and seemed technically confused). Time will tell.
    - geodel 9y ago
      > and currently very inefficient. Inefficient compared to what? Does it lead to Go apps use more memory? more CPU? longer execution time? Because I haven't seen much difference in those respect. In Fact in general Go apps use 2-10 times less memory than Java.
      - fmstephe 9y ago
        Good question, my claim was _very_ vague. The current GC in Go is inefficient in its use of CPU. Specifically because it uses a non-moving collector it has a sweep phase which frees each of the dead allocations. In Java, or similarly .net, the use of a moving collector means that dead objects aren't freed. The live objects are moved out of the current memory region and the whole region is then 'free'. If you have few live allocations and lots of dead allocations in that region then your GC cycle is much more efficient. I can't comment on memory usage experiences. I have written very careful Java programs that ran on the 64bit hotspot server in ~40mb or memory. I've written Java programs that used gigs, and I've written the same range in Go. I would like to quickly note that I actually like the Go GC. I think they are on a very promising path to a potentially great GC. But they are also given to some public hyperbole which I find awkward.

by jerven 9y ago

geodel 9y ago

Don't know about him but Gil seems happy by Go's choices GC area. Here are Gil's view: https://groups.google.com/forum/#!searchin/mechanical-sympathy/Go$20gc%7Csort:relevance/mechanical-sympathy/JJTUs_jXw5A/55cRmOUoCwAJ https://groups.google.com/forum/#!searchin/mechanical-sympat... About Go: "Go's GC story is evolving. And it's doing so at a fairly quick pace. Judging it based on a current implementation and current choices is premature, I think. I think that the Go team's implementation priority focus on latency is a good one. And unlike others, I don't actually think there is a strong latency vs. throughput tradeoff in the long run." And Kind words about Hotspot: "The overall approach with HotSpot can be thought of as "start with a STW generational collector and increment your way to lower average/typical pauses in the old generations". E.g. all practical collectors in HotSpot use a monolithic stop-the-world copying newgen collector (not concurrent or incremental, monolithic, run-to-completion STW, always). And all collectors in HotSpot have a fallback to a full STW oldgen pause. The various non-monolithic-STW "cool modes" (Partially Concurrent non-compacting Oldgen in CMS, Incremental STW compaction in G1) are filters in the oldgen that occur between full STW newgen pauses and full STW oldgen pauses. They are "happy" when the filters work well enough to keep the STW newgen pauses in the tens of msec and delay the full-STW pauses for hours or days. And they consider it to be "ok and still working" even when those levels are not met." You can check Shenandoah pause times (20-45ms) : https://www.slideshare.net/RedHatDevelopers/shenandoah-gc-java-without-the-garbage-collection-hiccups-christine-flood https://www.slideshare.net/RedHatDevelopers/shenandoah-gc-ja... Decent for Java world but not really competing with Go. To add they seem very critical of Generational GC: "Generational GC pay a steep penalty for copying Data."

pcwalton 9y ago
> About Go: You neglected to add the rest of the comment, which refutes the rest of your post: > I would predict that as Go's use application cases expand it's GC implementations will evolve to include generational moving collectors. (Emphasis mine.) > To add they seem very critical of Generational GC: That's their LRU cache benchmark, which is an artificial benchmark testing the worst case for generational GC, as they point out later in the slide deck. The takeaway from this should not be that generational GC is bad in general.
- fmstephe 9y ago
  I would like to add another choice quotation from that discussion. "It is common to see early GC-based runtime implementations that do not move objects around succumb to an architectural "hole" that creates a reliance on that fact early in the maturity cycles. This usually happens by allowing native code extensions to assume object addresses are fixed (i.e. that objects will not move during their lifetime) and getting stuck with that "assumed quality" in the long run due to practical compatibility needs. From what I can tell, it is not too late for Go to avoid this problem." Previously I was not concerned about the long-term outlook for Go's GC. It's low pause (with some pathological cases) and currently very inefficient. The long term plan had previously mentioned moving to a generational/moving collector in the future. Gil's endorsement was cheering. But, Ian's comments on non-moving collector's on the Golang-nuts mailing list were alarming (and seemed technically confused). Time will tell.
  - geodel 9y ago
    > and currently very inefficient. Inefficient compared to what? Does it lead to Go apps use more memory? more CPU? longer execution time? Because I haven't seen much difference in those respect. In Fact in general Go apps use 2-10 times less memory than Java.
    - fmstephe 9y ago
      Good question, my claim was _very_ vague. The current GC in Go is inefficient in its use of CPU. Specifically because it uses a non-moving collector it has a sweep phase which frees each of the dead allocations. In Java, or similarly .net, the use of a moving collector means that dead objects aren't freed. The live objects are moved out of the current memory region and the whole region is then 'free'. If you have few live allocations and lots of dead allocations in that region then your GC cycle is much more efficient. I can't comment on memory usage experiences. I have written very careful Java programs that ran on the 64bit hotspot server in ~40mb or memory. I've written Java programs that used gigs, and I've written the same range in Go. I would like to quickly note that I actually like the Go GC. I think they are on a very promising path to a potentially great GC. But they are also given to some public hyperbole which I find awkward.