5 ms·

"The execution time is now dominated by the allocation and the garbage collection of small objects (e.g. the Message struct), which make sense because memory ma

"The execution time is now dominated by the allocation and the garbage collection of small objects (e.g. the Message struct), which make sense because memory management operations are known to be relatively slow." A perfect example of why Go should switch to a generational garbage collector with bump allocation in the nursery, like Java HotSpot, .NET, V8, etc. etc. use. There are consequences to Go's decision to make extreme throughput sacrifices in order to optimize for latency (low pause times) above all else.

foldr 8y ago
It's worth noting that Go, unlike Java, can pass objects by value. That makes it less crucial for the GC to be able to deal efficiently with small allocations, since a typical Go program will have considerably fewer of these than a typical Java program.
- pcwalton 8y ago
  Same with .NET (C#), where the generational hypothesis certainly holds and therefore .NET has a generational garbage collector. It's also pretty bad from an ergonomics (not to mention performance) perspective if the programmer has to manually switch to by-value copying of structs in order to keep escape analysis happy and avoid allocation.
  - foldr 8y ago
    Nonetheless, having copy-by-value semantics available for small objects reduces the need for a generational collector. I'm not sure what you're referring to w.r.t ergonomics. Passing small structs by value should probably be the default choice in Go, and it is not difficult to switch from passing via a pointer to passing by value.
    - yxhuvud 8y ago
      It reduces the need for a collector, any collector, as there will be less garbage. You have not made any arguments that indicate or even hint that the actual share of young objects that will be generated will be reduced.
      - foldr 8y ago
        I believe there is generally thought to be a rough correlation between object size and object lifetime in most programs. So reducing the number of small objects that are allocated in the GCed heap should, on average, reduce the proportion of young objects. Indeed, if an object can be passed by value and/or allocated on the stack following escape analysis, that suggests that it is an object that would have had a short lifetime in the equivalent Java program. But I don't disagree with what you are saying. It may be that Go would perform better with a generational GC. My claim is not that Go is better off with a non-generational GC (I don't know if it is or it isn't), but simply that it is less in need of one than e.g. Java.
        pcwalton 8y ago
        I don't believe that's true, because the Java VM will promote heap objects to stack objects via escape analysis. It even can do this post-speculative-devirtualization.
        foldr 8y ago
        I'm not seeing the connection between that and anything that I said. Could you expand a bit? edit: Do you mean that adding pass-object-by-value semantics to Java wouldn't decrease the number of short-lived object allocations because Java already does escape analysis? That may well be true. My claim was that the availability of pass-object-by-value in Go probably reduces the number of short-lived object allocations in Go, compared to what it would be if objects could not be passed by value and everything else remained the same.
  - weberc2 8y ago
    > It's also pretty bad from an ergonomics (not to mention performance) perspective if the programmer has to manually switch to by-value copying of structs in order to keep escape analysis happy and avoid allocation. I kind of like that Go's escape analysis is simple enough that I can reason about it reliably without profiling. I also tend to pass and return values unless I have a compelling reason to pass pointers (like mutation or rarely performance in the case of large structs), so I don't think I experience the sort of ergonomic issues you cite (and incidentally, I think this habit of mine makes writing Rust feel quite unergonomic to me since it seems to prefer egregious use of pointers). And I haven't noticed a significant speedup by switching from value copying to pointer dereferencing (but I've only tested this a handful of times).
    - pcwalton 8y ago
      You shouldn't have to manually reason about escape analysis. As compiler optimizations of 6g/8g evolve, your ability to reason about it will probably dramatically decrease, due to the effects of inlining, SROA, GVN, LICM, etc. etc. That's why ultimately there's no escaping the need for generational GC. The beauty of bump allocation in the nursery is that heap allocation is as cheap as stack allocation, at least in the common case.
      - foldr 8y ago
        Even if heap allocation itself is not more expensive, there must clearly be performance benefits to stack allocation, or the JVM wouldn't bother doing escape analysis in the first place.
- andy0x2a 8y ago
  Java passes objects by value, not by reference. Try implementing swap(Object a, Object b) in Java, and you will see it is by Val not by ref
  - scott_s 8y ago
    Java passes references to objects by value. That is, in Java, a variable for a heap-allocated object is a reference to that object. When you pass that variable to a method, the method gets a copy of that reference. So if the method makes changes to the object itself, that is visible outside of the method, but if it makes changes to its own copy of the reference (say, makes it point to a different object), that's not visible outside of the method. I still consider this behavior "passing by reference" since it passes around references/pointers to objects, and does not incur an object copy.
  - foldr 8y ago
    Expanding on the sibling, Java is a pass by value language which is commonly mistakenly thought to be pass by reference. (That's probably the mistake you think I'm making.) It is not, however, possible to pass objects by value in Java.
cdoxsey 8y ago
Ian Lance Taylor replied to this question once: (https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBUH1mHCAAJ https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBU...) > Now let's consider a generational GC. The point of a generational GC relies on the generational hypothesis: that most values allocated in a program are quickly unused, so there is an advantage for the GC to spend more time looking at recently allocated objects. Here Go differs from many garbage collected languages in that many objects are allocated directly on the program stack. The Go compiler uses escape analysis to find objects whose lifetime is known at compile time, and allocates them on the stack rather than in garbage collected memory. So in general, in Go, compared to other languages, a larger percentage of the quickly-unused values that a generational GC looks for are never allocated in GC memory in the first place. So a generational GC would likely bring less advantage to Go than it does for other languages. > More subtly, the implicit point of most generational GC implementations is to reduce the amount of time that a program pauses for garbage collection. By looking at only the youngest generation during a pause, the pause is kept short. However, Go uses a concurrent garbage collector, and in Go the pause time is independent of the size of the youngest generation, or of any generation. Go is basically assuming that in a multi-threaded program it is better overall to spend slightly more total CPU time on GC, by running GC in parallel on a different core, rather than to minimize GC time but to pause overall program execution for longer. > All that said, generational GC could perhaps still bring significant value to Go, by reducing the amount of work the GC has to do even in parallel. It's a hypothesis that needs to be tested. Current GC work in Go is actually looking closely at a related but different hypothesis: that Go programs may tend to allocate memory on a per-request basis. This is described at https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ftk-tcRRJcDFANj2VwCB0/view https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ft... . This is work in progress and it remains to be seen whether it will be advantageous in reality.
- pcwalton 8y ago
  I'm aware of Ian's arguments. He's (a) too dismissive of the huge throughput benefits of generational GC; (b) ignoring the fact that both HotSpot and C# allocate objects on the stack too, with effectively superior escape analysis to that of Go because of speculative devirtualization; (c) neglecting the fact that Go's situation is almost the same as that of .NET, where the generational hypothesis certainly holds; (d) treating "request-oriented" GC as something other than generational GC, when it's really just a somewhat limited form of generational GC. It's just a pointless thing for the Go team to dig in their heels over. They should implement generational GC, like most other languages with optimizing compilers do.

foldr 8y ago

It's worth noting that Go, unlike Java, can pass objects by value. That makes it less crucial for the GC to be able to deal efficiently with small allocations, since a typical Go program will have considerably fewer of these than a typical Java program.

pcwalton 8y ago
Same with .NET (C#), where the generational hypothesis certainly holds and therefore .NET has a generational garbage collector. It's also pretty bad from an ergonomics (not to mention performance) perspective if the programmer has to manually switch to by-value copying of structs in order to keep escape analysis happy and avoid allocation.
- foldr 8y ago
  Nonetheless, having copy-by-value semantics available for small objects reduces the need for a generational collector. I'm not sure what you're referring to w.r.t ergonomics. Passing small structs by value should probably be the default choice in Go, and it is not difficult to switch from passing via a pointer to passing by value.
  - yxhuvud 8y ago
    It reduces the need for a collector, any collector, as there will be less garbage. You have not made any arguments that indicate or even hint that the actual share of young objects that will be generated will be reduced.
    - foldr 8y ago
      I believe there is generally thought to be a rough correlation between object size and object lifetime in most programs. So reducing the number of small objects that are allocated in the GCed heap should, on average, reduce the proportion of young objects. Indeed, if an object can be passed by value and/or allocated on the stack following escape analysis, that suggests that it is an object that would have had a short lifetime in the equivalent Java program. But I don't disagree with what you are saying. It may be that Go would perform better with a generational GC. My claim is not that Go is better off with a non-generational GC (I don't know if it is or it isn't), but simply that it is less in need of one than e.g. Java.
      - pcwalton 8y ago
        I don't believe that's true, because the Java VM will promote heap objects to stack objects via escape analysis. It even can do this post-speculative-devirtualization.
        foldr 8y ago
        I'm not seeing the connection between that and anything that I said. Could you expand a bit? edit: Do you mean that adding pass-object-by-value semantics to Java wouldn't decrease the number of short-lived object allocations because Java already does escape analysis? That may well be true. My claim was that the availability of pass-object-by-value in Go probably reduces the number of short-lived object allocations in Go, compared to what it would be if objects could not be passed by value and everything else remained the same.
- weberc2 8y ago
  > It's also pretty bad from an ergonomics (not to mention performance) perspective if the programmer has to manually switch to by-value copying of structs in order to keep escape analysis happy and avoid allocation. I kind of like that Go's escape analysis is simple enough that I can reason about it reliably without profiling. I also tend to pass and return values unless I have a compelling reason to pass pointers (like mutation or rarely performance in the case of large structs), so I don't think I experience the sort of ergonomic issues you cite (and incidentally, I think this habit of mine makes writing Rust feel quite unergonomic to me since it seems to prefer egregious use of pointers). And I haven't noticed a significant speedup by switching from value copying to pointer dereferencing (but I've only tested this a handful of times).
  - pcwalton 8y ago
    You shouldn't have to manually reason about escape analysis. As compiler optimizations of 6g/8g evolve, your ability to reason about it will probably dramatically decrease, due to the effects of inlining, SROA, GVN, LICM, etc. etc. That's why ultimately there's no escaping the need for generational GC. The beauty of bump allocation in the nursery is that heap allocation is as cheap as stack allocation, at least in the common case.
    - foldr 8y ago
      Even if heap allocation itself is not more expensive, there must clearly be performance benefits to stack allocation, or the JVM wouldn't bother doing escape analysis in the first place.
andy0x2a 8y ago
Java passes objects by value, not by reference. Try implementing swap(Object a, Object b) in Java, and you will see it is by Val not by ref
- scott_s 8y ago
  Java passes references to objects by value. That is, in Java, a variable for a heap-allocated object is a reference to that object. When you pass that variable to a method, the method gets a copy of that reference. So if the method makes changes to the object itself, that is visible outside of the method, but if it makes changes to its own copy of the reference (say, makes it point to a different object), that's not visible outside of the method. I still consider this behavior "passing by reference" since it passes around references/pointers to objects, and does not incur an object copy.
- foldr 8y ago
  Expanding on the sibling, Java is a pass by value language which is commonly mistakenly thought to be pass by reference. (That's probably the mistake you think I'm making.) It is not, however, possible to pass objects by value in Java.

cdoxsey 8y ago

Ian Lance Taylor replied to this question once: (https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBUH1mHCAAJ https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBU...) > Now let's consider a generational GC. The point of a generational GC relies on the generational hypothesis: that most values allocated in a program are quickly unused, so there is an advantage for the GC to spend more time looking at recently allocated objects. Here Go differs from many garbage collected languages in that many objects are allocated directly on the program stack. The Go compiler uses escape analysis to find objects whose lifetime is known at compile time, and allocates them on the stack rather than in garbage collected memory. So in general, in Go, compared to other languages, a larger percentage of the quickly-unused values that a generational GC looks for are never allocated in GC memory in the first place. So a generational GC would likely bring less advantage to Go than it does for other languages. > More subtly, the implicit point of most generational GC implementations is to reduce the amount of time that a program pauses for garbage collection. By looking at only the youngest generation during a pause, the pause is kept short. However, Go uses a concurrent garbage collector, and in Go the pause time is independent of the size of the youngest generation, or of any generation. Go is basically assuming that in a multi-threaded program it is better overall to spend slightly more total CPU time on GC, by running GC in parallel on a different core, rather than to minimize GC time but to pause overall program execution for longer. > All that said, generational GC could perhaps still bring significant value to Go, by reducing the amount of work the GC has to do even in parallel. It's a hypothesis that needs to be tested. Current GC work in Go is actually looking closely at a related but different hypothesis: that Go programs may tend to allocate memory on a per-request basis. This is described at https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ftk-tcRRJcDFANj2VwCB0/view https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ft... . This is work in progress and it remains to be seen whether it will be advantageous in reality.

pcwalton 8y ago
I'm aware of Ian's arguments. He's (a) too dismissive of the huge throughput benefits of generational GC; (b) ignoring the fact that both HotSpot and C# allocate objects on the stack too, with effectively superior escape analysis to that of Go because of speculative devirtualization; (c) neglecting the fact that Go's situation is almost the same as that of .NET, where the generational hypothesis certainly holds; (d) treating "request-oriented" GC as something other than generational GC, when it's really just a somewhat limited form of generational GC. It's just a pointless thing for the Go team to dig in their heels over. They should implement generational GC, like most other languages with optimizing compilers do.