4 ms·

More optimization based on making escape analysis happy to reduce allocations. This is yet another data point in favor of a generational garbage collector with

More optimization based on making escape analysis happy to reduce allocations. This is yet another data point in favor of a generational garbage collector with bump allocation in the nursery.

chrisseaton 8y ago
But escape analysis is useful no matter what collector you are using.
dullgiulio 8y ago
That's less than a fourth of what the article says. Copying data is sometimes faster than allocating it (and the TCMalloc-inspired Go allocator is quite quick.) Many small, allocations are inefficient in any language... There are other interesting points. And anyway, a bump allocator with nursery will spend longer time vetting the nursery and copying long-lived objects. In Go this would not pay back, as the nursery is mostly staying on the stack without special tricks to please the escape analysis. If you argued that Rust lifetimes system makes it easier to reason like the escape analysis would, I would agree, but you preferred to drop a tweet-size rant...
- pcwalton 8y ago
  tcmalloc cannot compare to allocation performance in HotSpot, where allocation is about 5 or 6 instructions. This is the same size as a function prologue and epilogue. You're denying that the generational hypothesis holds for Go heap objects. I see no reason why this would be the case. In fact, Go's situation is very close to that of .NET, where the generational hypothesis certainly holds and therefore .NET has a generational garbage collector. This very article is evidence in favor of the generational hypothesis, since much of the optimizations boil down to reducing short-lived heap allocations.
  - troytchell 8y ago
    The creators of Go are Rob Pike and Google. I think they would know if a different GC were better and use it. I used to sometimes doubt Rob Pike too and think Go had made some mistakes. After using Go and listening to his talks (like [0]) I realized how wrong I was. There's a reason he has helped build two wildly successful programming languages.. How many have you built? Are you smarter than Google? In the end, embracing Rob Pike's brilliant decisions and taste has made my programming life so much simpler and better. I switched to Acme [1], turned off syntax highlighting on Gitub PR reviews with a userscript[2], and my life is infinitely simpler. Rob Pike has been right on so many things and has made Go successful. I'm sure if a generational GC were better for Go, he would have built and used it, so you're obviously wrong. [0]: https://www.youtube.com/watch?v=rFejpH_tAHM https://www.youtube.com/watch?v=rFejpH_tAHM [1]: http://acme.cat-v.org/ http://acme.cat-v.org/ [2]: https://groups.google.com/d/msg/golang-nuts/hJHCAaiL0so/kG3BHV6QFfIJ https://groups.google.com/d/msg/golang-nuts/hJHCAaiL0so/kG3B...
  - lossolo 8y ago
    And again same thing about HotSpot that you repeat third time in last week-two weeks? Java has new GC which is non-generational and is focused on low latency like Go GC is. It's called ZGC. http://openjdk.java.net/jeps/333 http://openjdk.java.net/jeps/333
    - pcwalton 8y ago
      ZGC has a very different design. Go's GC is non-moving and has no read barrier. ZGC has a read barrier and bump allocation everywhere (using compaction to ensure that memory regions are contiguous). As a result it does not suffer the throughput issues of Go's GC.
      - lossolo 8y ago
        It's not moving/compacting because it doesn't need to - it doesn't have fragmentation issues. Bump allocation is efficient and fast in single threaded programs but almost all Go applications are multi threaded and it would require locks there. Go is using thread local caches for allocation, that's why there is no point in using bump allocation. This was mentioned many times by the Go team. Generational hypothesis says that most objects die young. Golang have value types (which Java do not have atm, but Valhalla is coming) and allocate on the stack based on escape analysis (which gets better with time). This means that adding generational GC would not benefit Go as much as you think. Go GC main focus is very low latency, not throughput and handling huge heaps with a lot of generations is not so easy with HotSpot. In many cases you need to tune GC in Java because of that. There are trade offs like with everything. I am running high performance service written in Golang and I was working on high performance services in Java. This are the pauses from Go service on XX GB heaps from today: https://imgur.com/tSXtP4a https://imgur.com/tSXtP4a This is without tuning GC or writing exceptionally unidiomatic Go code. This is impossible to achieve in Java with Hot Spot without tuning GC/writing your whole code specially to satisfy GC.
        chrisseaton 8y ago
        > Bump allocation is efficient and fast in single threaded programs but almost all Go applications are multi threaded and it would require locks there. If you’re using bump allocation you do it per thread! You don’t do it globally and use a lock!
  - gok 8y ago
    But a stack allocation is 0 or 1 instructions.
    - pcwalton 8y ago
      Not if you have to check for stack space, like Go does. It's no coincidence that a bump allocator and a function prologue have exactly the same structure: check against a limit and bump a thread-local pointer. There are runtimes like Chicken Scheme that actually implement heaps using the stack!
_ph_ 8y ago
Reducing allocations isn't about the speed of allocations. Yes, allocating with a generational GC is very fast, as is stack allocation. Reducing allocations is about reducing the time needed to perform garbage collection. That time is zero for stack based allocation and non-zero for any heap based allocation. Stack based allocation or memory pools etc. reduce the pressure on the GC. Having more GC runs means not only time spent in the GC, with generational GCs it also can mean premature aging of objects - while collecting the nursery is very fast, collecting older generations has a larger cost. So, with any GC and language, the less heap allocations, the faster your code will run. (Unless of course, your program code gets much worse by trying to avoid heap allocations).
- pcwalton 8y ago
  To nitpick, the time for stack allocation isn't zero. Stack objects still have to be traced. But yes, there is some benefit to reducing allocation. The biggest benefit to escape analysis doesn't really have anything to do with allocation at all. Rather it's that promoting objects to the stack enables SROA, which unlocks a lot more optimizations. This article wasn't about that, though. These were short-lived benchmarks that are dominated by throughput. In those types of benchmarks, allocation speed matters much more than mark and sweep time. So, the best way to optimize for these sorts of workloads is to add generational garbage collection with bump allocation in the nursery.
  - _ph_ 8y ago
    I didn't claim that the cost for stack allocation was zero. I claimed that the cost for stack deallocation is zero. Thats where the stack wins against garbage collectors.
gok 8y ago
You potentially would also have to do work to keep the generational GC happy. It can be easy to accidentally have allocations escape the nursery.

More optimization based on making escape analysis happy to reduce allocations. This is yet another data point in favor of a generational garbage collector with bump allocation in the nursery.

chrisseaton 8y ago

But escape analysis is useful no matter what collector you are using.

dullgiulio 8y ago

That's less than a fourth of what the article says. Copying data is sometimes faster than allocating it (and the TCMalloc-inspired Go allocator is quite quick.) Many small, allocations are inefficient in any language... There are other interesting points. And anyway, a bump allocator with nursery will spend longer time vetting the nursery and copying long-lived objects. In Go this would not pay back, as the nursery is mostly staying on the stack without special tricks to please the escape analysis. If you argued that Rust lifetimes system makes it easier to reason like the escape analysis would, I would agree, but you preferred to drop a tweet-size rant...

pcwalton 8y ago
tcmalloc cannot compare to allocation performance in HotSpot, where allocation is about 5 or 6 instructions. This is the same size as a function prologue and epilogue. You're denying that the generational hypothesis holds for Go heap objects. I see no reason why this would be the case. In fact, Go's situation is very close to that of .NET, where the generational hypothesis certainly holds and therefore .NET has a generational garbage collector. This very article is evidence in favor of the generational hypothesis, since much of the optimizations boil down to reducing short-lived heap allocations.
- troytchell 8y ago
  The creators of Go are Rob Pike and Google. I think they would know if a different GC were better and use it. I used to sometimes doubt Rob Pike too and think Go had made some mistakes. After using Go and listening to his talks (like [0]) I realized how wrong I was. There's a reason he has helped build two wildly successful programming languages.. How many have you built? Are you smarter than Google? In the end, embracing Rob Pike's brilliant decisions and taste has made my programming life so much simpler and better. I switched to Acme [1], turned off syntax highlighting on Gitub PR reviews with a userscript[2], and my life is infinitely simpler. Rob Pike has been right on so many things and has made Go successful. I'm sure if a generational GC were better for Go, he would have built and used it, so you're obviously wrong. [0]: https://www.youtube.com/watch?v=rFejpH_tAHM https://www.youtube.com/watch?v=rFejpH_tAHM [1]: http://acme.cat-v.org/ http://acme.cat-v.org/ [2]: https://groups.google.com/d/msg/golang-nuts/hJHCAaiL0so/kG3BHV6QFfIJ https://groups.google.com/d/msg/golang-nuts/hJHCAaiL0so/kG3B...
- lossolo 8y ago
  And again same thing about HotSpot that you repeat third time in last week-two weeks? Java has new GC which is non-generational and is focused on low latency like Go GC is. It's called ZGC. http://openjdk.java.net/jeps/333 http://openjdk.java.net/jeps/333
  - pcwalton 8y ago
    ZGC has a very different design. Go's GC is non-moving and has no read barrier. ZGC has a read barrier and bump allocation everywhere (using compaction to ensure that memory regions are contiguous). As a result it does not suffer the throughput issues of Go's GC.
    - lossolo 8y ago
      It's not moving/compacting because it doesn't need to - it doesn't have fragmentation issues. Bump allocation is efficient and fast in single threaded programs but almost all Go applications are multi threaded and it would require locks there. Go is using thread local caches for allocation, that's why there is no point in using bump allocation. This was mentioned many times by the Go team. Generational hypothesis says that most objects die young. Golang have value types (which Java do not have atm, but Valhalla is coming) and allocate on the stack based on escape analysis (which gets better with time). This means that adding generational GC would not benefit Go as much as you think. Go GC main focus is very low latency, not throughput and handling huge heaps with a lot of generations is not so easy with HotSpot. In many cases you need to tune GC in Java because of that. There are trade offs like with everything. I am running high performance service written in Golang and I was working on high performance services in Java. This are the pauses from Go service on XX GB heaps from today: https://imgur.com/tSXtP4a https://imgur.com/tSXtP4a This is without tuning GC or writing exceptionally unidiomatic Go code. This is impossible to achieve in Java with Hot Spot without tuning GC/writing your whole code specially to satisfy GC.
      - chrisseaton 8y ago
        > Bump allocation is efficient and fast in single threaded programs but almost all Go applications are multi threaded and it would require locks there. If you’re using bump allocation you do it per thread! You don’t do it globally and use a lock!
- gok 8y ago
  But a stack allocation is 0 or 1 instructions.
  - pcwalton 8y ago
    Not if you have to check for stack space, like Go does. It's no coincidence that a bump allocator and a function prologue have exactly the same structure: check against a limit and bump a thread-local pointer. There are runtimes like Chicken Scheme that actually implement heaps using the stack!

_ph_ 8y ago

Reducing allocations isn't about the speed of allocations. Yes, allocating with a generational GC is very fast, as is stack allocation. Reducing allocations is about reducing the time needed to perform garbage collection. That time is zero for stack based allocation and non-zero for any heap based allocation. Stack based allocation or memory pools etc. reduce the pressure on the GC. Having more GC runs means not only time spent in the GC, with generational GCs it also can mean premature aging of objects - while collecting the nursery is very fast, collecting older generations has a larger cost. So, with any GC and language, the less heap allocations, the faster your code will run. (Unless of course, your program code gets much worse by trying to avoid heap allocations).

pcwalton 8y ago
To nitpick, the time for stack allocation isn't zero. Stack objects still have to be traced. But yes, there is some benefit to reducing allocation. The biggest benefit to escape analysis doesn't really have anything to do with allocation at all. Rather it's that promoting objects to the stack enables SROA, which unlocks a lot more optimizations. This article wasn't about that, though. These were short-lived benchmarks that are dominated by throughput. In those types of benchmarks, allocation speed matters much more than mark and sweep time. So, the best way to optimize for these sorts of workloads is to add generational garbage collection with bump allocation in the nursery.
- _ph_ 8y ago
  I didn't claim that the cost for stack allocation was zero. I claimed that the cost for stack deallocation is zero. Thats where the stack wins against garbage collectors.

gok 8y ago

You potentially would also have to do work to keep the generational GC happy. It can be easy to accidentally have allocations escape the nursery.