6 ms·

Every perf guide recommends to minimize allocations to reduce GC times, but if you look at pprof of a Go app, GC mark phase is what takes time, not GC sweep. GC

Every perf guide recommends to minimize allocations to reduce GC times, but if you look at pprof of a Go app, GC mark phase is what takes time, not GC sweep. GC mark always starts with known live roots (goroutine stacks, globals, etc) and traverse references from there colouring every pointer. To minimize GC time it is best to avoid _long living_ allocations. Short lived allocations, those which GC mark phase will never reach, has almost neglible effect on GC times. Allocations of any kind have an effect on triggering GC earlier, but in real apps it is almost hopeless to avoid GC, except for very carefully written programs with no dependenciesm, and if GC happens, then reducing GC mark times gives bigger bang for the buck.

nurettin 2y ago
Is it worth making short lived allocations just to please the GC? You might just end up with too many allocations which will slow things down even more.
- aktau 2y ago
  It is not. Please see my answer (https://news.ycombinator.com/item?id=43545500 https://news.ycombinator.com/item?id=43545500).
MarkMarine 2y ago
Are you including in this analysis the amount of time/resources it takes to allocate? GC isn't the only thing you want to minimize for when you're making a high performance system.
- nopurpose 2y ago
  From that perspective it boils down to "do less", which is what any perf guide already includes, allocations is just no different from anything else what app do. My comment is more about "reduce allocations to reduce GC pressure" advice seen everywhere. It doesn't tell the whole story. Short lived allocation doesn't introduce any GC pressure: you'll be hard pressed to see GC sweep phase on pprof without zooming. People take this advice, spend time and energy hunting down allocations, just to see that total GC time remained the same after all that effort, because they were focusing on wrong type of allocations.
  - MarkMarine 2y ago
    Yeah I understand what you’re saying, but my point is you’re doing the opposite side of the same coin. Not doing full perf analysis and saying this one method works (yours is to reduce GC mark time, ignoring allocation, others are trying to reduce allocation time, ignoring GC time, or all these other methods listed in this doc.)
Capricorn2481 2y ago
Aren't allocations themselves pretty expensive regardless of GC?
- nu11ptr 2y ago
  Go allocations aren't that bad. A few years ago I benchmarked them at about 4x as expensive as a bump allocation. That is slow enough to make an arena beneficial in high allocation situations, but fast enough to not make it worth it most of the time.
  - aktau 2y ago
    Comparing with a fairly optimized malloc at $COMPANY, the Go allocator is (both in terms of relative cycles and fraction of cycles of all Go programs) significantly more expensive than the C/C++ counterpart (3-4x IIRC). For one, it has to do more work, like setting up GC metadata, and zeroing. There have recently been some optimizations to `runtime.mallocgc`, which may have decrease that 3-4x estimate a bit.
    - nu11ptr 2y ago
      How can that be true? If it is 3-4x more expensive than malloc, then per my measurements your malloc is a bump allocator, and that simply isn't true for any real world malloc implementation (typically a modified free list allocator afaik). `mallocgc` may not be fast, but I simply did not find it as slow as you are saying. My guess is it is about as fast as most decent malloc functions, but I have not measured, and it would be interesting to see a comparison (tough to do as you'd need to call malloc via CGo or write one in C and one in Go and trust the looping is roughly the same cost).
      - aktau 2y ago
        I should correct and clarify: I meant 3-4x more expensive in relative terms. Meaning: - For C++ programs, the allocator (allocating+freeing) consumes roughly 5% of cycles. - For Go programs, the allocator (runtime.mallocgc) used to consume ~20% of cycles (this is the data I referenced). I checked and recently it's become closer to 15%, thanks to optimizations. I have not tested the performance differential on a per-byte level (though that will also differ with object structure in Go).
raggi 2y ago
You might wanna look at a system profiler too, pprof doesn't show everything.
zmj 2y ago
Pretty similar story in .NET. Make sure your inner loops are allocation-free, then ensure allocations are short-lived, then clean up the long tail of large allocations.
- neonsunset 2y ago
  .NET is far more tolerant to high allocation traffic since its GC is generational and overall more sophisticated (even if at the cost of tail latency, although that is workload-dependent). Doing huge allocations which go to LOH is quite punishing, but even substantial inter-generational traffic won't kill it.
zbobet2012 2y ago
E.h., kind of. If you are allocating in a hot loop it's going to suck regardless. Object pools are really key if you want high perf because the general purpose allocator is way less efficient in comparison.
liquidgecka 2y ago
Its worth calling out that abstractions can kill you in unexpected ways with go. Anytime you use an interface it forces a heap allocation, even if the object is only used read only and within the same scope. That includes calls to things like fmt.Printf() so doing a for loop that prints the value of i forces the integer backing i to be heap allocated, along with every other value that you printed. So if you helpfully make every api in your library use an interface you are forcing the callers to use heap allocations for every single operation.
- slashdev 2y ago
  I thought surely an integer could be inlined into the interface, I thought Go used to do that. But I tried it on the playground, and it heap allocates it: https://go.dev/play/p/zHfnQfJ9OGc https://go.dev/play/p/zHfnQfJ9OGc
  - masklinn 2y ago
    Go did use to do that, it was removed years ago, in 1.4: https://go.dev/doc/go1.4#runtime https://go.dev/doc/go1.4#runtime
    - kbolino 2y ago
      Basically, anything that isn't a thin pointer (*T, chan, map) gets boxed nowadays. The end result is that both words of an interface value are always pointers [1], which is very friendly to the garbage collector (setting aside the extra allocations when escape analysis fails). I've seen some tricks in the standard library to avoid boxing, e.g. how strings and times are handled by log/slog [2]. [1]: https://github.com/teh-cmc/go-internals/blob/master/chapter2_interfaces/README.md#overview-of-the-datastructures https://github.com/teh-cmc/go-internals/blob/master/chapter2... [2]: https://cs.opensource.google/go/go/+/refs/tags/go1.24.1:src/log/slog/value.go;l=21 https://cs.opensource.google/go/go/+/refs/tags/go1.24.1:src/...
      - deleted 2y ago
        [deleted]
ncruces 2y ago
The point is not to avoid GC entirely, but to reduce allocation pressure. If you can avoid allocs in a hot loop, it definitely pays to do so. If you can't for some reason, and can use sync.Pool there, measure it. Cutting allocs in half may not matter much, but if you can cut them by 99% because you were allocating in every iteration of a 1 million loop, and now aren't, it will make a difference, even if all those allocs die instantly. I've gotten better than two fold performance increases on real code with both techniques.
bboreham 2y ago
Agree that mark phase is the expensive bit. Disagree that it’s not worth reducing short-lived allocations. I spend a lot of time analyzing Go program performance, and reducing bytes allocated per second is always beneficial.
- felixge 2y ago
  +1. In particular []byte slice allocations are often a significant driver of GC pace while also being relatively easy to optimize (e.g. via sync.Pool reuse).
aktau 2y ago
Side note: see https://tip.golang.org/doc/gc-guide https://tip.golang.org/doc/gc-guide for more on how the Go GC works and what triggers it. GC frequency is directly driven by allocation rate (in terms of bytes) and live heap size. Some examples: - If you halve the allocation rate, you halve the GC frequency. - If you double the live heap size, you halve the GC frequency (barring changes away from the default `GOGC=100`). > ...but if you look at pprof of a Go app, GC mark phase is what takes time, not GC sweep. It is true that sweeping is a lot cheaper than marking, which makes your next statement: > Short lived allocations, those which GC mark phase will never reach, has almost neglible effect on GC times. ...technically correct. Usually, this is the best kind of correct, but it omits two important considerations: - If you generate a ton of short-lived allocations instead of keeping them around, the GC will trigger more frequently. - If you reduce the live heap size (by not keeping anything around), the GC will trigger more frequently. So now you have cheaper GC cycles, but many more of them. On top of that, you have vastly increased allocation costs. It is not a priori clear to me this is a win. In my experience, it isn't.
- nopurpose 2y ago
  I enjoyed your detailed response, it adds value to this discussion, but I feel you missed the point of my comment. I am against blanket statements "reduce allocations to reduce GC pressure", which lead people wrong way: they compare libraries based on "allocs/op" from go bench, they trust rediculous (who allocates 8KB per iteration in tight loop??) microbenchmarks of sync.Pool like in the article above, hoping to resolve their GC problem. Spend considerabe amount of effort just to find that they barely moved a needle on GC times. If we generalize then my "avoid long-lived allocations" or yours "reduce allocation rate in terms of bytes" are much more useful in practice, than what this and many other articles preach.
- deepsun 2y ago
  Interesting, thank you. But I think those points are not correlated that much. For example if I create unnecessary wrappers in a loop, I might double the allocation rate, but I will not halve the live heap size, because I did not have those wrappers outside the loop before. Basically, I'm trying to come up with an real world example of a style change (like create wrappers for every error, or use naked integers instead of time.Time) to estimate its impact. And my feeling is that any such example would affect one of your points way more than the other, so we can still argument that e.g. "creating short-lived iterators is totally fine".
deepsun 2y ago
Interesting, and I think that is not specific to Go, other mark-and-sweep GCs (Java, C#) should behave the same. Which means that creating short lived objects (like iterators for loops, or some wrappers) is ok.
- int_19h 2y ago
  It should be noted that in C#, at least, the standard pattern is to use value types for enumerators, precisely so as to avoid heap allocations. This is the case for all (non-obsolete) collections in the .NET stdlib - e.g. List<T>.Enumerator is a struct.
- ted_dunning 2y ago
  Not entirely. Go still doesn't have a generational collector so high allocation rates cause more GC's that must examine long-lived objects. As such, short-lived objects have little impact in Java (thank god for that!). They will have second order effects in Go.
kgeist 2y ago
The runtime also forces GC every 2 minutes. So yeah, a lot of long living allocations can stress the GC, even if you don't allocate often. That's why Discord moved from Go to Rust for their Read States server.

nurettin 2y ago

Is it worth making short lived allocations just to please the GC? You might just end up with too many allocations which will slow things down even more.

aktau 2y ago
It is not. Please see my answer (https://news.ycombinator.com/item?id=43545500 https://news.ycombinator.com/item?id=43545500).

MarkMarine 2y ago

Are you including in this analysis the amount of time/resources it takes to allocate? GC isn't the only thing you want to minimize for when you're making a high performance system.

nopurpose 2y ago
From that perspective it boils down to "do less", which is what any perf guide already includes, allocations is just no different from anything else what app do. My comment is more about "reduce allocations to reduce GC pressure" advice seen everywhere. It doesn't tell the whole story. Short lived allocation doesn't introduce any GC pressure: you'll be hard pressed to see GC sweep phase on pprof without zooming. People take this advice, spend time and energy hunting down allocations, just to see that total GC time remained the same after all that effort, because they were focusing on wrong type of allocations.
- MarkMarine 2y ago
  Yeah I understand what you’re saying, but my point is you’re doing the opposite side of the same coin. Not doing full perf analysis and saying this one method works (yours is to reduce GC mark time, ignoring allocation, others are trying to reduce allocation time, ignoring GC time, or all these other methods listed in this doc.)

Capricorn2481 2y ago

Aren't allocations themselves pretty expensive regardless of GC?

nu11ptr 2y ago
Go allocations aren't that bad. A few years ago I benchmarked them at about 4x as expensive as a bump allocation. That is slow enough to make an arena beneficial in high allocation situations, but fast enough to not make it worth it most of the time.
- aktau 2y ago
  Comparing with a fairly optimized malloc at $COMPANY, the Go allocator is (both in terms of relative cycles and fraction of cycles of all Go programs) significantly more expensive than the C/C++ counterpart (3-4x IIRC). For one, it has to do more work, like setting up GC metadata, and zeroing. There have recently been some optimizations to `runtime.mallocgc`, which may have decrease that 3-4x estimate a bit.
  - nu11ptr 2y ago
    How can that be true? If it is 3-4x more expensive than malloc, then per my measurements your malloc is a bump allocator, and that simply isn't true for any real world malloc implementation (typically a modified free list allocator afaik). `mallocgc` may not be fast, but I simply did not find it as slow as you are saying. My guess is it is about as fast as most decent malloc functions, but I have not measured, and it would be interesting to see a comparison (tough to do as you'd need to call malloc via CGo or write one in C and one in Go and trust the looping is roughly the same cost).
    - aktau 2y ago
      I should correct and clarify: I meant 3-4x more expensive in relative terms. Meaning: - For C++ programs, the allocator (allocating+freeing) consumes roughly 5% of cycles. - For Go programs, the allocator (runtime.mallocgc) used to consume ~20% of cycles (this is the data I referenced). I checked and recently it's become closer to 15%, thanks to optimizations. I have not tested the performance differential on a per-byte level (though that will also differ with object structure in Go).

raggi 2y ago

You might wanna look at a system profiler too, pprof doesn't show everything.

zmj 2y ago

Pretty similar story in .NET. Make sure your inner loops are allocation-free, then ensure allocations are short-lived, then clean up the long tail of large allocations.

neonsunset 2y ago
.NET is far more tolerant to high allocation traffic since its GC is generational and overall more sophisticated (even if at the cost of tail latency, although that is workload-dependent). Doing huge allocations which go to LOH is quite punishing, but even substantial inter-generational traffic won't kill it.

zbobet2012 2y ago

E.h., kind of. If you are allocating in a hot loop it's going to suck regardless. Object pools are really key if you want high perf because the general purpose allocator is way less efficient in comparison.

liquidgecka 2y ago

Its worth calling out that abstractions can kill you in unexpected ways with go. Anytime you use an interface it forces a heap allocation, even if the object is only used read only and within the same scope. That includes calls to things like fmt.Printf() so doing a for loop that prints the value of i forces the integer backing i to be heap allocated, along with every other value that you printed. So if you helpfully make every api in your library use an interface you are forcing the callers to use heap allocations for every single operation.

slashdev 2y ago
I thought surely an integer could be inlined into the interface, I thought Go used to do that. But I tried it on the playground, and it heap allocates it: https://go.dev/play/p/zHfnQfJ9OGc https://go.dev/play/p/zHfnQfJ9OGc
- masklinn 2y ago
  Go did use to do that, it was removed years ago, in 1.4: https://go.dev/doc/go1.4#runtime https://go.dev/doc/go1.4#runtime
  - kbolino 2y ago
    Basically, anything that isn't a thin pointer (*T, chan, map) gets boxed nowadays. The end result is that both words of an interface value are always pointers [1], which is very friendly to the garbage collector (setting aside the extra allocations when escape analysis fails). I've seen some tricks in the standard library to avoid boxing, e.g. how strings and times are handled by log/slog [2]. [1]: https://github.com/teh-cmc/go-internals/blob/master/chapter2_interfaces/README.md#overview-of-the-datastructures https://github.com/teh-cmc/go-internals/blob/master/chapter2... [2]: https://cs.opensource.google/go/go/+/refs/tags/go1.24.1:src/log/slog/value.go;l=21 https://cs.opensource.google/go/go/+/refs/tags/go1.24.1:src/...
    - deleted 2y ago
      [deleted]

ncruces 2y ago

The point is not to avoid GC entirely, but to reduce allocation pressure. If you can avoid allocs in a hot loop, it definitely pays to do so. If you can't for some reason, and can use sync.Pool there, measure it. Cutting allocs in half may not matter much, but if you can cut them by 99% because you were allocating in every iteration of a 1 million loop, and now aren't, it will make a difference, even if all those allocs die instantly. I've gotten better than two fold performance increases on real code with both techniques.

bboreham 2y ago

Agree that mark phase is the expensive bit. Disagree that it’s not worth reducing short-lived allocations. I spend a lot of time analyzing Go program performance, and reducing bytes allocated per second is always beneficial.

felixge 2y ago
+1. In particular []byte slice allocations are often a significant driver of GC pace while also being relatively easy to optimize (e.g. via sync.Pool reuse).

aktau 2y ago

Side note: see https://tip.golang.org/doc/gc-guide https://tip.golang.org/doc/gc-guide for more on how the Go GC works and what triggers it. GC frequency is directly driven by allocation rate (in terms of bytes) and live heap size. Some examples: - If you halve the allocation rate, you halve the GC frequency. - If you double the live heap size, you halve the GC frequency (barring changes away from the default `GOGC=100`). > ...but if you look at pprof of a Go app, GC mark phase is what takes time, not GC sweep. It is true that sweeping is a lot cheaper than marking, which makes your next statement: > Short lived allocations, those which GC mark phase will never reach, has almost neglible effect on GC times. ...technically correct. Usually, this is the best kind of correct, but it omits two important considerations: - If you generate a ton of short-lived allocations instead of keeping them around, the GC will trigger more frequently. - If you reduce the live heap size (by not keeping anything around), the GC will trigger more frequently. So now you have cheaper GC cycles, but many more of them. On top of that, you have vastly increased allocation costs. It is not a priori clear to me this is a win. In my experience, it isn't.

nopurpose 2y ago
I enjoyed your detailed response, it adds value to this discussion, but I feel you missed the point of my comment. I am against blanket statements "reduce allocations to reduce GC pressure", which lead people wrong way: they compare libraries based on "allocs/op" from go bench, they trust rediculous (who allocates 8KB per iteration in tight loop??) microbenchmarks of sync.Pool like in the article above, hoping to resolve their GC problem. Spend considerabe amount of effort just to find that they barely moved a needle on GC times. If we generalize then my "avoid long-lived allocations" or yours "reduce allocation rate in terms of bytes" are much more useful in practice, than what this and many other articles preach.
deepsun 2y ago
Interesting, thank you. But I think those points are not correlated that much. For example if I create unnecessary wrappers in a loop, I might double the allocation rate, but I will not halve the live heap size, because I did not have those wrappers outside the loop before. Basically, I'm trying to come up with an real world example of a style change (like create wrappers for every error, or use naked integers instead of time.Time) to estimate its impact. And my feeling is that any such example would affect one of your points way more than the other, so we can still argument that e.g. "creating short-lived iterators is totally fine".

deepsun 2y ago

Interesting, and I think that is not specific to Go, other mark-and-sweep GCs (Java, C#) should behave the same. Which means that creating short lived objects (like iterators for loops, or some wrappers) is ok.

int_19h 2y ago
It should be noted that in C#, at least, the standard pattern is to use value types for enumerators, precisely so as to avoid heap allocations. This is the case for all (non-obsolete) collections in the .NET stdlib - e.g. List<T>.Enumerator is a struct.
ted_dunning 2y ago
Not entirely. Go still doesn't have a generational collector so high allocation rates cause more GC's that must examine long-lived objects. As such, short-lived objects have little impact in Java (thank god for that!). They will have second order effects in Go.

kgeist 2y ago

The runtime also forces GC every 2 minutes. So yeah, a lot of long living allocations can stress the GC, even if you don't allocate often. That's why Discord moved from Go to Rust for their Read States server.