6 ms·

It was a bit of a bummer when go switched from the conservative gc to a precise gc. One of the implications was that they needed to change how interface types

by chrsig 2y ago

It was a bit of a bummer when go switched from the conservative gc to a precise gc. One of the implications was that they needed to change how interface types were represented. They had a nice little optimization for word-sized values to store in-place rather than as a pointer out to a value. With the precise gc, they had to make the change to only storing pointers, leading to allocating small values. I don't know if they've done work to (or perhaps better put: had success) regain the performance hit from the extra allocation & gc load. On the flip side, my experience is that they've made the pretty unobtrusive with regards to latency and pauses. Or perhaps I'm just not stressing it as much as I had in the past. Random anecdote on gc tuning: I was once dealing with a go process that sped up (higher throughput, lower latency) by an alarming amount limiting the max processors. This was many years ago, and I wouldn't be able to say what version of go. It was after you no longer had to set GOMAXPROCS, but probably not very long after. Performance tuning is crazy sometimes.

znpy 2y ago
Performance tuning is still largely a dark art from what I see, having dabbled in the space. It’s both weird and beautiful because getting an end-to-end understanding is instrumental, so you often have to go look at many marginal things that might actually play a significant role.
- bqmjjx0kac 2y ago
  Disappointingly, it's a dark art often because the CPU is a black box. Intel X86 chips translate the instructions you give them to some internal microcode and then execute them speculatively, out of order, etc. I'm still mystified by the performance gains afforded by randomly inserting NOP instructions.
  - pjmlp 2y ago
    That is why tooling like VTune exist.
  - deleted 2y ago
    [deleted]
  - 10000truths 2y ago
    There is a plethora of information regarding instruction timings, throughout/latency, execution port usage, etc. that compilers make liberal use of for optimization purposes. You could, in theory, also use that information to establish an upper bound on how long a series of instructions would take to execute. The problem lies in the difference in magnitude between average case and worst case, due to dynamic execution state like CPU cache, branch prediction, kernel scheduling, and so on.
    - Mathnerd314 2y ago
      There is uiCA, it achieves an error of about 1% relative to actual measurements of basic block throughput across a wide range of microarchitectures. And then FACILE, similar to uiCA. I don't know of any compilers using these more accurate models, but it is certainly possible.
  - cogman10 2y ago
    Fortunately, at least in my experience, the variability that CPUs introduce (which do matter in many contexts) aren't often the source of slowness. In my experience plain old algorithmic complexity would go a long way in making stuff faster. I can't tell you the number of times I've fixed code like this matches = [] for (first : items) { for (second : items) { if (first.name == second.name) matches.add(first); } } Very frequently a bright red spot in most profiler output.
    - marginalia_nu 2y ago
      I think there's an element of selection bias in that observation. Since that is the type of performance issue that a profiler is good at finding, those are the performance issues you'll find looking at a profiler.
      - cogman10 2y ago
        > I think there's an element of selection bias in that observation. Almost certainly true, I can only speak of my own experiences. > Since that is the type of performance issue that a profiler is good at finding, those are the performance issues you'll find looking at a profiler. I have to disagree with you on this. Sampling profilers are good at finding out exactly what methods are eating performance. In fact, if anything they have a tendency to push you towards looking at single methods for problems rather than moving up a layer of two to see the big picture (it's why flame graphs are so important in profiling). I have, for example, seen plenty of times where the profiler indicated that double math was the root cause of problems yet popping a few layers up the stack revealed (sometimes non-obviously) that there was n^2 behavior going on.
  - Conscat 2y ago
    Intel also provides vtune to annotate sequenced instructions and profile the microseconds and power consumption down to the level of individual instructions. I assume those NOPs you mention exist for alignment padding. Clang and GCC let you configure the alignment padding of any or all functions, and Clang lets you explicitly align any for-loop anywhere you want with `[[clang::code_align]]`.
soegaard 2y ago
> Do you have more details (or a reference)?
- chrsig 2y ago
  It took me a bit to hunt down, but it was part of the go 1.4 release in 2014. the release notes[0] at the time stated > The implementation of interface values has been modified. In earlier releases, the interface contained a word that was either a pointer or a one-word scalar value, depending on the type of the concrete object stored. This implementation was problematical for the garbage collector, so as of 1.4 interface values always hold a pointer. In running programs, most interface values were pointers anyway, so the effect is minimal, but programs that store integers (for example) in interfaces will see more allocations. @rsc wrote in some detail[0] about it the initial layout on his blog. I couldn't find a detailed explanation about how the optimization interacted w/ the gc, but my understanding is that it couldn't discern between pointer and integer values [0] https://go.dev/doc/go1.4#runtime https://go.dev/doc/go1.4#runtime [1] https://research.swtch.com/interfaces https://research.swtch.com/interfaces
  - kbolino 2y ago
    I can't imagine it was impossible to keep the inline-value optimization; after all, the full type information is always right there in the other word of the interface value, but I think it would have been too complicated/expensive for too little benefit. The bigger issue IMO is and has been that fat pointers (strings, slices) can't be inlined in interface values. There's definitely some benefit to inlining integers and floats, but the indirection that comes from not inlining them isn't that significant compared to the double-indirection that comes from not inlining strings and slices. There was some discussion on the mailing list IIRC about expanding the interface to 3 words wide (to hold strings at least) or 4 words wide (to hold slices too), but this was rejected as going too far the other way (at the time). Interestingly, the log/slog package does inline string values at least [1], and demonstrates how such a thing can be done when needed, albeit with a fair deal of complexity. [1]: https://cs.opensource.google/go/go/+/refs/tags/go1.23.1:src/log/slog/value.go;l=21 https://cs.opensource.google/go/go/+/refs/tags/go1.23.1:src/...
hinkley 2y ago
The problem with precise GC is usually the same problem with malloc/free - if you allocate in an inner loop you have to free in that inner loop and the bookkeeping kills throughput. I don’t know Go. Is that the problem we are seeing here? One of the realtime GC solutions that stuck with me is amortized GC, which might be appropriate to Go. Instead of moving dead objects immediately to the free list you “just” need to stay ahead of allocation. You can accomplish that by freeing memory every time you allocate memory - but not a full GC, just finishing the free of a handful of dead objects. That puts an upper bound on allocation time without a lower bound on reallocation time.
- sltkr 2y ago
  That's not what precise GC means. Precise means that on a GC cycle, only true pointers to heap allocated objects are identified as roots, which means that an unreferenced object cannot be kept alive by a value that happens to have a similar bit pattern as a heap pointer. This guarantees that unreferenced objects are eventually cleaned up, but not that this cleanup happens immediately, certainly not as part of a busy loop. (Unless the system runs out of memory, but in that case, a GC iteration is required anyway.)
  - vlovich123 2y ago
    Since reference counting is GC, isn't reference counting a form of precise GC? That's certainly the scenario I had in mind reading what OP wrote where deallocation within a hot loop would be possible.
    - hinkley 2y ago
      No it’s usually called conservative unless you have loop detection.
  - hinkley 2y ago
    Oof. We used to just call that tracing.
    - 2y ago
silisili 2y ago
It's always important to know your bottlenecks before tuning. If you're disk io bound, throwing 100 goroutines at it just compounds the problem, for example. One very common thing I've noticed is that people new to the language overuse/abuse goroutines and channels. I completely get why, they are two selling points you learn about early on. But it's really easy to make things go wrong when spamming them everywhere.
matheusmoreira 2y ago
> With the precise gc, they had to make the change to only storing pointers, leading to allocating small values. Why? Many dynamic languages use a tagged pointer representation which isn't incompatible with precise garbage collectors. Couldn't Go do the same?
- skybrian 2y ago
  Perhaps because Go has pointers to the middle of records and arrays? (And records of arrays, etc.) It's an unusual language feature.
  - kbolino 2y ago
    This, plus Go pointers are real memory addresses, a property which is guaranteed by the runtime for unsafe pointer operations and C interop.
- deleted 2y ago
  [deleted]

by chrsig 2y ago

znpy 2y ago

Performance tuning is still largely a dark art from what I see, having dabbled in the space. It’s both weird and beautiful because getting an end-to-end understanding is instrumental, so you often have to go look at many marginal things that might actually play a significant role.

bqmjjx0kac 2y ago
Disappointingly, it's a dark art often because the CPU is a black box. Intel X86 chips translate the instructions you give them to some internal microcode and then execute them speculatively, out of order, etc. I'm still mystified by the performance gains afforded by randomly inserting NOP instructions.
- pjmlp 2y ago
  That is why tooling like VTune exist.
- deleted 2y ago
  [deleted]
- 10000truths 2y ago
  There is a plethora of information regarding instruction timings, throughout/latency, execution port usage, etc. that compilers make liberal use of for optimization purposes. You could, in theory, also use that information to establish an upper bound on how long a series of instructions would take to execute. The problem lies in the difference in magnitude between average case and worst case, due to dynamic execution state like CPU cache, branch prediction, kernel scheduling, and so on.
  - Mathnerd314 2y ago
    There is uiCA, it achieves an error of about 1% relative to actual measurements of basic block throughput across a wide range of microarchitectures. And then FACILE, similar to uiCA. I don't know of any compilers using these more accurate models, but it is certainly possible.
- cogman10 2y ago
  Fortunately, at least in my experience, the variability that CPUs introduce (which do matter in many contexts) aren't often the source of slowness. In my experience plain old algorithmic complexity would go a long way in making stuff faster. I can't tell you the number of times I've fixed code like this matches = [] for (first : items) { for (second : items) { if (first.name == second.name) matches.add(first); } } Very frequently a bright red spot in most profiler output.
  - marginalia_nu 2y ago
    I think there's an element of selection bias in that observation. Since that is the type of performance issue that a profiler is good at finding, those are the performance issues you'll find looking at a profiler.
    - cogman10 2y ago
      > I think there's an element of selection bias in that observation. Almost certainly true, I can only speak of my own experiences. > Since that is the type of performance issue that a profiler is good at finding, those are the performance issues you'll find looking at a profiler. I have to disagree with you on this. Sampling profilers are good at finding out exactly what methods are eating performance. In fact, if anything they have a tendency to push you towards looking at single methods for problems rather than moving up a layer of two to see the big picture (it's why flame graphs are so important in profiling). I have, for example, seen plenty of times where the profiler indicated that double math was the root cause of problems yet popping a few layers up the stack revealed (sometimes non-obviously) that there was n^2 behavior going on.
- Conscat 2y ago
  Intel also provides vtune to annotate sequenced instructions and profile the microseconds and power consumption down to the level of individual instructions. I assume those NOPs you mention exist for alignment padding. Clang and GCC let you configure the alignment padding of any or all functions, and Clang lets you explicitly align any for-loop anywhere you want with `[[clang::code_align]]`.

soegaard 2y ago

> Do you have more details (or a reference)?

chrsig 2y ago
It took me a bit to hunt down, but it was part of the go 1.4 release in 2014. the release notes[0] at the time stated > The implementation of interface values has been modified. In earlier releases, the interface contained a word that was either a pointer or a one-word scalar value, depending on the type of the concrete object stored. This implementation was problematical for the garbage collector, so as of 1.4 interface values always hold a pointer. In running programs, most interface values were pointers anyway, so the effect is minimal, but programs that store integers (for example) in interfaces will see more allocations. @rsc wrote in some detail[0] about it the initial layout on his blog. I couldn't find a detailed explanation about how the optimization interacted w/ the gc, but my understanding is that it couldn't discern between pointer and integer values [0] https://go.dev/doc/go1.4#runtime https://go.dev/doc/go1.4#runtime [1] https://research.swtch.com/interfaces https://research.swtch.com/interfaces
- kbolino 2y ago
  I can't imagine it was impossible to keep the inline-value optimization; after all, the full type information is always right there in the other word of the interface value, but I think it would have been too complicated/expensive for too little benefit. The bigger issue IMO is and has been that fat pointers (strings, slices) can't be inlined in interface values. There's definitely some benefit to inlining integers and floats, but the indirection that comes from not inlining them isn't that significant compared to the double-indirection that comes from not inlining strings and slices. There was some discussion on the mailing list IIRC about expanding the interface to 3 words wide (to hold strings at least) or 4 words wide (to hold slices too), but this was rejected as going too far the other way (at the time). Interestingly, the log/slog package does inline string values at least [1], and demonstrates how such a thing can be done when needed, albeit with a fair deal of complexity. [1]: https://cs.opensource.google/go/go/+/refs/tags/go1.23.1:src/log/slog/value.go;l=21 https://cs.opensource.google/go/go/+/refs/tags/go1.23.1:src/...

hinkley 2y ago

The problem with precise GC is usually the same problem with malloc/free - if you allocate in an inner loop you have to free in that inner loop and the bookkeeping kills throughput. I don’t know Go. Is that the problem we are seeing here? One of the realtime GC solutions that stuck with me is amortized GC, which might be appropriate to Go. Instead of moving dead objects immediately to the free list you “just” need to stay ahead of allocation. You can accomplish that by freeing memory every time you allocate memory - but not a full GC, just finishing the free of a handful of dead objects. That puts an upper bound on allocation time without a lower bound on reallocation time.

sltkr 2y ago
That's not what precise GC means. Precise means that on a GC cycle, only true pointers to heap allocated objects are identified as roots, which means that an unreferenced object cannot be kept alive by a value that happens to have a similar bit pattern as a heap pointer. This guarantees that unreferenced objects are eventually cleaned up, but not that this cleanup happens immediately, certainly not as part of a busy loop. (Unless the system runs out of memory, but in that case, a GC iteration is required anyway.)
- vlovich123 2y ago
  Since reference counting is GC, isn't reference counting a form of precise GC? That's certainly the scenario I had in mind reading what OP wrote where deallocation within a hot loop would be possible.
  - hinkley 2y ago
    No it’s usually called conservative unless you have loop detection.
- hinkley 2y ago
  Oof. We used to just call that tracing.
  - 2y ago

silisili 2y ago

It's always important to know your bottlenecks before tuning. If you're disk io bound, throwing 100 goroutines at it just compounds the problem, for example. One very common thing I've noticed is that people new to the language overuse/abuse goroutines and channels. I completely get why, they are two selling points you learn about early on. But it's really easy to make things go wrong when spamming them everywhere.

matheusmoreira 2y ago

> With the precise gc, they had to make the change to only storing pointers, leading to allocating small values. Why? Many dynamic languages use a tagged pointer representation which isn't incompatible with precise garbage collectors. Couldn't Go do the same?

skybrian 2y ago
Perhaps because Go has pointers to the middle of records and arrays? (And records of arrays, etc.) It's an unusual language feature.
- kbolino 2y ago
  This, plus Go pointers are real memory addresses, a property which is guaranteed by the runtime for unsafe pointer operations and C interop.
deleted 2y ago
[deleted]