24 ms·

Go's escape analysis and why my function return worked

42 points by bonniesimon 10mo ago

deleted 10mo ago
[deleted]
foldr 10mo ago
This seems to be a persistent source of confusion. Escape analysis is just an optimization. You don't need to think about it to understand why your Go code behaves the way it does. Just imagine that everything is allocated on the heap and you won't have any surprises.
- bonniesimon 10mo ago
  Makes sense. I need to rewire how I think about Go. I should see it how I see JS.
- Yokohiii 10mo ago
  I am currently learning go and your comment made me sort some things out, but probably in a counterintuitive way. Assuming to everything allocates on the heap, will solve this specific confusion. My understanding is that C will let you crash quite fast if the stack becomes too large, go will dynamically grow the stack as needed. So it's possible to think you're working on the heap, but you are actually threshing the runtime with expensive stack grow calls. Go certainly tries to be smart about it with various strategies, but a rapid stack grow rate will have it's cost.
  - foldr 10mo ago
    Go won’t put large allocations on the stack even if escape analysis would permit it, so generally speaking this should only be a concern if you have very deep recursion (in which case you might have to worry about stack overflows anyway).
    - Yokohiii 10mo ago
      Escape analysis accounts for size, so it wouldn't even permit it. The initial stack size seems to be 2kb, a more on a few systems. So far I understand you can allocate a large local i.e. 8kb, that doesn't escape and grow the stack immediately. (Of course that adds up if you have a chain of calls with smaller allocs). So recursion is certainly not the only concern.
      - foldr 10mo ago
        For that to be a problem you either have to have one function that allocates an enormous number of non-escaping objects below the size limit (if the Go compiler doesn't take the total size of all a function's non-escaping allocations into account – I don't know), or a very long series of nested function calls, which in practice is only likely to arise if there are recursive calls.
        Yokohiii 10mo ago
        I think we mix things up here. But be aware of my newbie knowledge. I am pretty sure the escape analysis doesn't affect the initial stack size. Escape analysis does determine where an allocation lives. So if your allocation is lower then what escape analysis considers heap and bigger then the initial stack size, the stack needs to grow. What I am certain about, is that I have runtime.newstack calls accounting for +20% of my benchmark times (go testing). My code is quite shallow (3-4 calls deep) and anything of size should be on the heap (global/preallocated) and the code has zero allocations. I don't use goroutines either, it might me I still make a mistake or it's the overhead from the testing benchmark. But this obviously doesn't seem to be anything super unusual.
        foldr 10mo ago
        I don't know about your code, but in general, goroutine stacks are designed to start small and grow. There is nothing concerning about this. A call to runtime.newstack triggered by a large stack-allocated value would generally be cheaper than the corresponding heap allocation.
- 9rx 10mo ago
  > This seems to be a persistent source of confusion. Why? It is the same as in C. #include <stdio.h> #include <stdlib.h> struct slice { int *data; size_t len; size_t cap; }; struct slice readLogsFromPartition() { int *data = malloc(2); data[0] = 1; data[1] = 2; return (struct slice){ data, 2, 2 }; } int main() { struct slice s = readLogsFromPartition(); for (int i = 0; i < s.len; i++) { printf("%d\n", s.data[i]); } free(s.data); }
  - simiones 10mo ago
    The point the GP was making was that the following Go snippet: func foo() { x := []int { 1 } //SNIP } Could translate to C either as: void foo() { int* x = malloc(1 * sizeof(int)); x[0] = 1; //... } Or as void foo() { int data[1] = {1}; int *x = data; //... } Depending on the content of //SNIP. However, some people think that the semantics can also match the semantics of the second version in C - when in fact the semantics of the Go code always match the first version, even when the actual implementation is the second version.
    - 9rx 10mo ago
      The semantics are clearly defined as being the same as the C code I posted earlier. Why would one try to complicate the situation by thinking that it would somehow magically change sometimes?
      - simiones 10mo ago
        Because people hear that Go supports value types and so is more efficient than Java because it can allocate on the stack*, and so they start thinking that they need to manage the stack. * Of course, in reality, Java also does escape analysis to allocate on the stack, though it's less likely to happen because of the lack of value types.
        9rx 10mo ago
nasretdinov 10mo ago
If the functions get inlined (which they might if they're small enough), then the code won't even need to allocate on heap! That's a kind of optimisation that's not really possible without transparent escape analysis.
samdoesnothing 10mo ago
Go is returning a copy of the slice, in the same way that C would return a copy of an int or struct if you returned it. The danger of C behaviour in this instance is that a stack allocated array decays into a pointer which points to the deallocated memory. Otherwise the behaviour is pretty similar between the languages.
- debugnik 10mo ago
  I first wrote an answer about how local variables can survive through a pointer, but deleted it because you're right that this Go code doesn't even address locals. It's a regular value copy.
deleted 10mo ago
[deleted]
potato-peeler 10mo ago
If the variable was defined in the calling function itself, and a pointer was passed, I guess the variable will still be in the heap?
- Yokohiii 10mo ago
  Pointers escape to the heap by default.
jstanley 10mo ago
It's not confusing that this works in Go. (In my opinion). A straightforward reading of the code suggests that it should do what it does. The confusion here is a property of C, not of Go. It's a property of C that you need to care about the difference between the stack and the heap, it's not a general fact about programming. I don't think Go is doing anything confusing.
- throwaway894345 10mo ago
  I like Go a lot, but I often wish we could be more explicit about where allocations are. It’s often important for writing performant code, but instead of having semantics we have to check against the stack analyzer which has poor ergonomics and may break at any time. But yeah, to your point, returning a slice in a GC language is not some exotic thing.
  - Yokohiii 10mo ago
    Can you elaborate on the stack analyzer? All I could figure out was to see runtime.morestack calls that affected the runtime, but as far I remember the caller timings did exclude the cost. Having a clearer view of stack grow rates would be really great.
    - throwaway894345 10mo ago
      I’m not sure what you mean? Are you asking for information about what it is or how to use it?
      - Yokohiii 10mo ago
        I never heard of "stack analyzer" and didn't get meaningful results for it, do you mean escape analysis?
        throwaway894345 10mo ago
        Sorry, yes, I meant “escape analyzer”. I’ve been jet lagged.
knorker 10mo ago
Are you sure this is what's happening? Looks to me like the slice object is returned by value, and the array was always on the heap. See https://go.dev/play/p/Bez0BgRny7G https://go.dev/play/p/Bez0BgRny7G (the address of the slice object changed, so it's not the same object on the heap) Sure, Go has escape analysis, but is that really what's happening here? Isn't this a better example of escape analysis: https://go.dev/play/p/qX4aWnnwQV2 https://go.dev/play/p/qX4aWnnwQV2 (the object retains its address, always on the heap, in both caller and callee)
- bonniesimon 10mo ago
  Interesting! This could be true. I'll play around with this in a bit.
  - knorker 10mo ago
    Yeah I think what you're describing, returning a slice thus copying a reference to the same array (but not copying the array), then destroying the callee slice not causing the array to be freed, is just basic garbage collection logic, not escape analysis.
- masklinn 10mo ago
  That’s the one. Since 1.17 it’s not impossible for escape analysis to come into play for slices but afaik that is only a consideration for slices with a statically known size under 64KiB.
- simiones 10mo ago
  Depending on escape analysis, the array underlying the slice can get allocated on the stack as well, if it doesn't escape the function context. Of course, in this case, because we are returning a pointer to it via the slice, that optimization isn't applicable.
  - knorker 10mo ago
    Agreed that it could in principle. But I can't immediately get it to do so: https://go.dev/play/p/9hLHattS8cf https://go.dev/play/p/9hLHattS8cf Both arrays in this example seem to be on the heap.
tdfirth 10mo ago
I don’t think this is confusing to the vast majority of people writing Go. In my experience, the average programmer isn’t even aware of the stack vs heap distinction these days. If you learned to write code in something like Python then coming at Go from “above” this will just work the way you expect. If you come at Go from “below” then yeah it’s a bit weird.
- onionisafruit 10mo ago
  Go has been my primary language for a few years now, and I’ve had to do extra work to make sure I’m avoiding the heap maybe five times. Stack and heap aren’t on my mind most of the time when designing and writing Go, even though I have a pretty good understanding of how it works. The same applies to the garbage collector. It just doesn’t matter most of the time. That said, when it matters it matters a lot. In those times I wish it was more visible in Go code, but I would want it to not get in the way the rest of the time. But I’m ok with the status quo of hunting down my notes on escape analysis every few months and taking a few minutes to get reacquainted. Side note: I love how you used “from above” and “from below”. It makes me feel angelic as somebody who came from above; even if Java and Ruby hardly seemed like heaven.
  - carb 10mo ago
    Why have you had to avoid the heap? Performance concerns?
    - malkia 10mo ago
      For me, avoiding heap, or rather avoiding gc came when I was working (at work) on backend and web server using Java, and there was default rule for our code that if gc takes more than 1% (I don't remember the exact value) then the server gets restarted. Coming (back then) from C/C++ gamedev - I was puzzled, then I understood the mantra - it's better for the process to die fast, instead of being pegged by GC and not answering to the client. Then we started looking what made it use GC so much. I guess it might be similar to Go - in the past I've seen some projects using a "baloon" - to circumvent Go's GC heuristic - e.g. if you blow this dummy baloon that takes half of your memory GC might not kick so much... Something like this... Then again obviously bad solution long term
gethly 10mo ago
> In C, you can't assign a value in a local function and then return it I am so glad I never taken up C. This sound like a nightmare of a DX to me.
- kjeetgill 10mo ago
  Depending on what your working on, it's actually super nice to know very clearly what lives on the stack vs the heap for performance and compactness reasons. Basically anything that didn't come from malloc or a function calling malloc lives on the stack and doesn't live past the function it was allocated in. And these days, if you're bothering with C you probably care about these things. Accidentally promoting from the stack to the heap would be annoying.
matthewaveryusa 10mo ago
Nope, this analysis is wrong. Decompile your code and look at what's going on: https://godbolt.org/z/f1nx9ffYK https://godbolt.org/z/f1nx9ffYK The thing being returned is a slice (a fat pointer) that has pointer, length, capacity. In the code linked you'll see the fat pointer being returned from the function as values. in C you'd get just AX (the pointer, without length and cap) command-line-arguments_readLogsFromPartition_pc122: MOVQ BX, AX // slice.ptr -> AX (first result register) MOVQ SI, BX // slice.len -> BX (second) MOVQ DX, CX // slice.cap -> CX (third) The gargabe collection is happening in the FUNCDATA/PCDATA annotations, but I don't really know how that works.
mwsherman 10mo ago
Shameless plug, if one wishes to track down allocations in Go, an allocations explorer for VS Code: https://marketplace.visualstudio.com/items?itemName=Clipperhouse.go-allocations-vsix https://marketplace.visualstudio.com/items?itemName=Clipperh...
- tgv 10mo ago
  That looks nice. Going to give it a try.
jasonthorsness 10mo ago
You can run the compiler with a flag that shows all the escapes with -gcflags “-m” and there’s also support in goland and vscode to show the escapes as inline annotations in the editor. This sort of thing IMO is one of the useful things about IDEs: showing hints from later parts of the tool chain about how things are going to turn out
metadat 10mo ago
Emojis in code comments make them unreadable. Why is this a thing?

42 points by bonniesimon 10mo ago

deleted 10mo ago

[deleted]

foldr 10mo ago

This seems to be a persistent source of confusion. Escape analysis is just an optimization. You don't need to think about it to understand why your Go code behaves the way it does. Just imagine that everything is allocated on the heap and you won't have any surprises.

bonniesimon 10mo ago
Makes sense. I need to rewire how I think about Go. I should see it how I see JS.
Yokohiii 10mo ago
I am currently learning go and your comment made me sort some things out, but probably in a counterintuitive way. Assuming to everything allocates on the heap, will solve this specific confusion. My understanding is that C will let you crash quite fast if the stack becomes too large, go will dynamically grow the stack as needed. So it's possible to think you're working on the heap, but you are actually threshing the runtime with expensive stack grow calls. Go certainly tries to be smart about it with various strategies, but a rapid stack grow rate will have it's cost.
- foldr 10mo ago
  Go won’t put large allocations on the stack even if escape analysis would permit it, so generally speaking this should only be a concern if you have very deep recursion (in which case you might have to worry about stack overflows anyway).
  - Yokohiii 10mo ago
    Escape analysis accounts for size, so it wouldn't even permit it. The initial stack size seems to be 2kb, a more on a few systems. So far I understand you can allocate a large local i.e. 8kb, that doesn't escape and grow the stack immediately. (Of course that adds up if you have a chain of calls with smaller allocs). So recursion is certainly not the only concern.
    - foldr 10mo ago
      For that to be a problem you either have to have one function that allocates an enormous number of non-escaping objects below the size limit (if the Go compiler doesn't take the total size of all a function's non-escaping allocations into account – I don't know), or a very long series of nested function calls, which in practice is only likely to arise if there are recursive calls.
      - Yokohiii 10mo ago
        I think we mix things up here. But be aware of my newbie knowledge. I am pretty sure the escape analysis doesn't affect the initial stack size. Escape analysis does determine where an allocation lives. So if your allocation is lower then what escape analysis considers heap and bigger then the initial stack size, the stack needs to grow. What I am certain about, is that I have runtime.newstack calls accounting for +20% of my benchmark times (go testing). My code is quite shallow (3-4 calls deep) and anything of size should be on the heap (global/preallocated) and the code has zero allocations. I don't use goroutines either, it might me I still make a mistake or it's the overhead from the testing benchmark. But this obviously doesn't seem to be anything super unusual.
        foldr 10mo ago
        I don't know about your code, but in general, goroutine stacks are designed to start small and grow. There is nothing concerning about this. A call to runtime.newstack triggered by a large stack-allocated value would generally be cheaper than the corresponding heap allocation.
9rx 10mo ago
> This seems to be a persistent source of confusion. Why? It is the same as in C. #include <stdio.h> #include <stdlib.h> struct slice { int *data; size_t len; size_t cap; }; struct slice readLogsFromPartition() { int *data = malloc(2); data[0] = 1; data[1] = 2; return (struct slice){ data, 2, 2 }; } int main() { struct slice s = readLogsFromPartition(); for (int i = 0; i < s.len; i++) { printf("%d\n", s.data[i]); } free(s.data); }
- simiones 10mo ago
  The point the GP was making was that the following Go snippet: func foo() { x := []int { 1 } //SNIP } Could translate to C either as: void foo() { int* x = malloc(1 * sizeof(int)); x[0] = 1; //... } Or as void foo() { int data[1] = {1}; int *x = data; //... } Depending on the content of //SNIP. However, some people think that the semantics can also match the semantics of the second version in C - when in fact the semantics of the Go code always match the first version, even when the actual implementation is the second version.
  - 9rx 10mo ago
    The semantics are clearly defined as being the same as the C code I posted earlier. Why would one try to complicate the situation by thinking that it would somehow magically change sometimes?
    - simiones 10mo ago
      Because people hear that Go supports value types and so is more efficient than Java because it can allocate on the stack*, and so they start thinking that they need to manage the stack. * Of course, in reality, Java also does escape analysis to allocate on the stack, though it's less likely to happen because of the lack of value types.
      - 9rx 10mo ago

nasretdinov 10mo ago

If the functions get inlined (which they might if they're small enough), then the code won't even need to allocate on heap! That's a kind of optimisation that's not really possible without transparent escape analysis.

samdoesnothing 10mo ago

Go is returning a copy of the slice, in the same way that C would return a copy of an int or struct if you returned it. The danger of C behaviour in this instance is that a stack allocated array decays into a pointer which points to the deallocated memory. Otherwise the behaviour is pretty similar between the languages.

debugnik 10mo ago
I first wrote an answer about how local variables can survive through a pointer, but deleted it because you're right that this Go code doesn't even address locals. It's a regular value copy.

deleted 10mo ago

[deleted]

potato-peeler 10mo ago

If the variable was defined in the calling function itself, and a pointer was passed, I guess the variable will still be in the heap?

Yokohiii 10mo ago
Pointers escape to the heap by default.

jstanley 10mo ago

It's not confusing that this works in Go. (In my opinion). A straightforward reading of the code suggests that it should do what it does. The confusion here is a property of C, not of Go. It's a property of C that you need to care about the difference between the stack and the heap, it's not a general fact about programming. I don't think Go is doing anything confusing.

throwaway894345 10mo ago
I like Go a lot, but I often wish we could be more explicit about where allocations are. It’s often important for writing performant code, but instead of having semantics we have to check against the stack analyzer which has poor ergonomics and may break at any time. But yeah, to your point, returning a slice in a GC language is not some exotic thing.
- Yokohiii 10mo ago
  Can you elaborate on the stack analyzer? All I could figure out was to see runtime.morestack calls that affected the runtime, but as far I remember the caller timings did exclude the cost. Having a clearer view of stack grow rates would be really great.
  - throwaway894345 10mo ago
    I’m not sure what you mean? Are you asking for information about what it is or how to use it?
    - Yokohiii 10mo ago
      I never heard of "stack analyzer" and didn't get meaningful results for it, do you mean escape analysis?
      - throwaway894345 10mo ago
        Sorry, yes, I meant “escape analyzer”. I’ve been jet lagged.

knorker 10mo ago

Are you sure this is what's happening? Looks to me like the slice object is returned by value, and the array was always on the heap. See https://go.dev/play/p/Bez0BgRny7G https://go.dev/play/p/Bez0BgRny7G (the address of the slice object changed, so it's not the same object on the heap) Sure, Go has escape analysis, but is that really what's happening here? Isn't this a better example of escape analysis: https://go.dev/play/p/qX4aWnnwQV2 https://go.dev/play/p/qX4aWnnwQV2 (the object retains its address, always on the heap, in both caller and callee)

bonniesimon 10mo ago
Interesting! This could be true. I'll play around with this in a bit.
- knorker 10mo ago
  Yeah I think what you're describing, returning a slice thus copying a reference to the same array (but not copying the array), then destroying the callee slice not causing the array to be freed, is just basic garbage collection logic, not escape analysis.
masklinn 10mo ago
That’s the one. Since 1.17 it’s not impossible for escape analysis to come into play for slices but afaik that is only a consideration for slices with a statically known size under 64KiB.
simiones 10mo ago
Depending on escape analysis, the array underlying the slice can get allocated on the stack as well, if it doesn't escape the function context. Of course, in this case, because we are returning a pointer to it via the slice, that optimization isn't applicable.
- knorker 10mo ago
  Agreed that it could in principle. But I can't immediately get it to do so: https://go.dev/play/p/9hLHattS8cf https://go.dev/play/p/9hLHattS8cf Both arrays in this example seem to be on the heap.

tdfirth 10mo ago

I don’t think this is confusing to the vast majority of people writing Go. In my experience, the average programmer isn’t even aware of the stack vs heap distinction these days. If you learned to write code in something like Python then coming at Go from “above” this will just work the way you expect. If you come at Go from “below” then yeah it’s a bit weird.

onionisafruit 10mo ago
Go has been my primary language for a few years now, and I’ve had to do extra work to make sure I’m avoiding the heap maybe five times. Stack and heap aren’t on my mind most of the time when designing and writing Go, even though I have a pretty good understanding of how it works. The same applies to the garbage collector. It just doesn’t matter most of the time. That said, when it matters it matters a lot. In those times I wish it was more visible in Go code, but I would want it to not get in the way the rest of the time. But I’m ok with the status quo of hunting down my notes on escape analysis every few months and taking a few minutes to get reacquainted. Side note: I love how you used “from above” and “from below”. It makes me feel angelic as somebody who came from above; even if Java and Ruby hardly seemed like heaven.
- carb 10mo ago
  Why have you had to avoid the heap? Performance concerns?
  - malkia 10mo ago
    For me, avoiding heap, or rather avoiding gc came when I was working (at work) on backend and web server using Java, and there was default rule for our code that if gc takes more than 1% (I don't remember the exact value) then the server gets restarted. Coming (back then) from C/C++ gamedev - I was puzzled, then I understood the mantra - it's better for the process to die fast, instead of being pegged by GC and not answering to the client. Then we started looking what made it use GC so much. I guess it might be similar to Go - in the past I've seen some projects using a "baloon" - to circumvent Go's GC heuristic - e.g. if you blow this dummy baloon that takes half of your memory GC might not kick so much... Something like this... Then again obviously bad solution long term

gethly 10mo ago

> In C, you can't assign a value in a local function and then return it I am so glad I never taken up C. This sound like a nightmare of a DX to me.

kjeetgill 10mo ago
Depending on what your working on, it's actually super nice to know very clearly what lives on the stack vs the heap for performance and compactness reasons. Basically anything that didn't come from malloc or a function calling malloc lives on the stack and doesn't live past the function it was allocated in. And these days, if you're bothering with C you probably care about these things. Accidentally promoting from the stack to the heap would be annoying.

matthewaveryusa 10mo ago

Nope, this analysis is wrong. Decompile your code and look at what's going on: https://godbolt.org/z/f1nx9ffYK https://godbolt.org/z/f1nx9ffYK The thing being returned is a slice (a fat pointer) that has pointer, length, capacity. In the code linked you'll see the fat pointer being returned from the function as values. in C you'd get just AX (the pointer, without length and cap) command-line-arguments_readLogsFromPartition_pc122: MOVQ BX, AX // slice.ptr -> AX (first result register) MOVQ SI, BX // slice.len -> BX (second) MOVQ DX, CX // slice.cap -> CX (third) The gargabe collection is happening in the FUNCDATA/PCDATA annotations, but I don't really know how that works.

mwsherman 10mo ago

Shameless plug, if one wishes to track down allocations in Go, an allocations explorer for VS Code: https://marketplace.visualstudio.com/items?itemName=Clipperhouse.go-allocations-vsix https://marketplace.visualstudio.com/items?itemName=Clipperh...

tgv 10mo ago
That looks nice. Going to give it a try.

jasonthorsness 10mo ago

You can run the compiler with a flag that shows all the escapes with -gcflags “-m” and there’s also support in goland and vscode to show the escapes as inline annotations in the editor. This sort of thing IMO is one of the useful things about IDEs: showing hints from later parts of the tool chain about how things are going to turn out

metadat 10mo ago

Emojis in code comments make them unreadable. Why is this a thing?