5 ms·

I can see why it's a surprise. Most languages that I know of fall into one of two types: (1) garbage collected and assigning a variable or passing to a function

I can see why it's a surprise. Most languages that I know of fall into one of two types: (1) garbage collected and assigning a variable or passing to a function actually passes a reference to the object (certainly true of Python, Java, C#); (2) memory is manually managed and you can take the address of an object (C, C++, Rust). That history makes it feel like "taking the address" is a really trivial operation - returning a numerical value that the compiler had access to at that point anyway. Here it's adding a reference to the object in some sense, and maybe even changing how it's allocated earlier in its lifetime (on the heap rather than the stack). I don't use Go and I agree that using &x for that operation feels a bit wrong as an outsider.

masklinn 4y ago
Fwiw the issue occurs in langages of category (1) though only in a subset of the cases. Generally closures as they implicitly take references on their lexical context. It also occurs in langages of category (2), specifically C++ lambdas where i think it can cause UAF/UB. I assume it also happens in C with the block extension (is that still Apple specific?) though I don’t know the details of that thing so maybe not.
- quietbritishjim 4y ago
  The discussion here is specifically about using & to extend the lifetime. In the first case you mention, you don't use the & operator. In the second case, you do (at least with C++ lambdas), but there's no lifetime extension going on.
  - foldr 4y ago
    There's no special 'lifetime extension' operation involved. Semantically, everything is heap allocated† in Go. The garbage collector takes care of deallocating it when nothing references it anymore. In other words, at the level of the language semantics, all tracking of lifetimes is done dynamically at runtime by the garbage collector. Go does in fact stack allocate variables which it can prove not to outlive their lexical scopes, but this is merely an optimization. Unless you are trying to write optimal code, there is never any reason to think about which values are stack allocated in Go. There's not really any such thing as a 'local variable' in Go. A variable has whatever scope it has, but there's nothing special, semantically speaking, about variables defined inside functions or inside loops. If the use of & in the example code is puzzling, it's probably because you're expecting Go to have some C-like concept of an automatic (i.e. stack allocated) variable – but it just doesn't. >That history makes it feel like "taking the address" is a really trivial operation - returning a numerical value that the compiler had access to at that point anyway. It is in fact a trivial operation in Go too, as I hope the above has clarified. --- † Strictly speaking 'semantically heap allocated' is nonsense, but hopefully you know what I mean. There is no way to declare a variable in Go in such a way as to force it to be deallocated at the end of a particular lexical scope. A variable's lexical scope and its lifetime are entirely divorced (as is typical in a GCed language).
    - quietbritishjim 4y ago
      > There's no special 'lifetime extension' operation involved. ... The garbage collector takes care of deallocating it when nothing references it anymore. I never used the word "special". As you say, adding a reference will mean the garbage collector won't deallocate it (until that reference is removed). In other words... its lifetime is extended. That's exactly what I meant. > If the use of & in the example code is puzzling, it's probably because you're expecting Go to have some C-like concept of an automatic (i.e. stack allocated) variable ... Not at all. In C++, you can use & on a reference variable and it will return the address of the object being referred to, regardless of whether it is allocated on the stack or the heap (or even statically allocated). Even in C, you can do &*x on a pointer to any object (which is silly by itself, but useful when combined with pointer arithmetic e.g. &x[3] translates to &*(x+3)). > It [the & operator in Go] is in fact a trivial operation in Go too, as I hope the above has clarified. Maybe I should have avoided the word "trivial" as its meaning is subjective, but I was careful to define what I meant by it: "returning a numerical value that the compiler had access to at that point anyway". Your comment just confirms that, as I said, it does more than that – it also adds a reference to the object. --- To be clear, I'm not saying that it's bad or wrong that Go uses the & operator to mean this. Once you're familiar with the language, you probably get used to it very quickly. My point was just that it's a surprise initially if you're not familiar with the language, that's all.
      - foldr 4y ago
        >Your comment just confirms that, as I said, it does more than that – it also adds a reference to the object. It simply evaluates to the address of the object, just as it does in C. if you think the & operator is doing something in addition to this, I think that must just be based on a misunderstanding. I am not quite sure what you mean by 'adding a reference' to the object. Let's take this function: func foo() *int { var x int return &x } All that happens is the following: - An integer is allocated (and initialized to zero). - The address of this integer is returned. If we dig into the implementation, we'll see that the integer is allocated on the heap. As far as Go's language semantics are concerned, everything is allocated on the heap and left to the GC to clean up. As an implementation detail, values that provably don't outlive their containing functions are (sometimes) stack allocated. As x outlives its containing function, it won't be stack allocated. That's it. There is no special operation of 'adding a reference' or 'extending a lifetime'. Nor does the compiler even analyze lifetimes except for the purposes of applying an optional optimisation which has no effect on the semantics of the program. If you turned this optimisation off (which you totally could) then there'd be no need for the compiler to worry about x's lifetime at all.
- vips7L 4y ago
  This cannot occur in Java. Any captured variables by lambdas must be effectively final.

masklinn 4y ago

Fwiw the issue occurs in langages of category (1) though only in a subset of the cases. Generally closures as they implicitly take references on their lexical context. It also occurs in langages of category (2), specifically C++ lambdas where i think it can cause UAF/UB. I assume it also happens in C with the block extension (is that still Apple specific?) though I don’t know the details of that thing so maybe not.

quietbritishjim 4y ago
The discussion here is specifically about using & to extend the lifetime. In the first case you mention, you don't use the & operator. In the second case, you do (at least with C++ lambdas), but there's no lifetime extension going on.
- foldr 4y ago
  There's no special 'lifetime extension' operation involved. Semantically, everything is heap allocated† in Go. The garbage collector takes care of deallocating it when nothing references it anymore. In other words, at the level of the language semantics, all tracking of lifetimes is done dynamically at runtime by the garbage collector. Go does in fact stack allocate variables which it can prove not to outlive their lexical scopes, but this is merely an optimization. Unless you are trying to write optimal code, there is never any reason to think about which values are stack allocated in Go. There's not really any such thing as a 'local variable' in Go. A variable has whatever scope it has, but there's nothing special, semantically speaking, about variables defined inside functions or inside loops. If the use of & in the example code is puzzling, it's probably because you're expecting Go to have some C-like concept of an automatic (i.e. stack allocated) variable – but it just doesn't. >That history makes it feel like "taking the address" is a really trivial operation - returning a numerical value that the compiler had access to at that point anyway. It is in fact a trivial operation in Go too, as I hope the above has clarified. --- † Strictly speaking 'semantically heap allocated' is nonsense, but hopefully you know what I mean. There is no way to declare a variable in Go in such a way as to force it to be deallocated at the end of a particular lexical scope. A variable's lexical scope and its lifetime are entirely divorced (as is typical in a GCed language).
  - quietbritishjim 4y ago
    > There's no special 'lifetime extension' operation involved. ... The garbage collector takes care of deallocating it when nothing references it anymore. I never used the word "special". As you say, adding a reference will mean the garbage collector won't deallocate it (until that reference is removed). In other words... its lifetime is extended. That's exactly what I meant. > If the use of & in the example code is puzzling, it's probably because you're expecting Go to have some C-like concept of an automatic (i.e. stack allocated) variable ... Not at all. In C++, you can use & on a reference variable and it will return the address of the object being referred to, regardless of whether it is allocated on the stack or the heap (or even statically allocated). Even in C, you can do &*x on a pointer to any object (which is silly by itself, but useful when combined with pointer arithmetic e.g. &x[3] translates to &*(x+3)). > It [the & operator in Go] is in fact a trivial operation in Go too, as I hope the above has clarified. Maybe I should have avoided the word "trivial" as its meaning is subjective, but I was careful to define what I meant by it: "returning a numerical value that the compiler had access to at that point anyway". Your comment just confirms that, as I said, it does more than that – it also adds a reference to the object. --- To be clear, I'm not saying that it's bad or wrong that Go uses the & operator to mean this. Once you're familiar with the language, you probably get used to it very quickly. My point was just that it's a surprise initially if you're not familiar with the language, that's all.
    - foldr 4y ago
      >Your comment just confirms that, as I said, it does more than that – it also adds a reference to the object. It simply evaluates to the address of the object, just as it does in C. if you think the & operator is doing something in addition to this, I think that must just be based on a misunderstanding. I am not quite sure what you mean by 'adding a reference' to the object. Let's take this function: func foo() *int { var x int return &x } All that happens is the following: - An integer is allocated (and initialized to zero). - The address of this integer is returned. If we dig into the implementation, we'll see that the integer is allocated on the heap. As far as Go's language semantics are concerned, everything is allocated on the heap and left to the GC to clean up. As an implementation detail, values that provably don't outlive their containing functions are (sometimes) stack allocated. As x outlives its containing function, it won't be stack allocated. That's it. There is no special operation of 'adding a reference' or 'extending a lifetime'. Nor does the compiler even analyze lifetimes except for the purposes of applying an optional optimisation which has no effect on the semantics of the program. If you turned this optimisation off (which you totally could) then there'd be no need for the compiler to worry about x's lifetime at all.
vips7L 4y ago
This cannot occur in Java. Any captured variables by lambdas must be effectively final.