6 ms·

I have to admit that, as a C/C++ programmer, I find this entire thing a bit bizarre. In C and C++, of course a for loop induction variable is reused for each i

by amluto 4y ago

I have to admit that, as a C/C++ programmer, I find this entire thing a bit bizarre. In C and C++, of course a for loop induction variable is reused for each iteration. And if you take the address of the variable and keep it around for too long, you get to keep all the pieces and possibly the exploitable UAF vulnerability, too. (Never mind that, prior to C99, C induction variables were always in an enclosing scope, and this would be fairly obvious from reading the code.) In Rust, code like the examples wouldn’t compile. If you want to copy a value to the heap and take a reference, you need to say so. And it will be quite clear whether you are keeping a boxed copy or whether you are keeping a reference to the original object. (And the object can’t mutate out from under you unexpectedly in either case — a non-mutable reference prevents mutation!) So the fact that, in Go, one might rely on automatic promotion of an induction variable (which obviously gets mutated every iteration!) to the heap, and thus get confused about precisely which value is promoted to the heap, seems weird from my perspective. IMO one might reasonably argue that the problem is that this pattern works at all, not that it works in the way that people usually don’t want. edit: in C++, with range-based for, the induction variable is gone after each iteration. So taking a reference that outlives the iteration is invalid, and if you’re lucky, the compiler will warn. The confusing cases from the Go proposal are that invalid in C++, not confusing.

rcme 4y ago
How often do you access your loop variables asynchronously?
- amluto 4y ago
  Rarely, if ever. But aliasing and unexpected mutability don’t need asynchrony or threading to be a problem. For example, the entire problem leading to this proposal is about unexpected mutability. You have a variable i, which presently has a specific value. You take a reference to it, and you give that reference to some other code that saves it. Then you mutate i, and you are suddenly surprised that the external code now has a reference to the new value.
  - rcme 4y ago
    Yes, but the reference isn't explicitly created. Rather, the reference is implicitly captured by a Go routine, so this happens in Go much more often than other languages because Go routines are a core feature of the language.
    - amluto 4y ago
      I’m not sure what you mean. In the OP, the examples involve closures that capture an induction variable. I don’t think Goroutines per se are involved. (Although, again, in C++ you had better not capture a reference and then have the closure in question outlive the referent, and you need to literally type & to get a closure to capture by reference. In Rust, references created by closure follow the same rules as any other reference.) And the earlier proposal: https://github.com/golang/go/discussions/56010 https://github.com/golang/go/discussions/56010 gives an explicit reference (with an &) as its motivating example. So I still believe that the whole issue being addressed is bizarre from the perspective of a C, C++ or Rust programmer. (I’m having trouble thinking of another language that promotes a local variable to be a heap-allocated slot when one explicitly takes a reference to it. Most GC’d languages have local variables that are, themselves, references to the heap such that the referent (the heap slot / box / whatever you want to call it) will outlive the variable if the reference is copied. Pure languages may not strongly distinguish between a value and a reference to a value because the referent can’t change, so a copy is as good as the original. I think that Go is rather odd here, but I could easily be missing something.) (P.S. C++ also has extends the lifetime of temporaries if a const reference is taken, but only to the end of the scope. This is too magical for my taste. At least the heap isn’t involved.)
      - comex 4y ago
        It’s true that the variant of the issue involving explicit references seems pretty unique to Go. But there are plenty of languages that have the variant of the issue where a closure implicitly captures a local variable by reference. JavaScript used to have it until `let` was created (and still does have it if you don’t use `let`). And I ran into it recently in some closure-heavy Python code I wrote recently. (Python makes it relatively ugly to work around, too.)
uluyol 4y ago
In Go, you can take the address of a local variable and return it. It will be allocated on the heap automatically. This is weird for C++ programmers that think about the stack vs heap, but perfectly natural for Go. In Go, stack allocation for variables is a compiler optimization, not part of the language semantics. Go also only uses scope for visibility, not RAII.
- woodruffw 4y ago
  I think the weirdness there is not with returning a local variable (C++ allows you to do that too, via RVO and NRVO), but the idea that you'd return a local variable by address. That's fundamentally very weird from any systems background, and suggests a language design error: if people are returning pointers to "locals" because returning by value does an implicit copy, then maybe it shouldn't have been specified to do that.
  - uluyol 4y ago
    I don't agree and my job requires that I program in C++. It's different, yes, but it's only strange if you learn about C or C++ first. Whether you agree or not, I definitely think that promoting loop variables to be heap allocated is consistent with the rest of Go.
    - woodruffw 4y ago
      I don't disagree that it's consistent! Consistency and weirdness are separate dimensions; it's possible to be consistently weird, and IMO Go's pattern of returning local-looking-but-actually-heap addresses is pretty weird, especially when you consider that they could have made return-by-value a transparently optimized move instead.
      - xyzzy_plugh 4y ago
        It's only weird because you are expecting a local variable to not be on the heap. If local variables were always on the heap, you wouldn't notice. That's precisely what is happening here.
haileys 4y ago
It really is remarkable that Go has repeated all these mistakes, even with the benefit of having all of our field's history to learn from. Loop variable scoping is not the only area where the designers of Go have failed to learn from past experience and instead opted for a design that lends itself to a more convenient implementation at the expense of exposing foot guns to the user: zero values are absolute dynamite, especially when combined with other language features like implicit zero initialisation of structs, multiple return values instead of sum types, or reflection-based magical JSON deserialization. When you have an empty string or a zero integer in Go, you can never be quite sure that the value you're holding is supposed to be what it is, rather than being an implicit zero value that snuck in at some point in place of a missing value. Go also fails to offer any facilities to aid programmers in writing safe and correct concurrent code, beyond some fundamentally superficial language features like syntax extensions for channel types and spawning tasks. Channels are difficult to use correctly, usually having to be used in conjunction with some other synchronisation primitive like a WaitGroup, and compose poorly with implicit zero values resulting in the need to define semantics for fundamentally nonsense operations [1]. The language offers no facilities for restricting mutation, indeed in this respect, it's even worse than C. It's too easy to write buggy, racy code in Go. There's no denying Go's initial allure, but in reality it fails to address in any systemic way the problems that have plagued the software development practice for decades. While other languages offer facilities for eliminating entire classes of bugs, all Go has to offer is an attitude that says "simply don't make mistakes." The whole language is a bitterly disappointing pile of missed opportunities. In the words of fasterthanlime, it's the billion dollar mistake all over again. [1]: https://dave.cheney.net/2014/03/19/channel-axioms https://dave.cheney.net/2014/03/19/channel-axioms
- philosopher1234 4y ago
  This is a very one sided view. It reads like you have not familiarized yourself much with the internal reasoning behind these decisions. You can disagree with the balance that has been struck, but you don't appear to be aware of any balancing at all.
- zbobet2012 4y ago
  > There's no denying Go's initial allure, but in reality it fails to address in any systemic way the problems that have plagued the software development practice for decade I find comments like this illustrate perfectly the gap between these focused on language theoretics, and those focused on building software. Go is successful because it strikes a strong balance between what practical software engineering organizations need from a language, of which safety is only one constraint. Languages also need to be: * Easy "enough" to learn * Easy to read * Easy to write * Performant * Compile quickly * Have strong tooling * Have a strong standard library * Provide good solutions for it's problem domains * Support large code bases with many dependents ... * And be "safe* Easy here is also influenced by what people already know, syntax and basic working methods need to be familiar to the vast majority of software engineers. Go is successful because for many engineers it provides enough of all of the above with *enough" safety. P.s. complex, difficult to understand languages incur more bugs then simple ones, even with additional safety. PPS: If you think multiple returns and sumtypes solve the same thing, you've not thought very hard on the subject...
emmelaich 4y ago
Sorta related, I very surprised that Rust supports variable shadowing. But this blog article satisfied me: https://ntietz.com/blog/rust-shadowing-idiomatic/ https://ntietz.com/blog/rust-shadowing-idiomatic/
nradov 4y ago
After coding in higher level languages the notion of routinely writing for loops like I used to do in C now seems so backwards. Iteration in general should almost always be pushed down into language constructs or libraries.
masklinn 4y ago
> So the fact that, in Go, one might rely on automatic promotion of an induction variable (which obviously gets mutated every iteration!) to the heap, and thus get confused about precisely which value is promoted to the heap, seems weird from my perspective. Automatic promotion is not that big of a factor, fundamentally this is a common issue with closures over mutable bindings, similar issues can be observed in other langages. Most (but not all) have mitigations already because it’s such a common issue especially around loops e.g. C# breaking-changed the semantics of range loops in 5.0, javascript added “let” and “const” which have block and per-iteration scoping, … C, C++, and Rust don’t suffer from it (ish) because one doesn’t have closures and the other two require more explicit management of the capture and its lifetime. C++ could have something similar with capture by reference but the code would probably not be straightforward, and the more straightforward code would have much bigger issues of dangling references.

by amluto 4y ago

rcme 4y ago

How often do you access your loop variables asynchronously?

amluto 4y ago
Rarely, if ever. But aliasing and unexpected mutability don’t need asynchrony or threading to be a problem. For example, the entire problem leading to this proposal is about unexpected mutability. You have a variable i, which presently has a specific value. You take a reference to it, and you give that reference to some other code that saves it. Then you mutate i, and you are suddenly surprised that the external code now has a reference to the new value.
- rcme 4y ago
  Yes, but the reference isn't explicitly created. Rather, the reference is implicitly captured by a Go routine, so this happens in Go much more often than other languages because Go routines are a core feature of the language.
  - amluto 4y ago
    I’m not sure what you mean. In the OP, the examples involve closures that capture an induction variable. I don’t think Goroutines per se are involved. (Although, again, in C++ you had better not capture a reference and then have the closure in question outlive the referent, and you need to literally type & to get a closure to capture by reference. In Rust, references created by closure follow the same rules as any other reference.) And the earlier proposal: https://github.com/golang/go/discussions/56010 https://github.com/golang/go/discussions/56010 gives an explicit reference (with an &) as its motivating example. So I still believe that the whole issue being addressed is bizarre from the perspective of a C, C++ or Rust programmer. (I’m having trouble thinking of another language that promotes a local variable to be a heap-allocated slot when one explicitly takes a reference to it. Most GC’d languages have local variables that are, themselves, references to the heap such that the referent (the heap slot / box / whatever you want to call it) will outlive the variable if the reference is copied. Pure languages may not strongly distinguish between a value and a reference to a value because the referent can’t change, so a copy is as good as the original. I think that Go is rather odd here, but I could easily be missing something.) (P.S. C++ also has extends the lifetime of temporaries if a const reference is taken, but only to the end of the scope. This is too magical for my taste. At least the heap isn’t involved.)
    - comex 4y ago
      It’s true that the variant of the issue involving explicit references seems pretty unique to Go. But there are plenty of languages that have the variant of the issue where a closure implicitly captures a local variable by reference. JavaScript used to have it until `let` was created (and still does have it if you don’t use `let`). And I ran into it recently in some closure-heavy Python code I wrote recently. (Python makes it relatively ugly to work around, too.)

uluyol 4y ago

In Go, you can take the address of a local variable and return it. It will be allocated on the heap automatically. This is weird for C++ programmers that think about the stack vs heap, but perfectly natural for Go. In Go, stack allocation for variables is a compiler optimization, not part of the language semantics. Go also only uses scope for visibility, not RAII.

woodruffw 4y ago
I think the weirdness there is not with returning a local variable (C++ allows you to do that too, via RVO and NRVO), but the idea that you'd return a local variable by address. That's fundamentally very weird from any systems background, and suggests a language design error: if people are returning pointers to "locals" because returning by value does an implicit copy, then maybe it shouldn't have been specified to do that.
- uluyol 4y ago
  I don't agree and my job requires that I program in C++. It's different, yes, but it's only strange if you learn about C or C++ first. Whether you agree or not, I definitely think that promoting loop variables to be heap allocated is consistent with the rest of Go.
  - woodruffw 4y ago
    I don't disagree that it's consistent! Consistency and weirdness are separate dimensions; it's possible to be consistently weird, and IMO Go's pattern of returning local-looking-but-actually-heap addresses is pretty weird, especially when you consider that they could have made return-by-value a transparently optimized move instead.
    - xyzzy_plugh 4y ago
      It's only weird because you are expecting a local variable to not be on the heap. If local variables were always on the heap, you wouldn't notice. That's precisely what is happening here.

haileys 4y ago

It really is remarkable that Go has repeated all these mistakes, even with the benefit of having all of our field's history to learn from. Loop variable scoping is not the only area where the designers of Go have failed to learn from past experience and instead opted for a design that lends itself to a more convenient implementation at the expense of exposing foot guns to the user: zero values are absolute dynamite, especially when combined with other language features like implicit zero initialisation of structs, multiple return values instead of sum types, or reflection-based magical JSON deserialization. When you have an empty string or a zero integer in Go, you can never be quite sure that the value you're holding is supposed to be what it is, rather than being an implicit zero value that snuck in at some point in place of a missing value. Go also fails to offer any facilities to aid programmers in writing safe and correct concurrent code, beyond some fundamentally superficial language features like syntax extensions for channel types and spawning tasks. Channels are difficult to use correctly, usually having to be used in conjunction with some other synchronisation primitive like a WaitGroup, and compose poorly with implicit zero values resulting in the need to define semantics for fundamentally nonsense operations [1]. The language offers no facilities for restricting mutation, indeed in this respect, it's even worse than C. It's too easy to write buggy, racy code in Go. There's no denying Go's initial allure, but in reality it fails to address in any systemic way the problems that have plagued the software development practice for decades. While other languages offer facilities for eliminating entire classes of bugs, all Go has to offer is an attitude that says "simply don't make mistakes." The whole language is a bitterly disappointing pile of missed opportunities. In the words of fasterthanlime, it's the billion dollar mistake all over again. [1]: https://dave.cheney.net/2014/03/19/channel-axioms https://dave.cheney.net/2014/03/19/channel-axioms

philosopher1234 4y ago
This is a very one sided view. It reads like you have not familiarized yourself much with the internal reasoning behind these decisions. You can disagree with the balance that has been struck, but you don't appear to be aware of any balancing at all.
zbobet2012 4y ago
> There's no denying Go's initial allure, but in reality it fails to address in any systemic way the problems that have plagued the software development practice for decade I find comments like this illustrate perfectly the gap between these focused on language theoretics, and those focused on building software. Go is successful because it strikes a strong balance between what practical software engineering organizations need from a language, of which safety is only one constraint. Languages also need to be: * Easy "enough" to learn * Easy to read * Easy to write * Performant * Compile quickly * Have strong tooling * Have a strong standard library * Provide good solutions for it's problem domains * Support large code bases with many dependents ... * And be "safe* Easy here is also influenced by what people already know, syntax and basic working methods need to be familiar to the vast majority of software engineers. Go is successful because for many engineers it provides enough of all of the above with *enough" safety. P.s. complex, difficult to understand languages incur more bugs then simple ones, even with additional safety. PPS: If you think multiple returns and sumtypes solve the same thing, you've not thought very hard on the subject...

emmelaich 4y ago

Sorta related, I very surprised that Rust supports variable shadowing. But this blog article satisfied me: https://ntietz.com/blog/rust-shadowing-idiomatic/ https://ntietz.com/blog/rust-shadowing-idiomatic/

nradov 4y ago

After coding in higher level languages the notion of routinely writing for loops like I used to do in C now seems so backwards. Iteration in general should almost always be pushed down into language constructs or libraries.

masklinn 4y ago

> So the fact that, in Go, one might rely on automatic promotion of an induction variable (which obviously gets mutated every iteration!) to the heap, and thus get confused about precisely which value is promoted to the heap, seems weird from my perspective. Automatic promotion is not that big of a factor, fundamentally this is a common issue with closures over mutable bindings, similar issues can be observed in other langages. Most (but not all) have mitigations already because it’s such a common issue especially around loops e.g. C# breaking-changed the semantics of range loops in 5.0, javascript added “let” and “const” which have block and per-iteration scoping, … C, C++, and Rust don’t suffer from it (ish) because one doesn’t have closures and the other two require more explicit management of the capture and its lifetime. C++ could have something similar with capture by reference but the code would probably not be straightforward, and the more straightforward code would have much bigger issues of dangling references.