5 ms·

Re: https://github.com/golang/go/wiki/LoopvarExperiment https://github.com/golang/go/wiki/LoopvarExperiment I've mainly used Java which doesn't suffer from thi

Re: https://github.com/golang/go/wiki/LoopvarExperiment https://github.com/golang/go/wiki/LoopvarExperiment I've mainly used Java which doesn't suffer from this particular problem. But I've often heard Java criticized on the grounds it captures values not variables, and that most other languages apparently capture variables. I remember learning Lisp at uni and being taught that it captured variables, for example. So do other languages suffer from this problem as well? How do they deal with it?

kazinator 3y ago
In some cases, Common Lisp makes it implementation-defined. E.g. in (dolist (item list) ...) it is implementation-defined whether x is a freshly bound lexical, or a stepped variable. In Lisp, you can easily write macros that have whatever semantics you want. You can easily write a my-dolist which is like dolist but freshly binds the item variable for every iteration, and next to it write a my-dolist2 which assigns a single variable. In languages like Go, the providers of your language provide all the syntax from behind a wall, and make these decisions for you. Knights in shining armor duke it out on the rooftop of an ivory tower, while lower vassals blog incessantly about the important work being done and decisions being made. By the way, MacCarthy's ancient Lisp (Lisp 1, Lisp 1.5) didn't have lexical scope, so it would not have made any difference whether each iteration stepped the variable with setq or bound it with let. There would just be the one dynamic variable in all cases, not captured by any lexical closures (those being nonexistent). You'd want to bind it at least once with let so the prior value is restored when the loop is done. In Common Lisp, if we use a special variable as the loop variable, it likewise won't make a difference whether it is stepped or bound each time, since the binding isn't lexical.
- germandiago 3y ago
  Macros is exactly what makes Lisp a one-man only language for every person unless you are really careful. That, combined with the fact that things are often poorly documented is what IMHO ruins the language ecosystem. I want to like Lisp and I love its interactivity but I find the language a bit more "construction material" language than a ready-to-use immediately language. What is bad about that is exactly what you mention: everyone will come up with a set of macros that others cannot fully grasp.
  - kazinator 3y ago
    A poorly documented and tested libraries will give you problems no matter what kinds of entities it defines. Someone who writes macros that nobody can easily grasp will also write functions nobody can easily grasp. If you have to reverse engineer someone's macros, you have all of the following going for you The macros run at compile time and then go away (unless the program is based on a dynamic compiling paradigm). They have no behaviors at run-time; their generated code does. No matter how complicated the macro, you can just run it and capture the output code, and through multiple examples get an understanding of what it's doing. If someone writes a buggered function, you may have to end up debugging it on a target system, perhaps an embedded one. The sky is the limit there. If it's a function in a kernel, you may have to debug some race condition between it, some interrupt handler and a piece of hardware. No such thing ever happens when debugging a macro itself. A macro doesn't interact with complex application state, because that doesn't exist. You never have to attach some gigabyte database and get some objects into the right state before reproducing some expansion problem in a macro. Code generated by a macro could be involved in a problem like that, possibly as a root cause, but tracing through the macro itself isn't. The people who write awful macros are usually not smart enough to do anything that is actually hard to understand. The problems you will find are lack of hygiene, multiple evaluations and such.
    - germandiago 3y ago
      > A poorly documented and tested libraries will give you problems no matter what kinds of entities it defines. Someone who writes macros that nobody can easily grasp will also write functions nobody can easily grasp Lisp makes this so easy to write macros and they have been promoted to be the "my-powerful-dsl-is-the-right-way-to-solve-a-problem" that, IMHO, it works actively against team work unless it is really well-dcumented. Now, mix that with the fact that code is not usually deeply documented and you get a cultural problem into the whole ecosystem. > A macro doesn't interact with complex application state, because that doesn't exist. You never have to attach some gigabyte database and get some objects into the right state before reproducing some expansion problem in a macro. Code generated by a macro could be involved in a problem like that, possibly as a root cause, but tracing through the macro itself isn't Tracing the macro can be as difficult as its expansion. If it gets obtuse it will add a lot of time to your workflow. I know they are powerful but it is probably the last feature I would use in a team, with a lot of care, and for trivial things or for really justified cases and with good documentation. If your code is full of macros, forget about readability, you threw it out through the window. > The people who write awful macros are usually not smart enough to do anything that is actually hard to understand. The problems you will find are lack of hygiene, multiple evaluations and such Because writing macros is not that simple in the first place. I find much easier to write regular code and use the well-known macros than grasp anonymous macro code. There are things you can only do with macros though, like lazy evaluation. But macros must be managed with extra care. For example, maybe instead of a macro it is better to use a higher order function with a closure to keep the code understandable than burying things in layers of macros.
assbuttbuttass 3y ago
In fact most languages suffer from this, but it's more acute in go because of how easy it is to create closures and threads. For example, what does the following Python code print? funcs = list() for i in range(10): funcs.append(lambda: print(i)) for f in funcs: f()
- sharno 3y ago
  Interesting, didn't think about closures in Python before. It seems that JS has the same behavior. Java forces you to final copy the variable.
  - agumonkey 3y ago
    I lost track of ES scoping rules but apparently `for (var i ...) { ... }` doesn't capture (legacy behavior) while `for (let i ...) { ... }` will.
    - mdaniel 3y ago
      I don't think that's limited to just `var`, as a `let` outside of the `for` will do the same let funcs = []; ;(() => { let i; for (i = 0; i < 10; i++) { funcs.push(function() { console.log('i=',i); }) } })() for (let f of funcs) { f.call(null); } (I used an IIFE to hide the "let i" from the .call to ensure it wasn't late binding to the name like the python example did)
      - agumonkey 3y ago
        you think they have a special rule in `for (let .. of .. ) ... ` then ?
        masklinn 3y ago
        The “special rule” is that for(let) and for(const) create a new binding for each iteration, which mitigates the issue.
masklinn 3y ago
Note that there are two separate components to the issue: 1. the interaction between mutable locations and closures In most languages with mutable bindings, if you close over a variable you're closing over a "cell" and will see future modifications to it. This is the case in Go, Python, Ruby, Javascript, C#, ... A few languages avoid it e.g. - Java, not because it "captures values" but because it only allows capturing "final" variables so you can't update the bindings (you can still mutate in place when capturing reference types) - C++, because you have to specify the capture mode and it'll only be affected if you capture by reference (which you wouldn't do for a loop variable), so the likelihood of hitting this issue is pretty low - Rust, because even if you capture by reference you can't modify a binding with an outstanding reference 2. the scoping of loop variables, if you're capturing the loop variable but each iteration has a different version of the loop variable then you don't really mind too much, because each closure will capture its own loop variable, this doesn't fix the above issue but it does fix the most common occurrence of it That's what Go is doing, other languages which have done that are C# (when using a `foreach` loop), Javascript (when using `let` or `const` for the loop variable), ... Also Ruby, kind-of: using the `each` method for iteration is very common, and since then the loop variable is a parameter to the block it's basically a per-iteration local, "for...in" still has the issue. I guess C# also has that when using the linq ForEach method but I don't know how common that is.
- munificent 3y ago
  Probably worth pointing out that C# used to have Go's behavior and they changed it (a rare breaking language change) specifically because it was such a notorious pitfall. I think Go is doing the right thing here. I'm somewhat surprised they didn't always do this. (Dart has created a new loop variable for each iteration all the way back since before 1.0.)
  - masklinn 3y ago
    Similarly in javascript, “let” and “const” we’re added specifically with block and per-iteration scoping, because the ever increasing popularity of callbacks for async was making the function-scoped var more infuriating every day.

kazinator 3y ago

In some cases, Common Lisp makes it implementation-defined. E.g. in (dolist (item list) ...) it is implementation-defined whether x is a freshly bound lexical, or a stepped variable. In Lisp, you can easily write macros that have whatever semantics you want. You can easily write a my-dolist which is like dolist but freshly binds the item variable for every iteration, and next to it write a my-dolist2 which assigns a single variable. In languages like Go, the providers of your language provide all the syntax from behind a wall, and make these decisions for you. Knights in shining armor duke it out on the rooftop of an ivory tower, while lower vassals blog incessantly about the important work being done and decisions being made. By the way, MacCarthy's ancient Lisp (Lisp 1, Lisp 1.5) didn't have lexical scope, so it would not have made any difference whether each iteration stepped the variable with setq or bound it with let. There would just be the one dynamic variable in all cases, not captured by any lexical closures (those being nonexistent). You'd want to bind it at least once with let so the prior value is restored when the loop is done. In Common Lisp, if we use a special variable as the loop variable, it likewise won't make a difference whether it is stepped or bound each time, since the binding isn't lexical.

germandiago 3y ago
Macros is exactly what makes Lisp a one-man only language for every person unless you are really careful. That, combined with the fact that things are often poorly documented is what IMHO ruins the language ecosystem. I want to like Lisp and I love its interactivity but I find the language a bit more "construction material" language than a ready-to-use immediately language. What is bad about that is exactly what you mention: everyone will come up with a set of macros that others cannot fully grasp.
- kazinator 3y ago
  A poorly documented and tested libraries will give you problems no matter what kinds of entities it defines. Someone who writes macros that nobody can easily grasp will also write functions nobody can easily grasp. If you have to reverse engineer someone's macros, you have all of the following going for you The macros run at compile time and then go away (unless the program is based on a dynamic compiling paradigm). They have no behaviors at run-time; their generated code does. No matter how complicated the macro, you can just run it and capture the output code, and through multiple examples get an understanding of what it's doing. If someone writes a buggered function, you may have to end up debugging it on a target system, perhaps an embedded one. The sky is the limit there. If it's a function in a kernel, you may have to debug some race condition between it, some interrupt handler and a piece of hardware. No such thing ever happens when debugging a macro itself. A macro doesn't interact with complex application state, because that doesn't exist. You never have to attach some gigabyte database and get some objects into the right state before reproducing some expansion problem in a macro. Code generated by a macro could be involved in a problem like that, possibly as a root cause, but tracing through the macro itself isn't. The people who write awful macros are usually not smart enough to do anything that is actually hard to understand. The problems you will find are lack of hygiene, multiple evaluations and such.
  - germandiago 3y ago
    > A poorly documented and tested libraries will give you problems no matter what kinds of entities it defines. Someone who writes macros that nobody can easily grasp will also write functions nobody can easily grasp Lisp makes this so easy to write macros and they have been promoted to be the "my-powerful-dsl-is-the-right-way-to-solve-a-problem" that, IMHO, it works actively against team work unless it is really well-dcumented. Now, mix that with the fact that code is not usually deeply documented and you get a cultural problem into the whole ecosystem. > A macro doesn't interact with complex application state, because that doesn't exist. You never have to attach some gigabyte database and get some objects into the right state before reproducing some expansion problem in a macro. Code generated by a macro could be involved in a problem like that, possibly as a root cause, but tracing through the macro itself isn't Tracing the macro can be as difficult as its expansion. If it gets obtuse it will add a lot of time to your workflow. I know they are powerful but it is probably the last feature I would use in a team, with a lot of care, and for trivial things or for really justified cases and with good documentation. If your code is full of macros, forget about readability, you threw it out through the window. > The people who write awful macros are usually not smart enough to do anything that is actually hard to understand. The problems you will find are lack of hygiene, multiple evaluations and such Because writing macros is not that simple in the first place. I find much easier to write regular code and use the well-known macros than grasp anonymous macro code. There are things you can only do with macros though, like lazy evaluation. But macros must be managed with extra care. For example, maybe instead of a macro it is better to use a higher order function with a closure to keep the code understandable than burying things in layers of macros.

assbuttbuttass 3y ago

In fact most languages suffer from this, but it's more acute in go because of how easy it is to create closures and threads. For example, what does the following Python code print? funcs = list() for i in range(10): funcs.append(lambda: print(i)) for f in funcs: f()

sharno 3y ago
Interesting, didn't think about closures in Python before. It seems that JS has the same behavior. Java forces you to final copy the variable.
- agumonkey 3y ago
  I lost track of ES scoping rules but apparently `for (var i ...) { ... }` doesn't capture (legacy behavior) while `for (let i ...) { ... }` will.
  - mdaniel 3y ago
    I don't think that's limited to just `var`, as a `let` outside of the `for` will do the same let funcs = []; ;(() => { let i; for (i = 0; i < 10; i++) { funcs.push(function() { console.log('i=',i); }) } })() for (let f of funcs) { f.call(null); } (I used an IIFE to hide the "let i" from the .call to ensure it wasn't late binding to the name like the python example did)
    - agumonkey 3y ago
      you think they have a special rule in `for (let .. of .. ) ... ` then ?
      - masklinn 3y ago
        The “special rule” is that for(let) and for(const) create a new binding for each iteration, which mitigates the issue.

masklinn 3y ago

Note that there are two separate components to the issue: 1. the interaction between mutable locations and closures In most languages with mutable bindings, if you close over a variable you're closing over a "cell" and will see future modifications to it. This is the case in Go, Python, Ruby, Javascript, C#, ... A few languages avoid it e.g. - Java, not because it "captures values" but because it only allows capturing "final" variables so you can't update the bindings (you can still mutate in place when capturing reference types) - C++, because you have to specify the capture mode and it'll only be affected if you capture by reference (which you wouldn't do for a loop variable), so the likelihood of hitting this issue is pretty low - Rust, because even if you capture by reference you can't modify a binding with an outstanding reference 2. the scoping of loop variables, if you're capturing the loop variable but each iteration has a different version of the loop variable then you don't really mind too much, because each closure will capture its own loop variable, this doesn't fix the above issue but it does fix the most common occurrence of it That's what Go is doing, other languages which have done that are C# (when using a `foreach` loop), Javascript (when using `let` or `const` for the loop variable), ... Also Ruby, kind-of: using the `each` method for iteration is very common, and since then the loop variable is a parameter to the block it's basically a per-iteration local, "for...in" still has the issue. I guess C# also has that when using the linq ForEach method but I don't know how common that is.

munificent 3y ago
Probably worth pointing out that C# used to have Go's behavior and they changed it (a rare breaking language change) specifically because it was such a notorious pitfall. I think Go is doing the right thing here. I'm somewhat surprised they didn't always do this. (Dart has created a new loop variable for each iteration all the way back since before 1.0.)
- masklinn 3y ago
  Similarly in javascript, “let” and “const” we’re added specifically with block and per-iteration scoping, because the ever increasing popularity of callbacks for async was making the function-scoped var more infuriating every day.