5 ms·

I like the deep dive into the source, but big picture, I think if you understand the general order of evaluation which applies to most languages, this isn't all

I like the deep dive into the source, but big picture, I think if you understand the general order of evaluation which applies to most languages, this isn't all that unintuitive. The arguments going into the outer [ ] get evaluated before the outer [ ], meaning the inner [] gets evaluated first. Once that happens the inner list is created, and there's no code left that even visually could create more inner lists. The more unintuitive example of this to me is in function defaults, because those feel more like part of the function signature--i.e. I'd expect them to be assigned at function call time. But apparently, function defaults get called once when the function is defined. >>> def grow(items=[0]): ... items.append(items[-1] + 1) ... return items ... >>> grow() [0, 1] >>> grow() [0, 1, 2] >>> grow() [0, 1, 2, 3]

Joker_vD 3y ago
Well, those things are called "default argument values", not "default argument expressions", you know. It's a value that's reused for every function call, not an expression that re-evaluated at every function call.
- throwaway0665 3y ago
  Sure there is a reason it is that way. But it doesn't mean it's not incredibly unintuitive and error-prone.
  - Joker_vD 3y ago
    Yes, the reason is that it's a tradition to have it this way. Python IMHO has decent enough assignment semantics that using "default argument expressions" model would've been fine, unlike e.g. in C++.
- chii 3y ago
  if you compare > a = [0] and > b = [0] these two expressions produces different 'value' objects for `[0]`, and modifying `a.append(a[-1] + 1)` does not change b. so it's unintuitive that using `[0]` as a default value in a function argument can cause that argument to retain the same value across multiple function calls, when programmers would assume the argument to a function is not the same across multiple invocations. The issue really here is the usage of an array object, which is not a value object (immutable), but a reference object. It would not have been a problem if the default argument was a string, or a number. This is a reason not to use arrays in default arguments.
  - Joker_vD 3y ago
    Very technically speaking, in "def f(a=[0]): ..." or in "a = [[0]] * 4" there is only one textual occurrence of "[0]", and the function definition is literally evaluated to define a function (you can try def "f(a=sys.stdout.write('Hi!')): return" in REPL to see when the default value gets constructed). On the other hand, if we abstract from the technical minutiae, yes, it would have been entirely possible for Python to treat default arguments in the function definitions as expressions, by using machinery similar to what gets used for comprehensions: after all, as you point out, function's arguments (and local variables) do get re-assigned on every function call.
- planede 3y ago
  I think calling it a default argument object would be much more clear. "Value" can still refer to an abstract, immutable concept, even the python docs use it like that at some places.
- kerkeslager 3y ago
  > Well, those things are called "default argument values", not "default argument expressions", you know. No, I didn't know that, and I'd guess that the vast majority of Python developers don't, because people don't generally learn Python by reading the docs front to back and finding out the official names of everything. > It's a value that's reused for every function call, not an expression that re-evaluated at every function call. No one is confused about that. I'm aware that's how it is, but is that how it should be? I.e. if we could choose for this to be reused for every function call or re-evaluated every function call, which results in more intuitive behavior?
njharman 3y ago
I've never understood peoples confusion about function defaults. They are part of function definition, they aren't in the function body. You should expect them to be "executed" when the function is defined.
- Scarblac 3y ago
  Or they could have been part of the function call, and run when the function is called without that argument.
  - klyrs 3y ago
    You want Python to be how much slower now?
    - Scarblac 3y ago
      This isn't about what I want, just noting that the other way around could sound just as intuitive. Anyway, it's awfully slow already, how much can it hurt ;-)
    - kerkeslager 3y ago
      This is a terrible excuse. Quickly doing the wrong thing isn't optimization.
      - klyrs 3y ago
        The language spec isn't wrong, you just don't like it.
        kerkeslager 3y ago
        The language spec isn't right you just like it. Sure, there are a lot of subjective aesthetics that go into the spec, but in this case, there are objective reasons for not liking this. It's a well-known footgun that causes bugs. And it's almost never what you want, so you end up doing something like this: def f(xs = None): # Are these two lines actually faster than the # interpreter creating defaults at call time? if xs is None: xs = [] ... Do you have any reasons at all for defending this decision?
bmitc 3y ago
You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket and then gives you a piece of paper that lists its address four times. Why does one want that, by default? In more sane languages, the expression [] means give me an empty list value, nothing more and nothing less. So then whatever the syntax for [[]]*4 is in such sane languages, it gives you four empty list values. If you want something more imperative, you can get it, but it most cases, you don't want or even need such things. Python makes no distinction, at a level able to be visualized by the user, between immutable and mutable data. There are a lot of bizarre situations in Python that will bite you like this. And in most cases, it's strange that such behaviors were chosen as the deault.
- Joker_vD 3y ago
  I really wonder what "more sane" languages you mean. Because a single [] does give you an empty list value in Python: one list, and not four different empty lists. Why would it give your four? That'd need to somehow bring in the surrounding context and in "sane" languages, (closed) expressions generally don't change the meaning because the context around them is changing. Actually, if you really want to have re-evaluting (sub)expressions, then Python does give you an option: comprehensions.
  - bmitc 3y ago
    F#: List.init 4 (fun _ -> []) [for _ in 1..4 -> []] Or any sane language that uses immutable values by default like Elixir, Erlang, Clojure, Racket, Scheme, OCaml, etc. > Actually, if you really want to have re-evaluting (sub)expressions, then Python does give you an option: comprehensions. That's the complaint, that [[]]*4 is not shorthand for list comprehensions and that it's something entirely different. This is common in Python where two things you'd expect to be the same are indeed different in subtle ways. List comprehensions in Python are closer to sane, but they still have unexpected behavior because of the way scoping works in Python.
layer8 3y ago
The * operator could be defined to use a deepcopy operation, but isn’t, presumably for efficiency reasons. This has little to do with order of evaluation. Alternatively, the issue is that Python does not distinguish between value and reference types.
abhi9u 3y ago
Yes, the default function argument values is another surprising behavior. In this case, the default arguments get compiled as part of the function code and are part of the function's environment. As a result every time that function is called it is always having the same environment, i.e. the default argument values are not reinterpreted on every invocation. So the mutations to the environment are visible across function calls.
- kerkeslager 3y ago
  > In this case, the default arguments get compiled as part of the function code and are part of the function's environment. Yeah, I understand that; I'm saying that was a choice they made, and I think having the defaults be evaluated at each call time would have been a better choice.

Joker_vD 3y ago

Well, those things are called "default argument values", not "default argument expressions", you know. It's a value that's reused for every function call, not an expression that re-evaluated at every function call.

throwaway0665 3y ago
Sure there is a reason it is that way. But it doesn't mean it's not incredibly unintuitive and error-prone.
- Joker_vD 3y ago
  Yes, the reason is that it's a tradition to have it this way. Python IMHO has decent enough assignment semantics that using "default argument expressions" model would've been fine, unlike e.g. in C++.
chii 3y ago
if you compare > a = [0] and > b = [0] these two expressions produces different 'value' objects for `[0]`, and modifying `a.append(a[-1] + 1)` does not change b. so it's unintuitive that using `[0]` as a default value in a function argument can cause that argument to retain the same value across multiple function calls, when programmers would assume the argument to a function is not the same across multiple invocations. The issue really here is the usage of an array object, which is not a value object (immutable), but a reference object. It would not have been a problem if the default argument was a string, or a number. This is a reason not to use arrays in default arguments.
- Joker_vD 3y ago
  Very technically speaking, in "def f(a=[0]): ..." or in "a = [[0]] * 4" there is only one textual occurrence of "[0]", and the function definition is literally evaluated to define a function (you can try def "f(a=sys.stdout.write('Hi!')): return" in REPL to see when the default value gets constructed). On the other hand, if we abstract from the technical minutiae, yes, it would have been entirely possible for Python to treat default arguments in the function definitions as expressions, by using machinery similar to what gets used for comprehensions: after all, as you point out, function's arguments (and local variables) do get re-assigned on every function call.
planede 3y ago
I think calling it a default argument object would be much more clear. "Value" can still refer to an abstract, immutable concept, even the python docs use it like that at some places.
kerkeslager 3y ago
> Well, those things are called "default argument values", not "default argument expressions", you know. No, I didn't know that, and I'd guess that the vast majority of Python developers don't, because people don't generally learn Python by reading the docs front to back and finding out the official names of everything. > It's a value that's reused for every function call, not an expression that re-evaluated at every function call. No one is confused about that. I'm aware that's how it is, but is that how it should be? I.e. if we could choose for this to be reused for every function call or re-evaluated every function call, which results in more intuitive behavior?

njharman 3y ago

I've never understood peoples confusion about function defaults. They are part of function definition, they aren't in the function body. You should expect them to be "executed" when the function is defined.

Scarblac 3y ago
Or they could have been part of the function call, and run when the function is called without that argument.
- klyrs 3y ago
  You want Python to be how much slower now?
  - Scarblac 3y ago
    This isn't about what I want, just noting that the other way around could sound just as intuitive. Anyway, it's awfully slow already, how much can it hurt ;-)
  - kerkeslager 3y ago
    This is a terrible excuse. Quickly doing the wrong thing isn't optimization.
    - klyrs 3y ago
      The language spec isn't wrong, you just don't like it.
      - kerkeslager 3y ago
        The language spec isn't right you just like it. Sure, there are a lot of subjective aesthetics that go into the spec, but in this case, there are objective reasons for not liking this. It's a well-known footgun that causes bugs. And it's almost never what you want, so you end up doing something like this: def f(xs = None): # Are these two lines actually faster than the # interpreter creating defaults at call time? if xs is None: xs = [] ... Do you have any reasons at all for defending this decision?

bmitc 3y ago

You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket and then gives you a piece of paper that lists its address four times. Why does one want that, by default? In more sane languages, the expression [] means give me an empty list value, nothing more and nothing less. So then whatever the syntax for [[]]*4 is in such sane languages, it gives you four empty list values. If you want something more imperative, you can get it, but it most cases, you don't want or even need such things. Python makes no distinction, at a level able to be visualized by the user, between immutable and mutable data. There are a lot of bizarre situations in Python that will bite you like this. And in most cases, it's strange that such behaviors were chosen as the deault.

Joker_vD 3y ago
I really wonder what "more sane" languages you mean. Because a single [] does give you an empty list value in Python: one list, and not four different empty lists. Why would it give your four? That'd need to somehow bring in the surrounding context and in "sane" languages, (closed) expressions generally don't change the meaning because the context around them is changing. Actually, if you really want to have re-evaluting (sub)expressions, then Python does give you an option: comprehensions.
- bmitc 3y ago
  F#: List.init 4 (fun _ -> []) [for _ in 1..4 -> []] Or any sane language that uses immutable values by default like Elixir, Erlang, Clojure, Racket, Scheme, OCaml, etc. > Actually, if you really want to have re-evaluting (sub)expressions, then Python does give you an option: comprehensions. That's the complaint, that [[]]*4 is not shorthand for list comprehensions and that it's something entirely different. This is common in Python where two things you'd expect to be the same are indeed different in subtle ways. List comprehensions in Python are closer to sane, but they still have unexpected behavior because of the way scoping works in Python.

layer8 3y ago

The * operator could be defined to use a deepcopy operation, but isn’t, presumably for efficiency reasons. This has little to do with order of evaluation. Alternatively, the issue is that Python does not distinguish between value and reference types.

abhi9u 3y ago

Yes, the default function argument values is another surprising behavior. In this case, the default arguments get compiled as part of the function code and are part of the function's environment. As a result every time that function is called it is always having the same environment, i.e. the default argument values are not reinterpreted on every invocation. So the mutations to the environment are visible across function calls.

kerkeslager 3y ago
> In this case, the default arguments get compiled as part of the function code and are part of the function's environment. Yeah, I understand that; I'm saying that was a choice they made, and I think having the defaults be evaluated at each call time would have been a better choice.