4 ms·

I've written less than 1000 lines of Python in total probably, but I correctly spotted the problem. Python has this misfeature whereby it didn't correctly crib

I've written less than 1000 lines of Python in total probably, but I correctly spotted the problem. Python has this misfeature whereby it didn't correctly crib Common Lisp's evaluation strategy for the expressions that give default values to optional function arguments. When you have an argument like foo=obj.whatever() the obj.whatever() is evaluated (would you believe it!) at the time the definition of the function is being processed, not at the time when the function is being called and the value is needed. Is suspect this was done on purpose, for efficiency. Python has another misfeature: it has no literal syntax for certain common objects like lists. There is [1, 2, 3], but that is a constructor and not a literal: it has to create a new list every time it is evaluated and stuff it with 1, 2, 3. (Unless a clever compiler can prove that this can be optimized away without harm.) The designer didn't want a parameter like list=[] to have to construct a new, empty list object each time the argument is omitted. In Lisp '(1 2 3) and '() are true literals. Whenever they are referenced, they denote the same object. The programmer has a choice here: they can use (list 1 2 3) as the default value expression or '(1 2 3). The former is like [1, 2, 3]: it yields a new object each time that is mutable; the other will (almost certainly) yield the same object and cannot be reliably, portably modified. Hey, modern popular languages have most of the features of Lisp, so you're not missing anything.

sdwr 2y ago
> When you have an argument like foo=obj.whatever(), the obj.whatever() is evaluated at the time the definition of the function is being processed, not at the time when the function is being called. This can't be correct, surely? What if .whatever() relies on internal state that changes after obj is initialized (or after the function surrounding foo is declared, not sure what you're saying)?
- chii 2y ago
  https://stackoverflow.com/questions/1132941/the-mutable-default-argument-in-python https://stackoverflow.com/questions/1132941/the-mutable-defa... and https://www.valentinog.com/blog/tirl-python-default-arguments/ https://www.valentinog.com/blog/tirl-python-default-argument... basically, having a default argument value in a function definition means to evaluate it during definition time of that function, not when the function is invoked. This is a foot gun.
  - bushbaba 2y ago
    if the default argument is an object it’s reused between invocations. Hence why setting default parameters to empty list / empty dict is flagged by static analysis suites
- MrJohz 2y ago
  It is correct, it's one of the most surprising things about Python and it causes a number of mistakes, even for experts. The easiest way to see this is by running something like this and seeing what gets printed out and when: print("1. start") def function(arg=print("2. func definition")): print("4. func call") print("3. after definition") function() function() function() You should see that the print statement in the default position is called once, when the function definition itself is being evaluated, not when the function gets called.
  - fragmede 2y ago
    My canonical example is that you want a function to have an argument that defaults to an empty list: def func(arg=[]): arg.append(1) print(arg) func() func() func() shoving a print into a function definition is weird and not something you'd do normally. But someone who doesn't know this footgun is going to write a function that defaults to an empty list, and then tear their hair out when things are broken.
    - deleted 2y ago
      [deleted]
    - MrJohz 2y ago
      That's true, this is the example that most people will run into in practice. But the print example is useful because it shows what's going on more explicitly: the default argument gets evaluated as part of the function's definition. I think this helps people get a better intuition for how Python's interpreter works when it comes to evaluating function definitions.
  - OptionOfT 2y ago
    And because it is evaluated once, unlike in say TypeScript, when you modify `arg` in the first call, the results are visible in the second call.
  - johnny22 2y ago
    I find this fascinating because while I don't write much python, this is the behaviour I would assume to be correct based on everything else I've seen. I wouldn't expect it to be evaluated on every call. Definitely shows that your previous familiarity can trip you up (or not)
- kazinator 2y ago
  There is an obj visible at the point of the function definition. When that function definition takes place, obj.whatever() is called, and the value stashed away. Thus the call obj.whatever() has to work at that time. The stashed value is retrieved whenever the corresponding function argument is missing.
voidnap 2y ago
While you are correct, that is not what is happening in the blog post. Their issue was inside a class definition, not a function.
lispm 2y ago
Good explanation...

sdwr 2y ago

> When you have an argument like foo=obj.whatever(), the obj.whatever() is evaluated at the time the definition of the function is being processed, not at the time when the function is being called. This can't be correct, surely? What if .whatever() relies on internal state that changes after obj is initialized (or after the function surrounding foo is declared, not sure what you're saying)?

chii 2y ago
https://stackoverflow.com/questions/1132941/the-mutable-default-argument-in-python https://stackoverflow.com/questions/1132941/the-mutable-defa... and https://www.valentinog.com/blog/tirl-python-default-arguments/ https://www.valentinog.com/blog/tirl-python-default-argument... basically, having a default argument value in a function definition means to evaluate it during definition time of that function, not when the function is invoked. This is a foot gun.
- bushbaba 2y ago
  if the default argument is an object it’s reused between invocations. Hence why setting default parameters to empty list / empty dict is flagged by static analysis suites
MrJohz 2y ago
It is correct, it's one of the most surprising things about Python and it causes a number of mistakes, even for experts. The easiest way to see this is by running something like this and seeing what gets printed out and when: print("1. start") def function(arg=print("2. func definition")): print("4. func call") print("3. after definition") function() function() function() You should see that the print statement in the default position is called once, when the function definition itself is being evaluated, not when the function gets called.
- fragmede 2y ago
  My canonical example is that you want a function to have an argument that defaults to an empty list: def func(arg=[]): arg.append(1) print(arg) func() func() func() shoving a print into a function definition is weird and not something you'd do normally. But someone who doesn't know this footgun is going to write a function that defaults to an empty list, and then tear their hair out when things are broken.
  - deleted 2y ago
    [deleted]
  - MrJohz 2y ago
    That's true, this is the example that most people will run into in practice. But the print example is useful because it shows what's going on more explicitly: the default argument gets evaluated as part of the function's definition. I think this helps people get a better intuition for how Python's interpreter works when it comes to evaluating function definitions.
- OptionOfT 2y ago
  And because it is evaluated once, unlike in say TypeScript, when you modify `arg` in the first call, the results are visible in the second call.
- johnny22 2y ago
  I find this fascinating because while I don't write much python, this is the behaviour I would assume to be correct based on everything else I've seen. I wouldn't expect it to be evaluated on every call. Definitely shows that your previous familiarity can trip you up (or not)
kazinator 2y ago
There is an obj visible at the point of the function definition. When that function definition takes place, obj.whatever() is called, and the value stashed away. Thus the call obj.whatever() has to work at that time. The stashed value is retrieved whenever the corresponding function argument is missing.

voidnap 2y ago

While you are correct, that is not what is happening in the blog post. Their issue was inside a class definition, not a function.

lispm 2y ago

Good explanation...