6 ms·

Well, it’s a tacit convention of the language that they perform related operations, like with add and radd or lt and gt. Part of the responsibil

Well, it’s a tacit convention of the language that they perform related operations, like with __add__ and __radd__ or __lt__ and __gt__. Part of the responsibility of the class author is to follow that convention (protocol, if you will). For immutable types, a = a + b and a += b do behave exactly the same. For mutable ones, the problem is that of the three statements (taking lists as an example) a = a + b # (1) a += b # (2) a.extend(b) # (3) a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. Python’s designers chose to make (2) and (3) the same, but other languages may reasonably choose (1) and (2) instead. As far as I can see, this problem is inherent in having references to mutable objects as values in the language.

__s 5y ago
Even for usually immutable objects 1 / 2 vary. Python optimizes += on strings with only a single reference (which the temporary in a = a + b will cause there to be at least 2 references)
- mananaysiempre 5y ago
  I think this should be relegated to the “implementation details, not guaranteed and unobservable unless you use escape hatches” pile. Unfortunately, the set of escape hatches in Python (id, sys.getrefcount, etc. but also in some cases is) is not particularly clearly defined. That doesn’t mean that knowing an implementation detail can’t be useful, just that I feel the way (1) and (2) for lists differ (“semantics”) is of a different nature from the way (1) and (2) for strings differ (“implementation”) and worth keeping in a different mental drawer.
codesections 5y ago
> a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. It's possible (likely? ) that I'm missing something, but why are they exclusive? In principle, I would have thought that an optimizing compiler could rewrite (1) into (3) and thus give all three the same semantics. (I'm not speaking about what would be possible for Python in particular, just in general terms)
- maxerickson 5y ago
  Python doesn't have typed variables, it has names (or references) and objects. If you start off with: a=c=[1,2,3] b=[4,5] then a=a+b with semantics that change 'a' would also change the object pointed to by 'c', which I think isn't what most people would expect (and is not the semantics specified by Python). The other forms explicitly change the object pointed to by 'a'.
- account42 5y ago
  The problem is that a and b are not lists but pointers to (or names for) lists but python syntax tries to hide that. a = a + b creates a new list and changes a to point to that while a.extend(b) modifies the list pointed to by a. Clerly (2) can't match both.
- necovek 5y ago
  It's true that an optimizing compiler could rewrite (1) into (3), but not in every case. Eg. my example somewhere below of keeping another reference to "a" (eg. "c = a") would stop it from doing that optimization.
necovek 5y ago
I think a reasonable programmer who would expect (1) and (2) to behave the same would also expect the following to happen: a = [1, 2]; b = [3, 4] c = a a += b print(c) # [1, 2] — assignment was a copy print(a) # [1, 2, 3, 4] So every list assignment would have to be a copy for that mental model not to break easily. And certainly, there are languages that do that. Just not Python — for performance reasons, you need to be explicit about copying large chunks of memory, though it includes a fair number of footguns like concatenation of strings and lists, or even slicing — eg. "for a in long_list[:-2]:" will make a duplicate of long_list in memory.

Well, it’s a tacit convention of the language that they perform related operations, like with __add__ and __radd__ or __lt__ and __gt__. Part of the responsibil

__s 5y ago

Even for usually immutable objects 1 / 2 vary. Python optimizes += on strings with only a single reference (which the temporary in a = a + b will cause there to be at least 2 references)

mananaysiempre 5y ago
I think this should be relegated to the “implementation details, not guaranteed and unobservable unless you use escape hatches” pile. Unfortunately, the set of escape hatches in Python (id, sys.getrefcount, etc. but also in some cases is) is not particularly clearly defined. That doesn’t mean that knowing an implementation detail can’t be useful, just that I feel the way (1) and (2) for lists differ (“semantics”) is of a different nature from the way (1) and (2) for strings differ (“implementation”) and worth keeping in a different mental drawer.

codesections 5y ago

> a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. It's possible (likely? ) that I'm missing something, but why are they exclusive? In principle, I would have thought that an optimizing compiler could rewrite (1) into (3) and thus give all three the same semantics. (I'm not speaking about what would be possible for Python in particular, just in general terms)

maxerickson 5y ago
Python doesn't have typed variables, it has names (or references) and objects. If you start off with: a=c=[1,2,3] b=[4,5] then a=a+b with semantics that change 'a' would also change the object pointed to by 'c', which I think isn't what most people would expect (and is not the semantics specified by Python). The other forms explicitly change the object pointed to by 'a'.
account42 5y ago
The problem is that a and b are not lists but pointers to (or names for) lists but python syntax tries to hide that. a = a + b creates a new list and changes a to point to that while a.extend(b) modifies the list pointed to by a. Clerly (2) can't match both.
necovek 5y ago
It's true that an optimizing compiler could rewrite (1) into (3), but not in every case. Eg. my example somewhere below of keeping another reference to "a" (eg. "c = a") would stop it from doing that optimization.

necovek 5y ago

I think a reasonable programmer who would expect (1) and (2) to behave the same would also expect the following to happen: a = [1, 2]; b = [3, 4] c = a a += b print(c) # [1, 2] — assignment was a copy print(a) # [1, 2, 3, 4] So every list assignment would have to be a copy for that mental model not to break easily. And certainly, there are languages that do that. Just not Python — for performance reasons, you need to be explicit about copying large chunks of memory, though it includes a fair number of footguns like concatenation of strings and lists, or even slicing — eg. "for a in long_list[:-2]:" will make a duplicate of long_list in memory.