7 ms·

Well, to be honest, I would expect a = a + b and a += b to behave exactly the same regardless of the type they operate on. Most people use += as a shorthand for

by dmz73 5y ago

Well, to be honest, I would expect a = a + b and a += b to behave exactly the same regardless of the type they operate on. Most people use += as a shorthand for the longer version so having them behave differently is dangerous.

nmg 5y ago
I agree, I use += as shorthand, I wasn't aware it was fundamentally different under the hood.
- disgruntledphd2 5y ago
  If you like this kind of thing, the python manual has this: https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html I found the book Fluent Python to be a great introduction to the ideas behind abstraction in Python. Apparently, it's cadged from the Art of the Metaobject Protocol, which is a great book (which annoyingly enough, is not available in ebook form, which is a shame as typing loads of code from a dead-tree book is time consuming).
mananaysiempre 5y ago
Well, it’s a tacit convention of the language that they perform related operations, like with __add__ and __radd__ or __lt__ and __gt__. Part of the responsibility of the class author is to follow that convention (protocol, if you will). For immutable types, a = a + b and a += b do behave exactly the same. For mutable ones, the problem is that of the three statements (taking lists as an example) a = a + b # (1) a += b # (2) a.extend(b) # (3) a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. Python’s designers chose to make (2) and (3) the same, but other languages may reasonably choose (1) and (2) instead. As far as I can see, this problem is inherent in having references to mutable objects as values in the language.
- __s 5y ago
  Even for usually immutable objects 1 / 2 vary. Python optimizes += on strings with only a single reference (which the temporary in a = a + b will cause there to be at least 2 references)
  - mananaysiempre 5y ago
    I think this should be relegated to the “implementation details, not guaranteed and unobservable unless you use escape hatches” pile. Unfortunately, the set of escape hatches in Python (id, sys.getrefcount, etc. but also in some cases is) is not particularly clearly defined. That doesn’t mean that knowing an implementation detail can’t be useful, just that I feel the way (1) and (2) for lists differ (“semantics”) is of a different nature from the way (1) and (2) for strings differ (“implementation”) and worth keeping in a different mental drawer.
- codesections 5y ago
  > a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. It's possible (likely? ) that I'm missing something, but why are they exclusive? In principle, I would have thought that an optimizing compiler could rewrite (1) into (3) and thus give all three the same semantics. (I'm not speaking about what would be possible for Python in particular, just in general terms)
  - maxerickson 5y ago
    Python doesn't have typed variables, it has names (or references) and objects. If you start off with: a=c=[1,2,3] b=[4,5] then a=a+b with semantics that change 'a' would also change the object pointed to by 'c', which I think isn't what most people would expect (and is not the semantics specified by Python). The other forms explicitly change the object pointed to by 'a'.
  - account42 5y ago
    The problem is that a and b are not lists but pointers to (or names for) lists but python syntax tries to hide that. a = a + b creates a new list and changes a to point to that while a.extend(b) modifies the list pointed to by a. Clerly (2) can't match both.
necovek 5y ago
Hum, I wouldn't for objects that are generally passed by reference (which is an early lesson one should learn with Python). Eg. a = [1, 2] b = [3, 4] c = a Would you really expect a += b and a = a + b to behave the same, and what would c be in that case?
- thaumasiotes 5y ago
  > Would you really expect a += b and a = a + b to behave the same, and what would c be in that case? Yes, I would, and it really is worth noting that they don't. c would be [1,2,3,4], and it would be an alias of a. But both of those points are also true of the way things actually do work. I don't really understand your question.
  - alisonkisk 5y ago
    reassigning a does not change c. a = [1] c = a a = a + a Now, what is c ? what is a? Then try again with a+=a
  - account42 5y ago
    >>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a = a + b >>> c [1, 2]
  - necovek 5y ago
    > But both of those points are also true of the way things actually do work. That is not true. >>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a += b >>> c [1, 2, 3, 4] And contrast that with: >>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a = a + b >>> c [1, 2] If "c" remains an alias of "a" after you've assigned something else to "a", then there is no way in language to keep a reference to what "a" was. Eg. what you seem to be proposing is that assignment works like this: a = 4 old_a = a a = 5 print(old_a) # prints out 5. Python is like it is for practical performance reasons: lists and dicts are by-reference because copying them is expensive. You've got to do it explicitely with .copy() or deepcopy module. That's a basic lesson on Python I was referring to. And sure, it could have been designed the other way around, when you'd have to introduce a "refcopy" or "&" operator when performance is needed, but it wasn't designed that way. There are other languages which do that. I can see the point of "a += b" unexpectedly updating in place, but just upon reading it, I'd struggle to figure out what's supposed to happen. In a code review, I'd suggest a developer to switch to "a.extend(b)". Basically, if they were to behave the same, I don't think an argument can be made for the common result of "c" to evaluate to [1, 2, 3, 4], but rather to [1, 2].
tialaramex 5y ago
Rust's std::ops says: "Implementations of operator traits should be unsurprising in their respective contexts, keeping in mind their usual meanings" Which seems like reasonable advice.
deleted 5y ago
[deleted]
hoppla 5y ago
Since lists are mutable, I can pass them around to functions as arguments, similar to C and perform operations without bothering to return the list. But if a function would do listA = listA + listB, I am not sure what listA would be outside the function scope… I am guessing listA would remain unchanged
- mark-r 5y ago
  You are correct. Inside the function you reassign listA but that doesn't change anything outside the function; you now have two different lists. See https://stackoverflow.com/q/986006/5987 https://stackoverflow.com/q/986006/5987.
alisonkisk 5y ago
Aliasing mutable objects is extremely because it's impossible to do "the right thing" for all use cases. If you don't do that, a=a+b and a+=b are denotationally the same.

by dmz73 5y ago

nmg 5y ago

I agree, I use += as shorthand, I wasn't aware it was fundamentally different under the hood.

disgruntledphd2 5y ago
If you like this kind of thing, the python manual has this: https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html I found the book Fluent Python to be a great introduction to the ideas behind abstraction in Python. Apparently, it's cadged from the Art of the Metaobject Protocol, which is a great book (which annoyingly enough, is not available in ebook form, which is a shame as typing loads of code from a dead-tree book is time consuming).

mananaysiempre 5y ago

Well, it’s a tacit convention of the language that they perform related operations, like with __add__ and __radd__ or __lt__ and __gt__. Part of the responsibility of the class author is to follow that convention (protocol, if you will). For immutable types, a = a + b and a += b do behave exactly the same. For mutable ones, the problem is that of the three statements (taking lists as an example) a = a + b # (1) a += b # (2) a.extend(b) # (3) a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. Python’s designers chose to make (2) and (3) the same, but other languages may reasonably choose (1) and (2) instead. As far as I can see, this problem is inherent in having references to mutable objects as values in the language.

__s 5y ago
Even for usually immutable objects 1 / 2 vary. Python optimizes += on strings with only a single reference (which the temporary in a = a + b will cause there to be at least 2 references)
- mananaysiempre 5y ago
  I think this should be relegated to the “implementation details, not guaranteed and unobservable unless you use escape hatches” pile. Unfortunately, the set of escape hatches in Python (id, sys.getrefcount, etc. but also in some cases is) is not particularly clearly defined. That doesn’t mean that knowing an implementation detail can’t be useful, just that I feel the way (1) and (2) for lists differ (“semantics”) is of a different nature from the way (1) and (2) for strings differ (“implementation”) and worth keeping in a different mental drawer.
codesections 5y ago
> a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. It's possible (likely? ) that I'm missing something, but why are they exclusive? In principle, I would have thought that an optimizing compiler could rewrite (1) into (3) and thus give all three the same semantics. (I'm not speaking about what would be possible for Python in particular, just in general terms)
- maxerickson 5y ago
  Python doesn't have typed variables, it has names (or references) and objects. If you start off with: a=c=[1,2,3] b=[4,5] then a=a+b with semantics that change 'a' would also change the object pointed to by 'c', which I think isn't what most people would expect (and is not the semantics specified by Python). The other forms explicitly change the object pointed to by 'a'.
- account42 5y ago
  The problem is that a and b are not lists but pointers to (or names for) lists but python syntax tries to hide that. a = a + b creates a new list and changes a to point to that while a.extend(b) modifies the list pointed to by a. Clerly (2) can't match both.

necovek 5y ago

Hum, I wouldn't for objects that are generally passed by reference (which is an early lesson one should learn with Python). Eg. a = [1, 2] b = [3, 4] c = a Would you really expect a += b and a = a + b to behave the same, and what would c be in that case?

thaumasiotes 5y ago
> Would you really expect a += b and a = a + b to behave the same, and what would c be in that case? Yes, I would, and it really is worth noting that they don't. c would be [1,2,3,4], and it would be an alias of a. But both of those points are also true of the way things actually do work. I don't really understand your question.
- alisonkisk 5y ago
  reassigning a does not change c. a = [1] c = a a = a + a Now, what is c ? what is a? Then try again with a+=a
- account42 5y ago
  >>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a = a + b >>> c [1, 2]
- necovek 5y ago
  > But both of those points are also true of the way things actually do work. That is not true. >>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a += b >>> c [1, 2, 3, 4] And contrast that with: >>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a = a + b >>> c [1, 2] If "c" remains an alias of "a" after you've assigned something else to "a", then there is no way in language to keep a reference to what "a" was. Eg. what you seem to be proposing is that assignment works like this: a = 4 old_a = a a = 5 print(old_a) # prints out 5. Python is like it is for practical performance reasons: lists and dicts are by-reference because copying them is expensive. You've got to do it explicitely with .copy() or deepcopy module. That's a basic lesson on Python I was referring to. And sure, it could have been designed the other way around, when you'd have to introduce a "refcopy" or "&" operator when performance is needed, but it wasn't designed that way. There are other languages which do that. I can see the point of "a += b" unexpectedly updating in place, but just upon reading it, I'd struggle to figure out what's supposed to happen. In a code review, I'd suggest a developer to switch to "a.extend(b)". Basically, if they were to behave the same, I don't think an argument can be made for the common result of "c" to evaluate to [1, 2, 3, 4], but rather to [1, 2].

tialaramex 5y ago

Rust's std::ops says: "Implementations of operator traits should be unsurprising in their respective contexts, keeping in mind their usual meanings" Which seems like reasonable advice.

deleted 5y ago

[deleted]

hoppla 5y ago

Since lists are mutable, I can pass them around to functions as arguments, similar to C and perform operations without bothering to return the list. But if a function would do listA = listA + listB, I am not sure what listA would be outside the function scope… I am guessing listA would remain unchanged

mark-r 5y ago
You are correct. Inside the function you reassign listA but that doesn't change anything outside the function; you now have two different lists. See https://stackoverflow.com/q/986006/5987 https://stackoverflow.com/q/986006/5987.

alisonkisk 5y ago

Aliasing mutable objects is extremely because it's impossible to do "the right thing" for all use cases. If you don't do that, a=a+b and a+=b are denotationally the same.