13 ms·
The difference between a=a+b and a+=b in Python
- thaumasiotes 5y agoWhat a headline. They specifically mean for strings and lists - not exactly the first use case you'd think of for + and += ...
- dmz73 5y agoWell, to be honest, I would expect a = a + b and a += b to behave exactly the same regardless of the type they operate on. Most people use += as a shorthand for the longer version so having them behave differently is dangerous.
- nmg 5y agoI agree, I use += as shorthand, I wasn't aware it was fundamentally different under the hood.
- disgruntledphd2 5y agoIf you like this kind of thing, the python manual has this: https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html I found the book Fluent Python to be a great introduction to the ideas behind abstraction in Python. Apparently, it's cadged from the Art of the Metaobject Protocol, which is a great book (which annoyingly enough, is not available in ebook form, which is a shame as typing loads of code from a dead-tree book is time consuming).
- mananaysiempre 5y agoWell, it’s a tacit convention of the language that they perform related operations, like with __add__ and __radd__ or __lt__ and __gt__. Part of the responsibility of the class author is to follow that convention (protocol, if you will). For immutable types, a = a + b and a += b do behave exactly the same. For mutable ones, the problem is that of the three statements (taking lists as an example) a = a + b # (1) a += b # (2) a.extend(b) # (3) a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. Python’s designers chose to make (2) and (3) the same, but other languages may reasonably choose (1) and (2) instead. As far as I can see, this problem is inherent in having references to mutable objects as values in the language.
- __s 5y agoEven for usually immutable objects 1 / 2 vary. Python optimizes += on strings with only a single reference (which the temporary in a = a + b will cause there to be at least 2 references)
- mananaysiempre 5y agoI think this should be relegated to the “implementation details, not guaranteed and unobservable unless you use escape hatches” pile. Unfortunately, the set of escape hatches in Python (id, sys.getrefcount, etc. but also in some cases is) is not particularly clearly defined. That doesn’t mean that knowing an implementation detail can’t be useful, just that I feel the way (1) and (2) for lists differ (“semantics”) is of a different nature from the way (1) and (2) for strings differ (“implementation”) and worth keeping in a different mental drawer.
- codesections 5y ago> a reasonable programmer can (separately) expect both (1) and (2) to be the same or (2) and (3) to be the same, but these are in fact mutually exclusive. It's possible (likely? ) that I'm missing something, but why are they exclusive? In principle, I would have thought that an optimizing compiler could rewrite (1) into (3) and thus give all three the same semantics. (I'm not speaking about what would be possible for Python in particular, just in general terms)
- maxerickson 5y agoPython doesn't have typed variables, it has names (or references) and objects. If you start off with: a=c=[1,2,3] b=[4,5] then a=a+b with semantics that change 'a' would also change the object pointed to by 'c', which I think isn't what most people would expect (and is not the semantics specified by Python). The other forms explicitly change the object pointed to by 'a'.
- account42 5y agoThe problem is that a and b are not lists but pointers to (or names for) lists but python syntax tries to hide that. a = a + b creates a new list and changes a to point to that while a.extend(b) modifies the list pointed to by a. Clerly (2) can't match both.
- necovek 5y agoHum, I wouldn't for objects that are generally passed by reference (which is an early lesson one should learn with Python). Eg. a = [1, 2] b = [3, 4] c = a Would you really expect a += b and a = a + b to behave the same, and what would c be in that case?
- thaumasiotes 5y ago> Would you really expect a += b and a = a + b to behave the same, and what would c be in that case? Yes, I would, and it really is worth noting that they don't. c would be [1,2,3,4], and it would be an alias of a. But both of those points are also true of the way things actually do work. I don't really understand your question.
- alisonkisk 5y agoreassigning a does not change c. a = [1] c = a a = a + a Now, what is c ? what is a? Then try again with a+=a
- account42 5y ago>>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a = a + b >>> c [1, 2]
- necovek 5y ago> But both of those points are also true of the way things actually do work. That is not true. >>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a += b >>> c [1, 2, 3, 4] And contrast that with: >>> a = [1, 2] >>> b = [3, 4] >>> c = a >>> a = a + b >>> c [1, 2] If "c" remains an alias of "a" after you've assigned something else to "a", then there is no way in language to keep a reference to what "a" was. Eg. what you seem to be proposing is that assignment works like this: a = 4 old_a = a a = 5 print(old_a) # prints out 5. Python is like it is for practical performance reasons: lists and dicts are by-reference because copying them is expensive. You've got to do it explicitely with .copy() or deepcopy module. That's a basic lesson on Python I was referring to. And sure, it could have been designed the other way around, when you'd have to introduce a "refcopy" or "&" operator when performance is needed, but it wasn't designed that way. There are other languages which do that. I can see the point of "a += b" unexpectedly updating in place, but just upon reading it, I'd struggle to figure out what's supposed to happen. In a code review, I'd suggest a developer to switch to "a.extend(b)". Basically, if they were to behave the same, I don't think an argument can be made for the common result of "c" to evaluate to [1, 2, 3, 4], but rather to [1, 2].
- tialaramex 5y agoRust's std::ops says: "Implementations of operator traits should be unsurprising in their respective contexts, keeping in mind their usual meanings" Which seems like reasonable advice.
- deleted 5y ago[deleted]
- hoppla 5y agoSince lists are mutable, I can pass them around to functions as arguments, similar to C and perform operations without bothering to return the list. But if a function would do listA = listA + listB, I am not sure what listA would be outside the function scope… I am guessing listA would remain unchanged
- mark-r 5y agoYou are correct. Inside the function you reassign listA but that doesn't change anything outside the function; you now have two different lists. See https://stackoverflow.com/q/986006/5987 https://stackoverflow.com/q/986006/5987.
- alisonkisk 5y agoAliasing mutable objects is extremely because it's impossible to do "the right thing" for all use cases. If you don't do that, a=a+b and a+=b are denotationally the same.
- mrpopo 5y ago>>> mylist_1 = [1, 2, 3] >>> mylist_2 = [4, 5] >>> print(id(mylist_1), id(mylist_2)) 1614327515784 1614319969800 >>> mylist_1 = mylist_1 + mylist_2 >>> print(mylist_1) >>> print(id(mylist_1)) [1, 2, 3, 4, 5, 4, 5] 1614319969416 Isn't there an error in the [1, 2, 3, 4, 5, 4, 5] result?
- maxerickson 5y agoThere clearly is. Here's what I get: >>> l1=[1,2,3] >>> l2=[4,5] >>> l1=l1+l2 >>> print(l1) [1, 2, 3, 4, 5]
- mg 5y agoAs seen from the below example, += is marginally faster than the + operator. Marginally for a single operation. But way faster for more operations. This needs 34s on my laptop: l1 = [1] l2 = [2] for i in range(int(10e4)): l1 = l1 + l2 While this needs 0.03s: l1 = [1] l2 = [2] for i in range(int(10e4)): l1 += l2
- marcan_42 5y agol1 + l2 is O(len(l1) + len(l2)). l1 += l2 is O(len(l2)). That means that your first snippet is O(n^2) while your second snippet is O(n). It's not about doing many operations, it's about the size of l1. In-place append doesn't care, copying into a new object does.
- thaumasiotes 5y agoExcellent point. You could redo the test without this error by reassigning l1 inside the loop: for i in xrange(big): l1 = [i] l1 += l2 for i in xrange(big): l1 = [i] l1 = l1 + l2
- altcognito 5y agoIt’s not n^2, it only iterates over the list or lists sequentially. You would have to visit items more than once to get n^2. I’m not denying that it’s worse and it’s a excellent point, just that it’s only unfortunate not exponentially unfortunate. (Edit) leaving the comment for integrity of the thread but I failed to notice that you absolutely revisit the first elements of the first array every time through the loop (which is growing in size), which absolutely makes it n^2 whereas the better version just tacks it on without reconstruction.
- account42 5y agoIt is n^2: n operations that on average copy n/2 elements.
- deleted 5y ago[deleted]
- ndr 5y agoUsing the id to prove it's reusing the same instance is a poor proxy, eg: >>> id([1,2,3]) == id([4,5,6]) True >>> id([1,2,3]), id([4,5,6]) (140720308985224, 140720308985224) It's clearly not the same instance, even though it's at the same address. Not in this case though, but I'm not sure why the author uses a black-box model to explain it rather than looking at the source, `iadd` uses `extend`: https://github.com/python/cpython/blob/main/Objects/listobject.c#L988 https://github.com/python/cpython/blob/main/Objects/listobje...
- qayxc 5y agoNot to be rude or anything, but if the target audience needs basic semantics explained to them, I don't know they would benefit from an in-depth explanation.
- alisonkisk 5y agoBut if if the author needs basic semantics explained to them, I don't know that the audience would benefit from reading it. And why wouldn't an explanation help the audience learn?
- algorias 5y agoYou need to keep the objects alive obviously, then it works exactly as expected. >>> a = [1,2,3] >>> b = [4,5,6] >>> id(a), id(b) (140659402612872, 140659403098568)
- fnord123 5y agoFWIW, another difference is that if __iadd__ raises an exception and you don't revert the changes then the object is trashed.
- metafunctor 5y agoOperator overloading, what a great idea?
- freemint 5y agoOperator overloading is a great idea but compared myadd2 or add_fractions a + is way more readable.
- lionkor 5y agomore readable, but not easier to understand. a = a + b still carries the same information as add_and_copy(a, b), but one is misleading and the other isnt.
- ordu 5y ago> a + is way more readable. I believe it works much like a self-fulfilling prophecy. Language designers think that it is more readable, so they do it. Language users use it and became habituated to it. So it becomes more readable because some people believe that it is more readable. What is unreadable with: a.extend(b) c = a.clone().extend(b); Programmer have all the control of memory management, and at the same time she is free to do any operations on lists, to chain operations, to do everything. And all her intent is here, it is written, it is obvious. While a += b... It leads to a need to write articles describing the difference between + and += and how could I possibly know did programmer who wrote the code read all such articles? Did he wrote he intended to, or what is nicey looking, but not great semantically? Did I read all these articles or I missed something? Even if it is more readable, does it justify all the obscurity?
- account42 5y agoDo you also believe all built-in types like integers should use named functions instead of operators?
- ordu 5y agoNo. I even believe that non-builtin types behaving like numbers might use operators. But lists, arrays, strings behave not like numbers. They allocate memory, they reuse memory for values, they have operations of non O(1) complexity, and so they shouldn't be using operators. If one really wants to, then he might invent new operators for this, like concatenation-operator. I didn't see any reasonable way to do it, but it doesn't mean that such way cannot be devised.
- personjerry 5y agoLow quality clickbait. This whole article could've been 2 sentences. Here's the entire article for you: "For lists, += and *= do the operation in place, whereas a=a+b will create a new object and then copy. The former is marginally faster."
- wodenokoto 5y agoMy understanding of python is that all operators are shorthands for calling methods on the objects. E.g. 1 + 2 translates to 1.__add__(2) and a += b translates to a.__iadd__(b) So the difference between `+` and `+=` are dependent on the object in question. Then in practise, there are a lot of optimizations and implementation details.
- FabHK 5y agoTLDR: The abbreviation mutates the LHS, while the long version assigns a newly created object to it. >>> orig = [1,2,3] >>> L1 = orig >>> L2 = [7,8] >>> L1 = L1 + L2 >>> orig [1, 2, 3] >>> orig = [1,2,3] >>> L1 = orig >>> L2 = [7,8] >>> L1 += L2 >>> orig [1, 2, 3, 7, 8]
- mdemare 5y agoIn case anyone wondered, in Ruby they are identical, for any type.
- deleted 5y ago[deleted]
- tcpekin 5y agoThis happens in numpy as well, and was confusing the first time I saw it: b = np.ones(1) c = np.ones(1) c = c/(2**73) b += c >>> ufunc 'add' output (typecode 'O') could not be coerced to provided output parameter (typecode 'd') according to the casting rule ''same_kind'' but `b = b + c` does work. [1] https://stackoverflow.com/questions/12588986/typeerror-generated-when-using-inplace-operations-on-numpy-arrays https://stackoverflow.com/questions/12588986/typeerror-gener...