6 ms·

Nicely written article! - Slightly off topic: I love seeing and reading Python code. Used to see many flaws in the language (things like `.append` changing the

Nicely written article! - Slightly off topic: I love seeing and reading Python code. Used to see many flaws in the language (things like `.append` changing the object, returning `None` instead of creating a copy and returning that), but after ten years of working with Python I really appreciate its versatility, it‘s ubiquitous availability, the large number of libraries and the community. There‘s nothing I can‘t solve with it. It‘s the swiss army knife in my pocket, available whenever I need to solve something by coding.

tusharsadhwani 5y ago
Thank you! Things like `list.append` modifying in-place might feel like a flaw to some, but I think Python is really consistent when it comes to its behaviour. If you ask a person who comes from an object-oriented world, they'll say it only makes sense for a method on an object to modify that object's data directly. There's always ways to do things the other way, for example you can use x = [*x, item] to append and create a new copy, while being quite a bit more explicit that a new list is being created.
- pokepim 5y ago
  Huh I never even thought we would need to create copy of an object when adding new item to it (like a new item to list for example). Is there any drawback on doing that in standard pythonic way? I actually learned to program using Python and it was my first language. Since then I only used JS. In both I like using functions a lot and rarely dabble in OOP since it is more conveniet to me.
  - tyingq 5y ago
    You can control the behavior manually, like: first = [0,1,2] second = [*a,3] # first is unchanged, second = [0,1,2,3] Or second=itertools.chain(first, [3]), which avoids the copy. Though, to me, it's asking for trouble later.
    - deleted 5y ago
      [deleted]
  - sgeisenh 5y ago
    You often lose performance in traditional imperative languages when aiming for persistence. When you have immutability guarantees (like in many functional programming languages like ML or Haskell) you can avoid making copies by sharing the parts of the data structure that don't change. If this kind of thing interests you, you should check out Chris Okasaki's book "Purely Functional Data Structures".
    - eximius 5y ago
      "avoid making copies" dors not always equal "performance". Depending on your access patterns, having the data colocated can be more important. But immutability sure is nice when you can have it.
  - tusharsadhwani 5y ago
    whether mutating data is better than creating a new copy for everything is a really long debate about immutability and functional programming, with good points on either sides, but that's really beyond the point here. In my opinion, you should use whichever method makes your code easy to read and understand for your usecase.
- tyingq 5y ago
  >Python is really consistent when it comes to its behaviour True, though you end up with things like: ' '.join(thelist) Instead of thelist.join(' ') Because of the somewhat aggressive mantra to be consistent.
  - Marazan 5y ago
    But that's good. Because a string just needs to know aboit interable to perform that operation whereas every iterable would need to implement it's own join if you had it the other way around.
    - goatlover 5y ago
      Yet Ruby and JS manage to do it somehow. To me it seems natural that join should be a method on the iterable, and I always have to pause to remember Python is different.
      - aidos 5y ago
        How does that work? Don’t you have to effectively convert your general iterable to an array and then join on that? Array.from(iterable).join(…)?
      - deleted 5y ago
        [deleted]
      - globular-toast 5y ago
        I don't think it should be a method at all. It's just a function: join(iterable, separator). It can also be implemented with reduce naturally: `reduce(lambda x, y: x + separator + y, iterable)`.
        Redoubts 5y ago
        Reduce sounds like a really slow way to do string building
- int_19h 5y ago
  One related area where Python is not consistent is operators like +=. In pretty much all other languages that have them, the expected behavior of A+=B is exactly the same as A=A+B, except that A is only evaluated once. Now lets look at lists in Python: xs = [1, 2] ys = xs ys = ys + [3] print(xs, ys) This prints [1, 2] [1, 2, 3], because the third line created a new list, and made ys reference that. On the other hand, this: xs = [1, 2] ys = xs ys += [3] print(xs, ys) prints [1, 2, 3] [1, 2, 3], because += changes the list itself, and both xs and ys refer to that same list. (Note that this is not the same as C++, because in the latter, the variables store values directly, while in Python, all variables are references to values.) The worst part of it is that Python isn't even self-consistent here. If you only define __add__ in your custom class, you can use both + and += with its instances, with the latter behaving normally. But if you define __iadd__, as list does, then you can do whatever you want - and the idiomatic behavior is to modify the instance! For comparison, C# lets you overload + but not +=, and automatically synthesizes the latter from the former to enforce the correct behavior.
  - milliams 5y ago
    The rule in Python is that `=` creates a name binding, where as `+=` does not. That's pretty consistent as far as I can tell.
    - roelschroeven 5y ago
      The inconsistency is that for immutable objects (or more generally, objects that have __add__ but not __iadd__), '+=' does create a name binding. After 'a = 300; b = 50000', a += b is exactly the same as a = a + b In this case, a new object is created with the value of a + b which then gets bound to the name a.
    - int_19h 5y ago
      Nope, += also creates a binding! Try this: xs = [1, 2] def foo(): xs += [3] foo() You'll get an exception saying that local variable xs was used before it was assigned - precisely because += created a new local binding for xs inside foo.
radarsat1 5y ago
> (things like `.append` changing the object, returning `None` instead of creating a copy and returning that) This would be horrendously inefficient without immutable data structures like Clojure's. Very few languages have that, so it's a strange assumption to make, especially for a language as old as Python. Although it is a very nice feature of Clojure.
- l33tc0der 5y ago
  One of the many reasons why I love Clojure so much! Rich implement his own brand of persistent data structures which makes Clojure's immutability a lot more efficient.
  - radarsat1 5y ago
    Thanks yeah, persistent is the term I was looking for, did not come to mind as I haven't used Clojure in a few years.
- Blikkentrekker 5y ago
  It should also be worth nothing that Clojure sacrificed quite a bit to make this as efficient as possible. “persistent vectors” are certainly an interesting data structure that strike a compromise between fast indexing and being able to relatively quickly create a copy where only one element changes, but it's a compromise and indexing is made slower to allow for the latter. — They also take up more memory on their own but are allowed to share memory with their copies. I will say that my ideal language contains them in the standard library alongside standard vectors that index in constant time. Further, it should be noted that much of the performance talk is on the assumption that accessing from memory is truly random access; — with the existence of c.p.u. caches that assumption is not entirely accurate and accessing from contiguous rather than scattered memory in practice is considerably cheaper so one also pays the price for their being scattered more in memory.
  - adrusi 5y ago
    Random access into a clojure vector is going to need more memory lookups than conventional sequential buffer array (I don't recall the constants used in the implementation, I think it's either 4 or 8 lookups). But when you're indexing into the vector sequentially, the memory layout plays rather well with memory caching behavior, and most lookups are going to be in L1 cache, just like they would be in a conventional array. So lookups are a bit more expensive, but not as much more expensive as one might imagine.
Alex3917 5y ago
> things like `.append` changing the object, returning `None` instead of creating a copy and returning that The obvious question is why it can't return a reference to the list instead of returning None. I feel like if I've been using the language on an almost daily basis for ten years now and I still get burned by that all the time, then it's just a poorly designed feature.
- canjobear 5y ago
  The advantage of mutating operations always returning None is that you can easily tell whether a mutation is happening by looking at the code. If you see y = f(x) that means x is unchanged, whereas if you see just f(x) on a line that means something stateful is happening.
  - macintux 5y ago
    There are counter-examples: functions that return values while also having side effects, cases where the developer simply made a mistake. Coming from a language with baked-in immutability, Python’s behavior in this regard was very difficult to get used to.
  - brundolf 5y ago
    Agreed. JavaScript's Array.sort is an example of this. Most of JavaScript's other array methods return a new array and people get used to chaining them, but sort mutates the array and also returns a reference to it. You can actually get pretty far before being bitten by this so long as you're sorting already-copied arrays. But then one day you hit a bizarre bug caused by behavior that's been sneaking past your radar the whole time.
  - Xavdidtheshadow 5y ago
    I really like ruby's method naming convention for this: * `sort(arr)` returns a sorted copy of the input * `sort!(arr)` returns the sorted original methods that return booleans end in `?`, like `arr.sorted?` It's just a convention, but it's a nice way to let the writer know what will happen.
brundolf 5y ago
> Used to see many flaws in the language (things like `.append` changing the object, returning `None` instead of creating a copy and returning that) I think it's pretty off-base to call this a "flaw". Immutable structures have their place and can be very helpful where appropriate, but making its core primitives work this way is far outside the scope or the philosophy of Python. If you want otherwise, you're really wanting an entirely different language. And there's nothing wrong with that! But I think it would be a "flaw" for Python to make these operations immutable, even though I love immutability personally.
listenallyall 5y ago
And also like a Swiss army knife, it's not particularly great at anything, can be awkward even when functional, and there's always a better tool for any specific job.

tusharsadhwani 5y ago

Thank you! Things like `list.append` modifying in-place might feel like a flaw to some, but I think Python is really consistent when it comes to its behaviour. If you ask a person who comes from an object-oriented world, they'll say it only makes sense for a method on an object to modify that object's data directly. There's always ways to do things the other way, for example you can use x = [*x, item] to append and create a new copy, while being quite a bit more explicit that a new list is being created.

pokepim 5y ago
Huh I never even thought we would need to create copy of an object when adding new item to it (like a new item to list for example). Is there any drawback on doing that in standard pythonic way? I actually learned to program using Python and it was my first language. Since then I only used JS. In both I like using functions a lot and rarely dabble in OOP since it is more conveniet to me.
- tyingq 5y ago
  You can control the behavior manually, like: first = [0,1,2] second = [*a,3] # first is unchanged, second = [0,1,2,3] Or second=itertools.chain(first, [3]), which avoids the copy. Though, to me, it's asking for trouble later.
  - deleted 5y ago
    [deleted]
- sgeisenh 5y ago
  You often lose performance in traditional imperative languages when aiming for persistence. When you have immutability guarantees (like in many functional programming languages like ML or Haskell) you can avoid making copies by sharing the parts of the data structure that don't change. If this kind of thing interests you, you should check out Chris Okasaki's book "Purely Functional Data Structures".
  - eximius 5y ago
    "avoid making copies" dors not always equal "performance". Depending on your access patterns, having the data colocated can be more important. But immutability sure is nice when you can have it.
- tusharsadhwani 5y ago
  whether mutating data is better than creating a new copy for everything is a really long debate about immutability and functional programming, with good points on either sides, but that's really beyond the point here. In my opinion, you should use whichever method makes your code easy to read and understand for your usecase.
tyingq 5y ago
>Python is really consistent when it comes to its behaviour True, though you end up with things like: ' '.join(thelist) Instead of thelist.join(' ') Because of the somewhat aggressive mantra to be consistent.
- Marazan 5y ago
  But that's good. Because a string just needs to know aboit interable to perform that operation whereas every iterable would need to implement it's own join if you had it the other way around.
  - goatlover 5y ago
    Yet Ruby and JS manage to do it somehow. To me it seems natural that join should be a method on the iterable, and I always have to pause to remember Python is different.
    - aidos 5y ago
      How does that work? Don’t you have to effectively convert your general iterable to an array and then join on that? Array.from(iterable).join(…)?
    - deleted 5y ago
      [deleted]
    - globular-toast 5y ago
      I don't think it should be a method at all. It's just a function: join(iterable, separator). It can also be implemented with reduce naturally: `reduce(lambda x, y: x + separator + y, iterable)`.
      - Redoubts 5y ago
        Reduce sounds like a really slow way to do string building
int_19h 5y ago
One related area where Python is not consistent is operators like +=. In pretty much all other languages that have them, the expected behavior of A+=B is exactly the same as A=A+B, except that A is only evaluated once. Now lets look at lists in Python: xs = [1, 2] ys = xs ys = ys + [3] print(xs, ys) This prints [1, 2] [1, 2, 3], because the third line created a new list, and made ys reference that. On the other hand, this: xs = [1, 2] ys = xs ys += [3] print(xs, ys) prints [1, 2, 3] [1, 2, 3], because += changes the list itself, and both xs and ys refer to that same list. (Note that this is not the same as C++, because in the latter, the variables store values directly, while in Python, all variables are references to values.) The worst part of it is that Python isn't even self-consistent here. If you only define __add__ in your custom class, you can use both + and += with its instances, with the latter behaving normally. But if you define __iadd__, as list does, then you can do whatever you want - and the idiomatic behavior is to modify the instance! For comparison, C# lets you overload + but not +=, and automatically synthesizes the latter from the former to enforce the correct behavior.
- milliams 5y ago
  The rule in Python is that `=` creates a name binding, where as `+=` does not. That's pretty consistent as far as I can tell.
  - roelschroeven 5y ago
    The inconsistency is that for immutable objects (or more generally, objects that have __add__ but not __iadd__), '+=' does create a name binding. After 'a = 300; b = 50000', a += b is exactly the same as a = a + b In this case, a new object is created with the value of a + b which then gets bound to the name a.
  - int_19h 5y ago
    Nope, += also creates a binding! Try this: xs = [1, 2] def foo(): xs += [3] foo() You'll get an exception saying that local variable xs was used before it was assigned - precisely because += created a new local binding for xs inside foo.

radarsat1 5y ago

> (things like `.append` changing the object, returning `None` instead of creating a copy and returning that) This would be horrendously inefficient without immutable data structures like Clojure's. Very few languages have that, so it's a strange assumption to make, especially for a language as old as Python. Although it is a very nice feature of Clojure.

l33tc0der 5y ago
One of the many reasons why I love Clojure so much! Rich implement his own brand of persistent data structures which makes Clojure's immutability a lot more efficient.
- radarsat1 5y ago
  Thanks yeah, persistent is the term I was looking for, did not come to mind as I haven't used Clojure in a few years.
Blikkentrekker 5y ago
It should also be worth nothing that Clojure sacrificed quite a bit to make this as efficient as possible. “persistent vectors” are certainly an interesting data structure that strike a compromise between fast indexing and being able to relatively quickly create a copy where only one element changes, but it's a compromise and indexing is made slower to allow for the latter. — They also take up more memory on their own but are allowed to share memory with their copies. I will say that my ideal language contains them in the standard library alongside standard vectors that index in constant time. Further, it should be noted that much of the performance talk is on the assumption that accessing from memory is truly random access; — with the existence of c.p.u. caches that assumption is not entirely accurate and accessing from contiguous rather than scattered memory in practice is considerably cheaper so one also pays the price for their being scattered more in memory.
- adrusi 5y ago
  Random access into a clojure vector is going to need more memory lookups than conventional sequential buffer array (I don't recall the constants used in the implementation, I think it's either 4 or 8 lookups). But when you're indexing into the vector sequentially, the memory layout plays rather well with memory caching behavior, and most lookups are going to be in L1 cache, just like they would be in a conventional array. So lookups are a bit more expensive, but not as much more expensive as one might imagine.

Alex3917 5y ago

> things like `.append` changing the object, returning `None` instead of creating a copy and returning that The obvious question is why it can't return a reference to the list instead of returning None. I feel like if I've been using the language on an almost daily basis for ten years now and I still get burned by that all the time, then it's just a poorly designed feature.

canjobear 5y ago
The advantage of mutating operations always returning None is that you can easily tell whether a mutation is happening by looking at the code. If you see y = f(x) that means x is unchanged, whereas if you see just f(x) on a line that means something stateful is happening.
- macintux 5y ago
  There are counter-examples: functions that return values while also having side effects, cases where the developer simply made a mistake. Coming from a language with baked-in immutability, Python’s behavior in this regard was very difficult to get used to.
- brundolf 5y ago
  Agreed. JavaScript's Array.sort is an example of this. Most of JavaScript's other array methods return a new array and people get used to chaining them, but sort mutates the array and also returns a reference to it. You can actually get pretty far before being bitten by this so long as you're sorting already-copied arrays. But then one day you hit a bizarre bug caused by behavior that's been sneaking past your radar the whole time.
- Xavdidtheshadow 5y ago
  I really like ruby's method naming convention for this: * `sort(arr)` returns a sorted copy of the input * `sort!(arr)` returns the sorted original methods that return booleans end in `?`, like `arr.sorted?` It's just a convention, but it's a nice way to let the writer know what will happen.

brundolf 5y ago

> Used to see many flaws in the language (things like `.append` changing the object, returning `None` instead of creating a copy and returning that) I think it's pretty off-base to call this a "flaw". Immutable structures have their place and can be very helpful where appropriate, but making its core primitives work this way is far outside the scope or the philosophy of Python. If you want otherwise, you're really wanting an entirely different language. And there's nothing wrong with that! But I think it would be a "flaw" for Python to make these operations immutable, even though I love immutability personally.

listenallyall 5y ago

And also like a Swiss army knife, it's not particularly great at anything, can be awkward even when functional, and there's always a better tool for any specific job.