4 ms·

Related to that, do heterogenous lists make sense as a default? I can see the argument for dictionaries easily, you might want to put a lot of different stuff i

by Zababa 5y ago

Related to that, do heterogenous lists make sense as a default? I can see the argument for dictionaries easily, you might want to put a lot of different stuff inside one. However, for lists, I fail to see the advantages of heterogenous lists as a default. Especially since they are going to be way slower, and nobody seem to use arrays in Python.

pdonis 5y ago
> do heterogenous lists make sense as a default? Yes, because then the list object doesn't have to do any extra work to check the types of objects. If your application needs that extra checking, you can add it in your application's code; but there's no reason to burden the basic list object with extra code that not everyone needs. > Especially since they are going to be way slower Way slower than what? They're slower than Python arrays, of course, but Python arrays can only store numeric values (since the amount of space each value takes up is constant and known when the array is created). For a container that needs to be able to store objects of any type (including instances of user-defined classes), a heterogeneous list is faster than a homogeneous one, because, as above, it doesn't have to do extra work to check the types of objects. All the other extra work is there because the size in memory of the objects isn't known in advance; and that will be true even if all of the objects are of the same type, since instances still won't necessarily all have the same in-memory size (for example, consider instances of user-defined classes with different attributes).
- Zababa 5y ago
  > For a container that needs to be able to store objects of any type (including instances of user-defined classes), a heterogeneous list is faster than a homogeneous one, because, as above, it doesn't have to do extra work to check the types of objects. My question is about that. Is storing objects of any type a common use case, compared to storing objects of one type?
  - dragonwriter 5y ago
    Storing objects of any size is common, because Python objects are rarely fixed size (Python integers aren't fixed size!) At that point, you need pointers anyway, so mixed type doesn't add much overhead. Especially given that you need to handle mixed concrete types even if you have single “typechecking” types because of inheritance (including via declared ABCs and union types.
  - sseagull 5y ago
    I have found, at least in my own use of python, lists are for "homogeneous" collections of indeterminate length. But these are not strictly homogeneous - there may be None entries, for example. So I do use the heterogeneous capabilities. For heterogeneous collections with a fixed length, I tend to use a tuple. That makes more sense to me, conceptually (coming from C++).
    - joconde 5y ago
      In Python’s type annotation system, “class T or None” is represented by “typing.Optional[T]”. I think C++ has something similar for STL containers. Even numpy has NaN values. So your usage still fits some reasonable definition of homogeneous :)
  - pdonis 5y ago
    > Is storing objects of any type a common use case, compared to storing objects of one type? I'm not sure it's possible to know a general answer to that, since Python is used in such a wide variety of contexts. In my own programming, I use lists that can contain objects of different types often enough for that use case to be significant for me. One other thing to keep in mind is that Python has duck typing, so, for example, if you are iterating over a list and retrieving the same attribute or calling the same method on each object, that doesn't require each object to be of the same type, just to have an attribute with the required name, or a method with the required name and calling signature.
kzrdude 5y ago
If you use the static typing eyes, the Python list is homogenous: every element is of the type reference to python object. There's no funny business with types, just one reference after another.
- Zababa 5y ago
  That's a fair point, but if static typing was like that it would be useless. "Object" is a type, but it's pretty useless most of the time.
duckerude 5y ago
Python's semantics make homogeneity less useful and hard to enforce. Everything's passed by reference, so the list has to contain pointers even if you know all its elements have the same type. And some objects can change their type by assigning to __class__, which can happen from a distance (because everything is by reference). A careful implementation can solve this for some builtin types. PyPy does it for ints: if a list only contains ints it's represented as a proper numeric array. But then as soon as you append some other type the whole list is converted to a more expensive more traditional representation. PyPy's ints still have to pretend to be heap-allocated objects at all times. In particular they have to do something tricky with the id() function, which on CPython just returns an object's memory address. The return value has to be unique and consistent for as long as an object lives.
dragonwriter 5y ago
> Especially since they are going to be way slower, and nobody seem to use arrays in Python. Arrays are widely used in Python for serious work (mostly NumPy arrays, either directly or via higher-level libraries, not the stdlib ones.)

by Zababa 5y ago

pdonis 5y ago

> do heterogenous lists make sense as a default? Yes, because then the list object doesn't have to do any extra work to check the types of objects. If your application needs that extra checking, you can add it in your application's code; but there's no reason to burden the basic list object with extra code that not everyone needs. > Especially since they are going to be way slower Way slower than what? They're slower than Python arrays, of course, but Python arrays can only store numeric values (since the amount of space each value takes up is constant and known when the array is created). For a container that needs to be able to store objects of any type (including instances of user-defined classes), a heterogeneous list is faster than a homogeneous one, because, as above, it doesn't have to do extra work to check the types of objects. All the other extra work is there because the size in memory of the objects isn't known in advance; and that will be true even if all of the objects are of the same type, since instances still won't necessarily all have the same in-memory size (for example, consider instances of user-defined classes with different attributes).

Zababa 5y ago
> For a container that needs to be able to store objects of any type (including instances of user-defined classes), a heterogeneous list is faster than a homogeneous one, because, as above, it doesn't have to do extra work to check the types of objects. My question is about that. Is storing objects of any type a common use case, compared to storing objects of one type?
- dragonwriter 5y ago
  Storing objects of any size is common, because Python objects are rarely fixed size (Python integers aren't fixed size!) At that point, you need pointers anyway, so mixed type doesn't add much overhead. Especially given that you need to handle mixed concrete types even if you have single “typechecking” types because of inheritance (including via declared ABCs and union types.
- sseagull 5y ago
  I have found, at least in my own use of python, lists are for "homogeneous" collections of indeterminate length. But these are not strictly homogeneous - there may be None entries, for example. So I do use the heterogeneous capabilities. For heterogeneous collections with a fixed length, I tend to use a tuple. That makes more sense to me, conceptually (coming from C++).
  - joconde 5y ago
    In Python’s type annotation system, “class T or None” is represented by “typing.Optional[T]”. I think C++ has something similar for STL containers. Even numpy has NaN values. So your usage still fits some reasonable definition of homogeneous :)
- pdonis 5y ago
  > Is storing objects of any type a common use case, compared to storing objects of one type? I'm not sure it's possible to know a general answer to that, since Python is used in such a wide variety of contexts. In my own programming, I use lists that can contain objects of different types often enough for that use case to be significant for me. One other thing to keep in mind is that Python has duck typing, so, for example, if you are iterating over a list and retrieving the same attribute or calling the same method on each object, that doesn't require each object to be of the same type, just to have an attribute with the required name, or a method with the required name and calling signature.

kzrdude 5y ago

If you use the static typing eyes, the Python list is homogenous: every element is of the type reference to python object. There's no funny business with types, just one reference after another.

Zababa 5y ago
That's a fair point, but if static typing was like that it would be useless. "Object" is a type, but it's pretty useless most of the time.

duckerude 5y ago

Python's semantics make homogeneity less useful and hard to enforce. Everything's passed by reference, so the list has to contain pointers even if you know all its elements have the same type. And some objects can change their type by assigning to __class__, which can happen from a distance (because everything is by reference). A careful implementation can solve this for some builtin types. PyPy does it for ints: if a list only contains ints it's represented as a proper numeric array. But then as soon as you append some other type the whole list is converted to a more expensive more traditional representation. PyPy's ints still have to pretend to be heap-allocated objects at all times. In particular they have to do something tricky with the id() function, which on CPython just returns an object's memory address. The return value has to be unique and consistent for as long as an object lives.

dragonwriter 5y ago

> Especially since they are going to be way slower, and nobody seem to use arrays in Python. Arrays are widely used in Python for serious work (mostly NumPy arrays, either directly or via higher-level libraries, not the stdlib ones.)