4 ms·

attrs also has a feature that dataclasses don't currently [0]: an easy way to use slots [1]. It cuts down on the per-instance memory overhead, for cases wh

attrs also has a feature that dataclasses don't currently [0]: an easy way to use __slots__ [1]. It cuts down on the per-instance memory overhead, for cases where you're creating a ton of these objects. It can be useful even when not memory-constrained, because it will throw AttributeError, rather than succeeding silently, if you make a typo when assigning to an object attribute. 0: https://www.python.org/dev/peps/pep-0557/#support-for-automatically-setting-slots https://www.python.org/dev/peps/pep-0557/#support-for-automa... 1: http://www.attrs.org/en/stable/examples.html#slots http://www.attrs.org/en/stable/examples.html#slots

kstrauser 8y ago
Does that still matter now that PEP 412 (https://www.python.org/dev/peps/pep-0412/ https://www.python.org/dev/peps/pep-0412/) is implemented in Python 3.3 and newer?
- xapata 8y ago
  No, I wouldn't bother with __slots__ in 3.7, especially with the newly optimized dict.
  - markrwilliams 8y ago
    PEP 412 makes __dict__s more memory efficient than they were before, but not more efficient than no __dict__, which is the point of __slots__. The following program demonstrates the difference. Note that it lowers the available address space to 1GB so that memory exhaustion occurs sooner, and thus only works on UNIX-like systems that provide the resource module. import resource import sys class WithoutSlots: def __init__(self, a, b): self.a = a self.b = b class WithSlots: __slots__ = ('a', 'b') def __init__(self, a, b): self.a = a self.b = b resource.setrlimit(resource.RLIMIT_AS, (1024 ** 3, 1024 ** 3)) cls = WithSlots if sys.argv[1:] == ['slots'] else WithoutSlots count, instances = 0, [] while True: try: instances.append(cls(1, 2)) except MemoryError: break count = len(instances) del instances print(cls, count) Here are numbers from my laptop: $ python3.6 /tmp/slots.py <class '__main__.WithoutSlots'> 5830382 $ python3.6 /tmp/slots.py slots <class '__main__.WithSlots'> 16081964 That's almost 3x more instances with __slots__! This isn't the case with PyPy, though, thanks to a more efficient representation of objects: https://morepypy.blogspot.com/2010/11/efficiently-implementing-python-objects.html https://morepypy.blogspot.com/2010/11/efficiently-implementi...
    - xapata 8y ago
      That's a silly example. If you're making billions of integers, use NumPy. If it's just one pass, use a generator. If you're making lots of objects with more interesting attributes, the attribute storage will overwhelm the difference the instance dicts make. My point was not that __slots__ does nothing, but that there are more important things to worry about.
      - marvy 8y ago
        Suppose I want to run algorithms on large arrays of 2D points while maximizing readability. I want to store the x and y coordinates using Python integers so I don't have to worry about overflow errors, but I expect that most of the time the numbers will be small and this is "just in case". I claim that in this case, __slots__ is exactly the right thing to worry about.
        xapata 8y ago
        It's hard for me to imagine that situation coming up, but yes, __slots__ does indeed have a purpose. BTW, have you considered using the complex type to handle that for you? It's 2d and ints should be safe in float representation. If it overflows it'll crash nicely.
        marvy 8y ago
        Good one. But let's say I want something mutable, so complex won't do.
    - kstrauser 8y ago
      That's an interesting example, and thanks for demonstrating it with a modern version! I definitely wouldn't have expected that result.

attrs also has a feature that dataclasses don't currently [0]: an easy way to use __slots__ [1]. It cuts down on the per-instance memory overhead, for cases wh

kstrauser 8y ago

Does that still matter now that PEP 412 (https://www.python.org/dev/peps/pep-0412/ https://www.python.org/dev/peps/pep-0412/) is implemented in Python 3.3 and newer?

xapata 8y ago
No, I wouldn't bother with __slots__ in 3.7, especially with the newly optimized dict.
- markrwilliams 8y ago
  PEP 412 makes __dict__s more memory efficient than they were before, but not more efficient than no __dict__, which is the point of __slots__. The following program demonstrates the difference. Note that it lowers the available address space to 1GB so that memory exhaustion occurs sooner, and thus only works on UNIX-like systems that provide the resource module. import resource import sys class WithoutSlots: def __init__(self, a, b): self.a = a self.b = b class WithSlots: __slots__ = ('a', 'b') def __init__(self, a, b): self.a = a self.b = b resource.setrlimit(resource.RLIMIT_AS, (1024 ** 3, 1024 ** 3)) cls = WithSlots if sys.argv[1:] == ['slots'] else WithoutSlots count, instances = 0, [] while True: try: instances.append(cls(1, 2)) except MemoryError: break count = len(instances) del instances print(cls, count) Here are numbers from my laptop: $ python3.6 /tmp/slots.py <class '__main__.WithoutSlots'> 5830382 $ python3.6 /tmp/slots.py slots <class '__main__.WithSlots'> 16081964 That's almost 3x more instances with __slots__! This isn't the case with PyPy, though, thanks to a more efficient representation of objects: https://morepypy.blogspot.com/2010/11/efficiently-implementing-python-objects.html https://morepypy.blogspot.com/2010/11/efficiently-implementi...
  - xapata 8y ago
    That's a silly example. If you're making billions of integers, use NumPy. If it's just one pass, use a generator. If you're making lots of objects with more interesting attributes, the attribute storage will overwhelm the difference the instance dicts make. My point was not that __slots__ does nothing, but that there are more important things to worry about.
    - marvy 8y ago
      Suppose I want to run algorithms on large arrays of 2D points while maximizing readability. I want to store the x and y coordinates using Python integers so I don't have to worry about overflow errors, but I expect that most of the time the numbers will be small and this is "just in case". I claim that in this case, __slots__ is exactly the right thing to worry about.
      - xapata 8y ago
        It's hard for me to imagine that situation coming up, but yes, __slots__ does indeed have a purpose. BTW, have you considered using the complex type to handle that for you? It's 2d and ints should be safe in float representation. If it overflows it'll crash nicely.
        marvy 8y ago
        Good one. But let's say I want something mutable, so complex won't do.
  - kstrauser 8y ago
    That's an interesting example, and thanks for demonstrating it with a modern version! I definitely wouldn't have expected that result.