3 ms·
The question was asked in 2010, but the benchmark post I was referencing is from 2014. It doesn't appear as if `namedtuple` has changed much: https://hg.pytho
by gshulegaard 9y ago
The question was asked in 2010, but the benchmark post I was referencing is from 2014.
It doesn't appear as if `namedtuple` has changed much:
https://hg.python.org/cpython/file/tip/Lib/collections/__init__.py#l302 https://hg.python.org/cpython/file/tip/Lib/collections/__ini...
Running the same tests from the post, I get similar results:
from collections import namedtuple
STest = namedtuple("TEST", "a b c")
a = STest(a=1,b=2,c=3)
class Test(object):
__slots__ = ["a","b","c"]
a=1
b=2
c=3
b = Test()
c = {'a':1, 'b':2, 'c':3}
d = (1,2,3)
e = [1,2,3]
f = (1,2,3)
g = [1,2,3]
key = 2
if __name__ == '__main__':
from timeit import timeit
print("Named tuple with a, b, c:")
print(timeit("z = a.c", "from __main__ import a"))
print("Named tuple, using index:")
print(timeit("z = a[2]", "from __main__ import a"))
print("Class using __slots__, with a, b, c:")
print(timeit("z = b.c", "from __main__ import b"))
print("Dictionary with keys a, b, c:")
print(timeit("z = c['c']", "from __main__ import c"))
print("Tuple with three values, using a constant key:")
print(timeit("z = d[2]", "from __main__ import d"))
print("List with three values, using a constant key:")
print(timeit("z = e[2]", "from __main__ import e"))
print("Tuple with three values, using a local key:")
print(timeit("z = d[key]", "from __main__ import d, key"))
print("List with three values, using a local key:")
print(timeit("z = e[key]", "from __main__ import e, key"))
Output:
Named tuple with a, b, c:
0.147349834442
Named tuple, using index:
0.0364408493042
Class using __slots__, with a, b, c:
0.0334758758545
Dictionary with keys a, b, c:
0.0339260101318
Tuple with three values, using a constant key:
0.0425179004669
List with three values, using a constant key:
0.0301761627197
Tuple with three values, using a local key:
0.0436310768127
List with three values, using a local key:
0.0309669971466
- aaronchall 9y agoI just noticed your slotted object uses class variables. That is completely wrong - so your analysis is probably wrong. In Python, I think it's important to do what's directly semantically correct. However, we can compare apples and oranges on what they have in common, so here's a bit of my own analysis: >>> from timeit import repeat >>> from collections import namedtuple >>> class Slotted: __slots__ = 'a', 'b', 'c', 'd', 'e' ... >>> NT = namedtuple('NT', 'a b c d e') >>> >>> nt = NT(1,2,3,4,5) >>> s = Slotted() >>> s.a, s.b, s.c, s.d, s.e = 1,2,3,4,5 >>> min(repeat(lambda: (s.a, s.b, s.c, s.d, s.e))) 0.2850431999977445 >>> min(repeat(lambda: (nt.a, nt.b, nt.c, nt.d, nt.e))) 0.418955763001577 I'm showing the namedtuple to be a little slower, but I think the thing to remember is to do the semantically correct thing first. If you've met your technical requirements, you're done. Only if you're too slow, then you look for ways to optimize for bottlenecks.
- gshulegaard 9y agoI was just copying the benchmark I was quoting as a sanity check since I was just trying to validate references. I also feel compelled to point out that __slots__ changes the way objects are initialized [1]: > By default, instances of both old and new-style classes have a dictionary for attribute storage. This wastes space for objects having very few instance variables. The space consumption can become acute when creating large numbers of instances. > > The default can be overridden by defining __slots__ in a new-style class definition. The __slots__ declaration takes a sequence of instance variables and reserves just enough space in each instance to hold a value for each variable. Space is saved because __dict__ is not created for each instance. Which means that, in actuality, simply defining __slots__ changes the nature of the object. Quick trial demonstrates this: >>> class mySlottedObj(object): ... __slots__ = ('a', 'b') ... c = 1 ... >>> x = mySlottedObj() >>> x <mySlottedObj object at 0x10e6a3510> >>> x.c 1 >>> x.__dict__ Traceback (most recent call last): File "<input>", line 1, in <module> AttributeError: 'mySlottedObj' object has no attribute '__dict__' >>> x.__weakref__ Traceback (most recent call last): File "<input>", line 1, in <module> AttributeError: 'mySlottedObj' object has no attribute '__weakref__' >>> x.a = 1 >>> x.c = 2 Traceback (most recent call last): File "<input>", line 1, in <module> AttributeError: 'mySlottedObj' object attribute 'c' is read-only The benefit of namedtuples, in the context of Python is reducing the object footprint to being no greater than a tuple. The performance hit, which I am trying to demonstrate, is in lookup and is shown in the source (linked above). _field_template = '''\ {name} = _property(_itemgetter({index:d}), doc='Alias for field number {index:d}') ''' As I understand it this means when you access using __getitem__, a named tuple first maps to a property that then calls a __getitem__ using the aliased index on the base `tuple` object that `namedtuple` is storing. But at any rate, the benchmark script was not mine and refining the test to be as close to the same code path is welcome. And I wholeheartedly agree: > Only if you're too slow, then you look for ways to optimize for bottlenecks. I was merely responding to the original assertion: > This provides the same performance benefits, and is very similar in a lot of ways to a Scala case class -- aside from removing the boilerplate of stating each attribute twice in __init__ and again in __slots__, And I think at this point it is quite clear that namedtuple has a bit more caveats than just "removing the boilerplate" of __slots__.