5 ms·

I believe this is just a typing shorthand for `collections.namedtuple`: https://docs.python.org/3/library/typing.html?highlight=namedtuple https://docs.python.

I believe this is just a typing shorthand for `collections.namedtuple`: https://docs.python.org/3/library/typing.html?highlight=namedtuple https://docs.python.org/3/library/typing.html?highlight=name... Last I checked `collections.namedtuple` had some infamous performance implications (although I vaguely recall some discussion about changing it's implementation): https://stackoverflow.com/questions/2646157/what-is-the-fastest-to-access-struct-like-object-in-python https://stackoverflow.com/questions/2646157/what-is-the-fast... I believe that `collections.namedtuple` is beneficial for memory consumption [1], but you pay a cost when accessing members by `__getitem__`. In general, judging by timings of the StackOverflow above, I think it is better to use `__slots__` in most cases than `namedtuple`. [1] http://blog.explainmydata.com/2012/07/expensive-lessons-in-python-performance.html http://blog.explainmydata.com/2012/07/expensive-lessons-in-p... Edit: Also, fun fact, is that namedtuple does a clever bit of metaprogramming where a string class definition is formatted and passed to `exec`: https://hg.python.org/cpython/file/b14308524cff/Lib/collections/__init__.py#l232 https://hg.python.org/cpython/file/b14308524cff/Lib/collecti...

gipp 9y ago
That post is from 2010, before even 2.7 was released. It is extremely out of date. Re-running the same tests I get: In [1]: from typing import NamedTuple In [2]: class A(NamedTuple): ...: a = 1 ...: b = 2 ...: c = 3 ...: In [3]: a = A() In [4]: class B: ...: __slots__ = ('a', 'b', 'c') ...: def __init__(self, a=1, b=2, c=3): ...: self.a = a ...: self.b = b ...: self.c = c ...: In [5]: b = B() In [6]: c = dict(a=1, b=2, c=3) In [7]: d = (1,2,3) In [8]: e = [1,2,3] In [9]: key = 2 In [10]: %timeit z = a.c 38.2 ns ± 0.07 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [11]: %timeit z = b.c 48.1 ns ± 0.0461 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [12]: %timeit z = c['c'] 38 ns ± 0.062 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [13]: %timeit z = d[2] 38.8 ns ± 0.0425 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [14]: %timeit z = e[2] 39.8 ns ± 0.0641 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [15]: %timeit z = d[key] 48.6 ns ± 0.0713 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [16]: %timeit z = e[key] 49.4 ns ± 0.0369 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) Namedtuple looks faster than slots here. And for comparison's sake: In [17]: class F: ...: def __init__(self, a=1, b=2, c=3): ...: self.a = a ...: self.b = b ...: self.c = c ...: In [18]: f = F() In [19]: %timeit z = f.c 53.7 ns ± 0.265 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
- gshulegaard 9y ago
  The question was asked in 2010, but the benchmark post I was referencing is from 2014. It doesn't appear as if `namedtuple` has changed much: https://hg.python.org/cpython/file/tip/Lib/collections/__init__.py#l302 https://hg.python.org/cpython/file/tip/Lib/collections/__ini... Running the same tests from the post, I get similar results: from collections import namedtuple STest = namedtuple("TEST", "a b c") a = STest(a=1,b=2,c=3) class Test(object): __slots__ = ["a","b","c"] a=1 b=2 c=3 b = Test() c = {'a':1, 'b':2, 'c':3} d = (1,2,3) e = [1,2,3] f = (1,2,3) g = [1,2,3] key = 2 if __name__ == '__main__': from timeit import timeit print("Named tuple with a, b, c:") print(timeit("z = a.c", "from __main__ import a")) print("Named tuple, using index:") print(timeit("z = a[2]", "from __main__ import a")) print("Class using __slots__, with a, b, c:") print(timeit("z = b.c", "from __main__ import b")) print("Dictionary with keys a, b, c:") print(timeit("z = c['c']", "from __main__ import c")) print("Tuple with three values, using a constant key:") print(timeit("z = d[2]", "from __main__ import d")) print("List with three values, using a constant key:") print(timeit("z = e[2]", "from __main__ import e")) print("Tuple with three values, using a local key:") print(timeit("z = d[key]", "from __main__ import d, key")) print("List with three values, using a local key:") print(timeit("z = e[key]", "from __main__ import e, key")) Output: Named tuple with a, b, c: 0.147349834442 Named tuple, using index: 0.0364408493042 Class using __slots__, with a, b, c: 0.0334758758545 Dictionary with keys a, b, c: 0.0339260101318 Tuple with three values, using a constant key: 0.0425179004669 List with three values, using a constant key: 0.0301761627197 Tuple with three values, using a local key: 0.0436310768127 List with three values, using a local key: 0.0309669971466
  - aaronchall 9y ago
    I just noticed your slotted object uses class variables. That is completely wrong - so your analysis is probably wrong. In Python, I think it's important to do what's directly semantically correct. However, we can compare apples and oranges on what they have in common, so here's a bit of my own analysis: >>> from timeit import repeat >>> from collections import namedtuple >>> class Slotted: __slots__ = 'a', 'b', 'c', 'd', 'e' ... >>> NT = namedtuple('NT', 'a b c d e') >>> >>> nt = NT(1,2,3,4,5) >>> s = Slotted() >>> s.a, s.b, s.c, s.d, s.e = 1,2,3,4,5 >>> min(repeat(lambda: (s.a, s.b, s.c, s.d, s.e))) 0.2850431999977445 >>> min(repeat(lambda: (nt.a, nt.b, nt.c, nt.d, nt.e))) 0.418955763001577 I'm showing the namedtuple to be a little slower, but I think the thing to remember is to do the semantically correct thing first. If you've met your technical requirements, you're done. Only if you're too slow, then you look for ways to optimize for bottlenecks.
ameliaquining 9y ago
namedtuple's startup performance is terrible, because formatting a string class definition and passing it to exec is slow. The controversy (https://lwn.net/Articles/730915/ https://lwn.net/Articles/730915/) was about whether startup performance is worth optimizing (the decision was yes and namedtuple is apparently going to be rewritten in C). Instances of namedtuple are plenty fast.
- gshulegaard 9y ago
  Startup performance is understandably terrible, the bit of clever metaprogramming I reference above is the reason. I am talking about member access as far as performance goes. The references I link I think go into better detail than I can.

gipp 9y ago

That post is from 2010, before even 2.7 was released. It is extremely out of date. Re-running the same tests I get: In [1]: from typing import NamedTuple In [2]: class A(NamedTuple): ...: a = 1 ...: b = 2 ...: c = 3 ...: In [3]: a = A() In [4]: class B: ...: __slots__ = ('a', 'b', 'c') ...: def __init__(self, a=1, b=2, c=3): ...: self.a = a ...: self.b = b ...: self.c = c ...: In [5]: b = B() In [6]: c = dict(a=1, b=2, c=3) In [7]: d = (1,2,3) In [8]: e = [1,2,3] In [9]: key = 2 In [10]: %timeit z = a.c 38.2 ns ± 0.07 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [11]: %timeit z = b.c 48.1 ns ± 0.0461 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [12]: %timeit z = c['c'] 38 ns ± 0.062 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [13]: %timeit z = d[2] 38.8 ns ± 0.0425 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [14]: %timeit z = e[2] 39.8 ns ± 0.0641 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [15]: %timeit z = d[key] 48.6 ns ± 0.0713 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [16]: %timeit z = e[key] 49.4 ns ± 0.0369 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) Namedtuple looks faster than slots here. And for comparison's sake: In [17]: class F: ...: def __init__(self, a=1, b=2, c=3): ...: self.a = a ...: self.b = b ...: self.c = c ...: In [18]: f = F() In [19]: %timeit z = f.c 53.7 ns ± 0.265 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

gshulegaard 9y ago
The question was asked in 2010, but the benchmark post I was referencing is from 2014. It doesn't appear as if `namedtuple` has changed much: https://hg.python.org/cpython/file/tip/Lib/collections/__init__.py#l302 https://hg.python.org/cpython/file/tip/Lib/collections/__ini... Running the same tests from the post, I get similar results: from collections import namedtuple STest = namedtuple("TEST", "a b c") a = STest(a=1,b=2,c=3) class Test(object): __slots__ = ["a","b","c"] a=1 b=2 c=3 b = Test() c = {'a':1, 'b':2, 'c':3} d = (1,2,3) e = [1,2,3] f = (1,2,3) g = [1,2,3] key = 2 if __name__ == '__main__': from timeit import timeit print("Named tuple with a, b, c:") print(timeit("z = a.c", "from __main__ import a")) print("Named tuple, using index:") print(timeit("z = a[2]", "from __main__ import a")) print("Class using __slots__, with a, b, c:") print(timeit("z = b.c", "from __main__ import b")) print("Dictionary with keys a, b, c:") print(timeit("z = c['c']", "from __main__ import c")) print("Tuple with three values, using a constant key:") print(timeit("z = d[2]", "from __main__ import d")) print("List with three values, using a constant key:") print(timeit("z = e[2]", "from __main__ import e")) print("Tuple with three values, using a local key:") print(timeit("z = d[key]", "from __main__ import d, key")) print("List with three values, using a local key:") print(timeit("z = e[key]", "from __main__ import e, key")) Output: Named tuple with a, b, c: 0.147349834442 Named tuple, using index: 0.0364408493042 Class using __slots__, with a, b, c: 0.0334758758545 Dictionary with keys a, b, c: 0.0339260101318 Tuple with three values, using a constant key: 0.0425179004669 List with three values, using a constant key: 0.0301761627197 Tuple with three values, using a local key: 0.0436310768127 List with three values, using a local key: 0.0309669971466
- aaronchall 9y ago
  I just noticed your slotted object uses class variables. That is completely wrong - so your analysis is probably wrong. In Python, I think it's important to do what's directly semantically correct. However, we can compare apples and oranges on what they have in common, so here's a bit of my own analysis: >>> from timeit import repeat >>> from collections import namedtuple >>> class Slotted: __slots__ = 'a', 'b', 'c', 'd', 'e' ... >>> NT = namedtuple('NT', 'a b c d e') >>> >>> nt = NT(1,2,3,4,5) >>> s = Slotted() >>> s.a, s.b, s.c, s.d, s.e = 1,2,3,4,5 >>> min(repeat(lambda: (s.a, s.b, s.c, s.d, s.e))) 0.2850431999977445 >>> min(repeat(lambda: (nt.a, nt.b, nt.c, nt.d, nt.e))) 0.418955763001577 I'm showing the namedtuple to be a little slower, but I think the thing to remember is to do the semantically correct thing first. If you've met your technical requirements, you're done. Only if you're too slow, then you look for ways to optimize for bottlenecks.

ameliaquining 9y ago

namedtuple's startup performance is terrible, because formatting a string class definition and passing it to exec is slow. The controversy (https://lwn.net/Articles/730915/ https://lwn.net/Articles/730915/) was about whether startup performance is worth optimizing (the decision was yes and namedtuple is apparently going to be rewritten in C). Instances of namedtuple are plenty fast.

gshulegaard 9y ago
Startup performance is understandably terrible, the bit of clever metaprogramming I reference above is the reason. I am talking about member access as far as performance goes. The references I link I think go into better detail than I can.