3 ms·

Nametuple is designed for backwards compatibility with tuples. It's to make an easy transition from using tuples as structs to using something with attributes,

by xapata 9y ago

Nametuple is designed for backwards compatibility with tuples. It's to make an easy transition from using tuples as structs to using something with attributes, without breaking all the code you just demonstrated.

eesmith 9y ago
Yes, that was the "one good use" I mentioned. However, in practice it's not used that way. This specific LWN article concerns the following code from functools.py: _CacheInfo = namedtuple("CacheInfo", ["hits", "misses", "maxsize", "currsize"]) It is used like this: def decorating_function(user_function): wrapper = _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo) ... def _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo): ... def cache_info(): """Report cache statistics""" with lock: return _CacheInfo(hits, misses, maxsize, cache_len()) The _lru_cache_wrapper is then replaced by a C extension in _functoolsmodule.c which only uses the cache_info_type as: static PyObject * lru_cache_cache_info(lru_cache_object *self, PyObject *unused) { return PyObject_CallFunction(self->cache_info_type, "nnOn", self->hits, self->misses, self->maxsize_O, PyDict_GET_SIZE(self->cache)); } This is basically the same code, just in C. There is no reason for this to expose a tuple API. And yet it does. >>> @functools.lru_cache(maxsize=None) ... def fib(i): ... if i <= 1: return 1 ... return fib(i-1)+fib(i-2) ... >>> for i in range(10): ... print(i, fib(i)) ... 0 1 1 1 2 2 3 3 4 5 5 8 6 13 7 21 8 34 9 55 >>> fib.cache_info() CacheInfo(hits=17, misses=10, maxsize=None, currsize=10) >>> for x in fib.cache_info(): ... print(x) ... 17 10 None 10 A spot check shows no uses of namedtuple in the Python standard library which are used to keep backwards compatibility with tuples, though I only looked at a few. So while I agree with you, it appears that the core developers do not agree with us.
- xapata 9y ago
  Quoting the code as it is doesn't show its development history. The cache_info could have been a tuple originally then updated to a namedtuple. Regardless, through conversation I am confident that at least one of the core devs thinks about namedtuple in that manner. Not as namedtuple's only purpose, but as one major benefit. Further, the core devs are not a single entity, but many people who don't always agree with each other. Edit: note Raymond's comment elsewhere on this page.
  - eesmith 9y ago
    You are correct, and I apologize for not looking into this more deeply first. For the specific case under discussion, ncoghlan added cache_info() on 30 Nov 2010 - https://github.com/python/cpython/commit/234515afe594e5f9ca1b3f460735c68b04c031e2#diff-da04cc419a12dd6f8b6f44e11a459ef https://github.com/python/cpython/commit/234515afe594e5f9ca1... Previously, "Performance statistics stored in f.cache_hits and f.cache_misses." With the change "Significant statistics (maxsize, size, hits, misses) are available through the f.cache_info() named tuple." There was no time when it was a tuple, much less in a tuple in a public release. I'll look first at the namedtuples which have been present since at least 2.7.10. The named tuples in "urllib/parse", "difflib", "inspect", "sched", and "doctest" were originally tuples. The decimal module's uses a DecimalTuple as a return value from to_tuple(), so I didn't check that history. ssl uses a DefaultVerifyPaths added by tiran on 9 Jun 2013 - https://github.com/python/cpython/commit/6d7ad13a458afdf2cbdd0c3349b0589d7776cc8a#diff-c49248c7181161e24048bec5e35ba953 https://github.com/python/cpython/commit/6d7ad13a458afdf2cbd... . It did not replace a tuple. I assume _ASN1Object is the same way. Now for the namedtuples in recent version control: dis.py:_Instruction was added in https://github.com/python/cpython/commit/b39fd0c9b8dc6683924205265ff43cc597d1dfb9#diff-84f8560ccab8ce5d2705b509e1b67c29 https://github.com/python/cpython/commit/b39fd0c9b8dc6683924... . It did not replace a tuple. aifc.py:_aifc_params replaces a tuple. crypt.py:_Method replaced a class with a namedtuple in https://github.com/python/cpython/commit/daa5799cb88667855430bb49f428a4ff6c093456#diff-b913c52417d4f87c7eeb27cdfbb91ffd https://github.com/python/cpython/commit/daa5799cb8866785543... . nntplib.py:GroupInfo and ArticleInfo added in https://github.com/python/cpython/commit/69ab95105f5105b8337c757346c899d6c7e7d9bb https://github.com/python/cpython/commit/69ab95105f5105b8337... . It did not replace a tuple. pkgutil.py:ModuleInfo replaces a tuple platform.py:uname_result replaces a tuple sndhdr.py:SndHeader replaces a tuple sunau.py:_sunau_params replaces a tuple tokenize.py:TokenInfo replaces a tuple typing.py:nm_tpl ... is above my paygrade to understand. I believe it's required to be a namedtuple since that what it's trying to implement. wave.py:_wave_params replaces a tuple. Finally, urllib/robotparser.py appears to contain a bug in the following: req_rate = collections.namedtuple('req_rate', 'requests seconds') entry.req_rate = req_rate entry.req_rate.requests = int(numbers[0]) entry.req_rate.seconds = int(numbers[1]) As I read it, this should fail as the req_rate is immutable. This appears to be written by someone who thinks that it's a lightweight object. The commit is https://github.com/python/cpython/commit/960e848f0d32399824d61dce2ff5736bb596ed44#diff-cf9fffa7c76f65305ebf434feffede7a https://github.com/python/cpython/commit/960e848f0d32399824d... . It did not replace a tuple. In conclusion, 8 of the current uses of namedtuple are not to emulate a tuple. 13 of the current uses replace a tuple.
- raymondh 9y ago
  > There is no reason for this to expose a tuple API. FWIW, that is a norm in the standard library that predates named tuples. For example, sys.version_info and time.localtime() return structseq instances. Those both have a named tuple API. The are tuple subclasses with named fields. Accordingly, they are indexable, iterable, hashable, space-efficient, and have a self-documenting __repr__().
  - eesmith 9y ago
    In my original comment I wrote "'namedtuple' has one good use - migration from a tuple object to a real object while allowing backwards compatability to the tuple API. For example, the old tuples once returned from os.stat() and time.gmtime()." The two examples you gave are examples of that good use. The time.localtime() used to return a tuple. See https://github.com/python/cpython/blob/770e4042db8a1c277b5d9b678cf6045a7aa2bd33/Modules/timemodule.c#L232 https://github.com/python/cpython/blob/770e4042db8a1c277b5d9... . The change to StructTimeType is an example of how to do that sort of migration. Similarly, sys.version_info() used to return a tuple. See https://github.com/python/cpython/blob/193a3f6d37c1440b049f7b1c62ff30c1d8cae38a/Python/sysmodule.c#L764 https://github.com/python/cpython/blob/193a3f6d37c1440b049f7... . Again, a good example of how to migrate. However, the "this" in the "There is no reason for this to expose a tuple API" which you quoted refers specifically to the namedtuple that cache_info() returns. That did not replace a tuple, it replaced two attributes, as you can see at https://github.com/python/cpython/commit/234515afe594e5f9ca1b3f460735c68b04c031e2#diff-da04cc419a12dd6f8b6f44e11a459ef https://github.com/python/cpython/commit/234515afe594e5f9ca1... . And returning a simple type with a couple of attributes while also (and needlessly) treating it as a tuple was never a norm in Python before namedtuples, if only because it was a lot of work.
    - xapata 9y ago
      > it was a lot of work Thus the desire for types.SimpleNamespace as others have noted. Though we might have better design choices today than yesterday, there's also a downside to code churn. Case in point the annoyance of trying to modernize threading from isAlive to is_alive. Even when writing new code, it's often wiser to stick with a "worse" standard than to use a new design.

by xapata 9y ago

eesmith 9y ago

Yes, that was the "one good use" I mentioned. However, in practice it's not used that way. This specific LWN article concerns the following code from functools.py: _CacheInfo = namedtuple("CacheInfo", ["hits", "misses", "maxsize", "currsize"]) It is used like this: def decorating_function(user_function): wrapper = _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo) ... def _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo): ... def cache_info(): """Report cache statistics""" with lock: return _CacheInfo(hits, misses, maxsize, cache_len()) The _lru_cache_wrapper is then replaced by a C extension in _functoolsmodule.c which only uses the cache_info_type as: static PyObject * lru_cache_cache_info(lru_cache_object *self, PyObject *unused) { return PyObject_CallFunction(self->cache_info_type, "nnOn", self->hits, self->misses, self->maxsize_O, PyDict_GET_SIZE(self->cache)); } This is basically the same code, just in C. There is no reason for this to expose a tuple API. And yet it does. >>> @functools.lru_cache(maxsize=None) ... def fib(i): ... if i <= 1: return 1 ... return fib(i-1)+fib(i-2) ... >>> for i in range(10): ... print(i, fib(i)) ... 0 1 1 1 2 2 3 3 4 5 5 8 6 13 7 21 8 34 9 55 >>> fib.cache_info() CacheInfo(hits=17, misses=10, maxsize=None, currsize=10) >>> for x in fib.cache_info(): ... print(x) ... 17 10 None 10 A spot check shows no uses of namedtuple in the Python standard library which are used to keep backwards compatibility with tuples, though I only looked at a few. So while I agree with you, it appears that the core developers do not agree with us.

xapata 9y ago
Quoting the code as it is doesn't show its development history. The cache_info could have been a tuple originally then updated to a namedtuple. Regardless, through conversation I am confident that at least one of the core devs thinks about namedtuple in that manner. Not as namedtuple's only purpose, but as one major benefit. Further, the core devs are not a single entity, but many people who don't always agree with each other. Edit: note Raymond's comment elsewhere on this page.
- eesmith 9y ago
  You are correct, and I apologize for not looking into this more deeply first. For the specific case under discussion, ncoghlan added cache_info() on 30 Nov 2010 - https://github.com/python/cpython/commit/234515afe594e5f9ca1b3f460735c68b04c031e2#diff-da04cc419a12dd6f8b6f44e11a459ef https://github.com/python/cpython/commit/234515afe594e5f9ca1... Previously, "Performance statistics stored in f.cache_hits and f.cache_misses." With the change "Significant statistics (maxsize, size, hits, misses) are available through the f.cache_info() named tuple." There was no time when it was a tuple, much less in a tuple in a public release. I'll look first at the namedtuples which have been present since at least 2.7.10. The named tuples in "urllib/parse", "difflib", "inspect", "sched", and "doctest" were originally tuples. The decimal module's uses a DecimalTuple as a return value from to_tuple(), so I didn't check that history. ssl uses a DefaultVerifyPaths added by tiran on 9 Jun 2013 - https://github.com/python/cpython/commit/6d7ad13a458afdf2cbdd0c3349b0589d7776cc8a#diff-c49248c7181161e24048bec5e35ba953 https://github.com/python/cpython/commit/6d7ad13a458afdf2cbd... . It did not replace a tuple. I assume _ASN1Object is the same way. Now for the namedtuples in recent version control: dis.py:_Instruction was added in https://github.com/python/cpython/commit/b39fd0c9b8dc6683924205265ff43cc597d1dfb9#diff-84f8560ccab8ce5d2705b509e1b67c29 https://github.com/python/cpython/commit/b39fd0c9b8dc6683924... . It did not replace a tuple. aifc.py:_aifc_params replaces a tuple. crypt.py:_Method replaced a class with a namedtuple in https://github.com/python/cpython/commit/daa5799cb88667855430bb49f428a4ff6c093456#diff-b913c52417d4f87c7eeb27cdfbb91ffd https://github.com/python/cpython/commit/daa5799cb8866785543... . nntplib.py:GroupInfo and ArticleInfo added in https://github.com/python/cpython/commit/69ab95105f5105b8337c757346c899d6c7e7d9bb https://github.com/python/cpython/commit/69ab95105f5105b8337... . It did not replace a tuple. pkgutil.py:ModuleInfo replaces a tuple platform.py:uname_result replaces a tuple sndhdr.py:SndHeader replaces a tuple sunau.py:_sunau_params replaces a tuple tokenize.py:TokenInfo replaces a tuple typing.py:nm_tpl ... is above my paygrade to understand. I believe it's required to be a namedtuple since that what it's trying to implement. wave.py:_wave_params replaces a tuple. Finally, urllib/robotparser.py appears to contain a bug in the following: req_rate = collections.namedtuple('req_rate', 'requests seconds') entry.req_rate = req_rate entry.req_rate.requests = int(numbers[0]) entry.req_rate.seconds = int(numbers[1]) As I read it, this should fail as the req_rate is immutable. This appears to be written by someone who thinks that it's a lightweight object. The commit is https://github.com/python/cpython/commit/960e848f0d32399824d61dce2ff5736bb596ed44#diff-cf9fffa7c76f65305ebf434feffede7a https://github.com/python/cpython/commit/960e848f0d32399824d... . It did not replace a tuple. In conclusion, 8 of the current uses of namedtuple are not to emulate a tuple. 13 of the current uses replace a tuple.
raymondh 9y ago
> There is no reason for this to expose a tuple API. FWIW, that is a norm in the standard library that predates named tuples. For example, sys.version_info and time.localtime() return structseq instances. Those both have a named tuple API. The are tuple subclasses with named fields. Accordingly, they are indexable, iterable, hashable, space-efficient, and have a self-documenting __repr__().
- eesmith 9y ago
  In my original comment I wrote "'namedtuple' has one good use - migration from a tuple object to a real object while allowing backwards compatability to the tuple API. For example, the old tuples once returned from os.stat() and time.gmtime()." The two examples you gave are examples of that good use. The time.localtime() used to return a tuple. See https://github.com/python/cpython/blob/770e4042db8a1c277b5d9b678cf6045a7aa2bd33/Modules/timemodule.c#L232 https://github.com/python/cpython/blob/770e4042db8a1c277b5d9... . The change to StructTimeType is an example of how to do that sort of migration. Similarly, sys.version_info() used to return a tuple. See https://github.com/python/cpython/blob/193a3f6d37c1440b049f7b1c62ff30c1d8cae38a/Python/sysmodule.c#L764 https://github.com/python/cpython/blob/193a3f6d37c1440b049f7... . Again, a good example of how to migrate. However, the "this" in the "There is no reason for this to expose a tuple API" which you quoted refers specifically to the namedtuple that cache_info() returns. That did not replace a tuple, it replaced two attributes, as you can see at https://github.com/python/cpython/commit/234515afe594e5f9ca1b3f460735c68b04c031e2#diff-da04cc419a12dd6f8b6f44e11a459ef https://github.com/python/cpython/commit/234515afe594e5f9ca1... . And returning a simple type with a couple of attributes while also (and needlessly) treating it as a tuple was never a norm in Python before namedtuples, if only because it was a lot of work.
  - xapata 9y ago
    > it was a lot of work Thus the desire for types.SimpleNamespace as others have noted. Though we might have better design choices today than yesterday, there's also a downside to code churn. Case in point the annoyance of trying to modernize threading from isAlive to is_alive. Even when writing new code, it's often wiser to stick with a "worse" standard than to use a new design.