7 ms·
Python: The Dictionary Playbook
- nu2ycombinator 14y agoThis is ... wait for it.. Legendary. :)
- danielwozniak 14y agoNice! Some good information here. Good for beginners, good for intermediates, some expert will probably say it's good for them too.
- elbear 14y agoYou mentioned it a bit, but I want to make it clear that, even if you don't use 2.7, you can still count by doing: from collections import defaultdict counter = defaultdict(int) There is a difference though, because you have to count manually, i.e: for i in 'supercalifragilisticexpialidocius': counter[i] += 1 Also, because defaultdict accepts any callable, you can have a dict of counters by doing: counters = defaultdict(lambda: defaultdict(int)) for word in ['apple', 'berry', 'grape']: for letter in word: counters[word][letter] += 1 This is not very obvious, so I don't use it a lot, but sometimes it's the most elegant solution.
- icebraining 14y agoThere's also an implementation of Counter for 2.5+ on ActiveState: http://code.activestate.com/recipes/576611-counter-class/ http://code.activestate.com/recipes/576611-counter-class/
- ciupicri 14y agoFor Python 2.7 there's collections.Counter [1]: import collections c = collections.Counter('supercalifragilisticexpialidocius') print c # Counter({'i': 7, 'a': 3, 'c': 3, 'l': 3, 's': 3, 'e': 2, # 'p': 2, 'r': 2, 'u': 2, 'd': 1, 'g': 1, 'f': 1, # 'o': 1, 't': 1, 'x': 1}) [1] http://docs.python.org/2.7/library/collections.html#collections.Counter http://docs.python.org/2.7/library/collections.html#collecti...
- andreasvc 14y agoIt's odd that with all the 'high-brow' datatype names such as dictionaries, tuples, and sets, they decided to call this one Counter instead of the obvious choice 'multiset' (or 'bag'); it supports set operations such as intersection, so it's more generally useful than just collecting top ten lists &c.
- deleted 14y ago[deleted]
- hartror 14y agoI hate to be that guy but what about dictionary comprehensions?
- andreasvc 14y agoOr dictionary views: they stay current as the dictionary is updated, and allow set operations.
- dougws 14y agoHave used all of the techniques in the article (plus dict/set comprehensions) but had never heard of dict views [1] until now. Good stuff! [1] http://docs.python.org/2/library/stdtypes.html#dictionary-view-objects http://docs.python.org/2/library/stdtypes.html#dictionary-vi...
- te 14y agoI don't get it. Why would I use a view instead of the dictionary itself?
- e12e 14y agohttp://stackoverflow.com/questions/340850/python-3-0-dict-methods-return-views-why http://stackoverflow.com/questions/340850/python-3-0-dict-me... Views are lighter than a full copy of a list, yet behaves like a list (eg: supports `key in view`). [edit] Also, this seems to be the relevant PEP: http://www.python.org/dev/peps/pep-3106/ http://www.python.org/dev/peps/pep-3106/
- andreasvc 14y agoIt's not meant as a replacement; views form the new implementation of the keys(), values(), and items() methods in Python 3. The have been made available in Python 2 as viewkeys() etc. As said, they have the advantage of providing set operations (e.g., intersection with the keys of another dictionary), and they are more efficient than keys() or iterkeys(), because they have less overhead.
- avolcano 14y agoYou know a guide is good when it makes you want to go back and refactor old code. Great information, thanks :)
- alexpopescu 14y agoInterestingly after reading the post I've realized that while I've been using the more advanced stuff, I was missing the first 2 basic ones.
- DeepDuh 14y agoHappened to me too, I always used 'for key in mydict: do_something(key, mydict[key])' instead of 'for key, value' I guess python leaves so many ways to do things that you tend to settle on some style quickly even if it's not the most efficient one. When self learning new languages I still miss an efficient way to get all the idioms. However I think that in python it's not really that important, as long as you get stuff done - your code is still going to be quite readable.
- RyanMcGreal 14y agoPython doesn't always hew too closely to TSBOAPOOOWTDI, despite Ken Peters' lovely entreaty. http://www.python.org/dev/peps/pep-0020/ http://www.python.org/dev/peps/pep-0020/
- simgidacav 14y agoMy good friend, refactoring is sometimes good, but remember the ancient wisdom: don't fix what's not broken! Your old code may not be pythonic, but still it works...
- solox3 14y ago"not key in dct", or "not (key in dct)", is never slower than "key not in dct" because when key is found in dct, the expression is immediately true.
- aidos 14y agoThis is a really nice guide. I've been up to my eyes in python dicts over the last few days so already had most of these figured out. setdefault is new to me, which is cool. Unfortunately I can only see one place to use it in my code and it would be inefficient [0]. Best stash it away for later use :) [0] r = re_subs.setdefault(s, re.compile(s))
- andrewcooke 14y agoiirc re caches regexps. so unless you're using many (more than the cache size [edit: 100]), i wouldn't worry (well, and also measuring before optimising...)
- aidos 14y agoI did wonder if that was the case but I couldn't find anything about it. Just discovered re._MAXCACHE which implies it does cache. Tested and in my case the performance is the same. Thanks for the tip (I was posting half hoping someone would have a better solution). That's another 3 lines of code removed :)
- blt 14y agoRocking it out! Awesome! Getting down! Make it happen!
- kriro 14y agoPretty cool, bookmarked. I always enjoy different forms of presenting the info, playbook was a nice touch :)
- simon_weber 14y agoGood advice. I've got one to add: #x and y are dictionaries z = dict(x.items() + y.items()) It merges two dictionaries, giving precedence to the second (in Python 2 - Python 3 is a bit more nasty: http://stackoverflow.com/questions/38987/how-can-i-merge-union-two-python-dictionaries-in-a-single-expression http://stackoverflow.com/questions/38987/how-can-i-merge-uni...).
- goronbjorn 14y agoThanks for writing this; it's very useful. Are you planning on doing similar posts about other parts of Python in the future?
- stillinbeta 14y agoIn most circumstances for (key, val) in data won't work, as the default dictionary iterable only contains the keys. You want for key, val in data.items():
- js2 14y agoOr iteritems() for an iterable.
- pcote 14y agoI use dictionary comprehensions personally but I have mixed feelings about the syntax. It looks too much like set comprehensions on first glance. Compare the following to see what I mean. myset = {x for x in "This is my stuff".split()} mydict = {x:len(x) for x in "This is my stuff".split()}
- andreasvc 14y agoWell I see what you mean but then again a comma looks very much like a period ... The advantage of this syntax is that : unambiguously introduces a key: value pair, whereas (key, value) could also occur in a list comprehension (e.g., by accident).
- andreasvc 14y agoErm, you can leave out 'in most circumstances' and 'default'; this the way they always work.
- pcote 14y agoI love curiosity sparking posts like this. >>key in dct is much better than key in dct.keys() Of course, that got me curios to find out if there is a magic method out there that takes advantage of keyword "in". Turns out __contains__ does that. Always exciting to stumble upon new stuff in my favorite language.
- Goopplesoft 14y agoOne my favorites, safe deep searching: by returning a dict you can run another .get var = {'a' : 'b' , 'c' : {'d' : 'f'}} print var.get('c', {}).get('d') print var.get('DNE', {}).get('d')
- bismark 14y agoI also find the combination of lambda and defaultdict quite useful: d = defaultdict(lambda: False) or d = defaultdict(lambda: {'foo':set(), 'bar':False}) d['baz']['foo'].add(1)
- ralph 14y agoFor the first I do defaultdict(bool) ;-)
- textminer 14y agoLove defaultdict. It and dict/set/list compressions are a big part of what makes Python so fast to write in. Great practice for 2.7 that's probably quashed in 3.0. For large dicts, no need to create a giant set en route when iterating over keys, values, or both. Use "for k in d.iterkeys()", "for v in d.itervals", "for k,v in d.iteritems." While I'm at it-- if you're ever finding yourself using a huge amount of awfully rigid objects from a single class, use __slots__ to allocate needed variables! Python will otherwise define the object's namespace in a dict (called __dict__) which allocates a whole kilobyte per object. Bad news if you have several hundred thousand... Guessing this is why Guido loves namedtuples so much for basic attributed storage.
- cardamomo 14y agoIn Python 3.3: Iterate over keys: "for k in d.keys()" ...over values: "for v in d.values()" ...over both: "for k, v in d.items()"
- textminer 14y agoLove it. It's that way with range, too, right? Does one end up casting things to lists more often if so much is naturally an iterator?
- pyre 14y agohttp://www.python.org/dev/peps/pep-3106/ http://www.python.org/dev/peps/pep-3106/ The original plan was to simply let .keys(), .values() and .items() return an iterator, i.e. exactly what iterkeys(), itervalues() and iteritems() return in Python 2.x. However, the Java Collections Framework [1] suggests that a better solution is possible: the methods return objects with set behavior (for .keys() and .items()) or multiset (== bag) behavior (for .values()) that do not contain copies of the keys, values or items, but rather reference the underlying dict and pull their values out of the dict as needed.
- andreasvc 14y ago"for k in d" is even shorter. I wonder what they point of the .keys() method is actually, perhaps it's just as redundant as .has_key(). Most times you can iterate over the dictionary itself; when you need to explicitly pass an iterator or list, iter(d) or list(d) is shorter than d.keys().
- simgidacav 14y agoThanks, I didn't know about "x not in D"! On the same note: http://dacavtricks.wordpress.com/2011/05/23/python-default-values-in-a-dictionary/ http://dacavtricks.wordpress.com/2011/05/23/python-default-v...
- taejo 14y agoIn item 3, the "boilerplate" and the "awesome way" are not equivalent. The boilerplate does the equivalent of setdefault, which is mentioned later dct[key] = dct.setdefault(key, 0) + 1
- isbadawi 14y agoSince you're assigning to dct[key] anyway, it doesn't really matter.
- deleted 14y ago[deleted]
- mixedbit 14y agoCan logical operator in Python be used to conditionally initialize an item in a dict? For example, can: group = dct.setdefault(key, []) group.append(value) be replaced with some equivalent of this Ruby snippet: (dict[key] ||= []) << value ?
- deleted 14y ago[deleted]
- masklinn 14y agoNo. But of course if ``group`` is not used afterwards it can be inlined to dct.setdefault(key, []).append(value)
- manojarcom 14y agomy Bitdefender is blocking the site; says it's insecure.
- eagle9 14y agonice! very useful, thanks for putting this together.