7 ms·
Python Tips and Traps
- dalke 12y agoThe namedtuple example is wrong. The constructor requires all of the parameters, and an attribute cannot be set: >>> from collections import namedtuple >>> LightObject = namedtuple('LightObject', ['shortname', 'otherprop']) >>> m = LightObject() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: __new__() takes exactly 3 arguments (1 given) >>> m = LightObject("first", "second") >>> m.shortname 'first' >>> m.shortname = 'athing' Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: can't set attribute Also, bare try/excepts as in: try: # get API data data = db.find(id='foo') # may raise exception except: # log the failure and bail out log.warn("Could not retrieve FOO") return are really bad. The failure might be caused by a ^C or MemoryError, or even a SystemExit, should db.find() desire to do that. Instead, qualify it by catching Exception: try: # get API data data = db.find(id='foo') # may raise exception except Exception: # log the failure and bail out log.warn("Could not retrieve FOO") return It's also poor form to "return True" in the exit method of the context manager. If there is no exception then that's not needed at all, and if there is an exception ... well, that code will swallow AttributeError and NameError and ZeroDivisionError, and leave people confused as to the source of the error.
- ryan_sb 12y agoThanks for correcting me, I'll get those updated ASAP.
- pdonis 12y agoA good rule of thumb: if you show actual code, run it before you post it!
- mkesper 12y agoShouldn't you qualify the exception class as properly as you can? If you can't I'd expect to check for certain error classes you can handle and else reraise the Exception.
- dalke 12y agoYes, I would do that in nearly every case. However, I don't know the larger context. If this were at the top-level of a web services handler where a "return None" indicates a 400 - Internal Server Error, then logging the database failure and stopping is likely acceptable. Even then, I wouldn't call it good code. However, the goal of this essay seemed to be to give the minimal example, and a more complete example would have required introducing a fake database module with its own exception type. I believe that would have obscured the intent. I would have preferred real, working code. In this case, with sqlite3. That's fundamentally a pedagogical choice though.
- timdierks 12y agoI frequently want ad-hoc slots to create a structure of related values without the verbosity of a dict with static keys. What's the pythonic method? E.g. when I want to carry a foo, a bar, and a baz around, but namedtuple isn't right (e.g. I need mutation). I'd prefer config = Someclass() config.foo = 1 config.bar = "quux" config.baz = 3.3 and then using config.bar, etc. over: config = { 'foo': 1, 'bar': "quux", 'baz': 3.3 } and then using config['bar']
- gknoy 12y agoThanks for the reminder about integer division changing! I had seen some code in our (2.7) codebase that used (a/b), and I had wondered if I should be explicit about using math.floor, but this is even better.
- jamiesonbecker 12y agoExcellent article. Loved the specific examples for collection types, esp defaultdict and namedtuples!
- 3JPLW 12y agoHow would you create that hypothetical recurise defaultdict with defaults of defaultdicts? Is such a construct possible without creating a new defaultdefaultdict class?
- deleted 12y ago[deleted]
- birken 12y ago>> nested_dd = collections.defaultdict(lambda: nested_dd) >> nested_dd['a']['b']['c']['d'] = 'hello'
- jonathanpoulter 12y agoFor reference, this is called autovivification. http://en.wikipedia.org/wiki/Autovivification#Python http://en.wikipedia.org/wiki/Autovivification#Python
- rnhmjoj 12y agofrom collections import defaultdict defaultobj = lambda: type('defaultobj', (defaultdict,), { '__getattr__': lambda self, x: self.__getitem__(x), '__setattr__': lambda self, x, v: self.__setitem__(x, v) })(defaultobj) names = defaultobj() names.mammalia.primates.homo['H. Sapiens'] = 'Human being' print(names) The same but also allows dot notation.
- bdr 12y agoI don't think that's what they wanted: >>> nested_dd['d'] 'hello'
- choochootrain 12y agotry nested_dd = lambda: defaultdict(nested_dd)
- jamiesonbecker 12y ago
- settrans 12y agoset() is a great way to deduplicate small lists, but it's important to note that it requires O(n) extra space (in-place sorting can avoid this overhead, but is more complex).
- raymondh 12y agoSorting is almost always the wrong way to do it. (Wordy and slow). It is fragile design to write code that depends on 1) the data is so large that you don't have room for a set() BUT 2) it is small enough for an in-memory sort. (IOW, the almost-out-of-memory case invariably degrades over time to flat-out-of-memory). Another thought: people seem to place too much concern about about the size of various data structures rather than thinking about the data itself. Python containers don't contain anything, they just hold references. (Usually, the weight of a bucket of water is mostly the water, not the bucket itself). Finally, if your task is to dedup a lot of data, it doesn't make sense to read it all into memory in the first place (which you would need for a sorting approach). It is better dedup it using a set as you read in the data: # Only the unique lines are ever kept in memory with open('hugefile.txt') as f: uniq_lines = set(f) Dude, sorry to go off like this, but the advice you gave is almost always the wrong way to do it.
- meowface 12y agoYou're still reading all of the lines into memory in that example as soon as you call `set(f)` (which is basically equivalent to set(f.readlines()), though, which may not necessarily be what you want.
- jzwinck 12y agoNo, that's not at all what's happening. f.readlines() creates and returns a full list of all lines, loaded into memory. But set(f) uses f as an iterator, which reads chunks of the file and yields one line at a time, which can then be inserted into the set, de-duping on the fly. Your parent is correct (and clever).
- 12y ago
- rhapsodyv 12y agoI think I have a karma with python. Every python project a needed to touch I lost a lot of time dealing with mixed spaces and tabs scattered throughout every source file. I really know I am unlucky, this cannot be normal.
- TheLoneWolfling 12y agoHave a save hook that converts everything to <tabs or spaces>.
- pjmlp 12y agoSome of us just use Python friendly editors.
- dalke 12y agoDon't use tabs. See https://www.python.org/dev/peps/pep-0008/#tabs-or-spaces https://www.python.org/dev/peps/pep-0008/#tabs-or-spaces which says to prefer spaces. You can start by converting all tabs into 8 spaces. This can be tricky should some strings have tabs. That's a bad idea in the first place. Use "\t". Don't mix tabs and spaces to get the same indentation level. Python 3 prohibits it. With Python 2 use "-t" or "-tt", which respectively warns and raises an exception if both spaces and tabs are used in the same block.
- rhapsodyv 12y agoYeah, I ended converting all the project to spaces.
- afarrell 12y agoYou are correct that this is not normal. Both sublimetext and vim make this easy to avoid. I recommend sublime
- kasabali 12y agoIf you want to read hundreds of these look no further than Python Cookbook [0]. [0] http://chimera.labs.oreilly.com/books/1230000000393/ http://chimera.labs.oreilly.com/books/1230000000393/
- deleted 12y ago[deleted]
- bdevine 12y agoAlways glad to see tips, tricks, and otherwise for Python. But for anyone checking out Python who sees how useful the defaultdict construct is but doesn't necessarily need nested attributes, the Counter class[0] has been available for some time now. If you just want to keep track of, well, counts, it's very handy and versatile. [0] https://docs.python.org/3.1/library/collections.html#collections.Counter https://docs.python.org/3.1/library/collections.html#collect...
- ryan_sb 12y agoTotally true, I'll add that in
- raymondh 12y agoNice article. It well expresses the delight that comes from seeing the expressiveness of just a handful of tools that fit well together.
- task_queue 12y agoUse contextlib to write your context managers. Thought the exception handling code was going to catch the all-encompassing use of 'except:' but nope. Don't do that.
- ryan_sb 12y agoI don't know how I've missed out on contextlib all this time. I'll update that section.
- tekromancr 12y agoI have used python for years, how did I never encounter python's collections module? I have implemented functionally partial versions of some of these. Looks like I gotta brush up on the batteries included!
- pjmlp 12y agoWhile you are at it, check the itertools, functools as well, in case you also don't know them.
- tekromancr 12y agoitertools I know and love, functools is a mystery I should look into.