12 ms·
Python idioms I wish I'd learned earlier
- desdiv 12y agoI'm not much of a Python guy, but that chained comparison operator is sweet! Sure, it's just syntax sugar, but it saves a lot of keystrokes, especially if the variable name is long. Is Python the only language with this feature?
- euphemize 12y agoI'm not sure if other languages have it, but I have to say that pycharm is excellent at suggesting chained comparisons to you. I didn't know this existed before I switched IDEs. if 2 > 3 and 2 < 7 becomes if 3 < 2 < 7
- kazinator 12y agoIf this Pycharm really had brains it would tell you that (2>3 and 2<7) reduces to (false and true), to false, at compile time, so the code wrapped in the if is unreachable.
- Alphasite_ 12y agoI'm not sure about pycharm, but i know for certain IntelliJ warns you about constant conditions like that. That said, he could have picked that example just to illustrate a point, rather than something it literally suggests.
- dragonwriter 12y ago> Is Python the only language with this feature? No. See, e.g., http://stackoverflow.com/questions/4090845/language-support-for-chained-comparison-operators-x-y-z http://stackoverflow.com/questions/4090845/language-support-...
- pwr22 12y agoPerl6 has it too > perl6 > 3 < 4 < 5 True
- dllthomas 12y ago3 < 4 < 5 evaluates to true (well, 1) in C as well. What does 5 > 4 > 3 give?
- kazinator 12y agoCommon Lisp (and other dialects): (< a b c d) ;; T if a < b < c < d (<= a b c d) ;; T if a < b < c < d Also: (lcm a b c d ...) ;; lowest common multiple (+) -> 0 (+ a) -> a (+ a b) -> a + b (+ a b c) -> (a + b) + c (*) -> 1 (* a) -> a (* a b) -> a * b (* a b c) -> (a * b) * c Is it just syntactic sugar? (< a b c) evaluates a, b and c only once, which matters if they are expensive external function calls or have side effects. (and (< (foo) (bar)) (< (bar) (baz))) isn't the same as (< (foo) (bar) (baz)) By the way, this could be turned into a short-circuiting operator: more semantic variation. Suppose < is allowed to control evaluation. Then an expression like (< a b c d) could avoid evaluating the c and d terms, if the a < b comparison fails.
- hasenj 12y agoBut can any lisp dialect do: a < b >= c ?
- baddox 12y agoGiven that lisps tend to use prefix notation, the example doesn't even translate meaningfully.
- dllthomas 12y agoI assume the above was intended to be a Python expression. The answer, to the best of my knowledge, is "not really". There is no built in or reasonably standard function that lets you check simultaneously that a is less than b which in turn is greater than or equal to c. Not that we couldn't define one ad-hoc, although making it reusable in a way that's reasonably idiomatic might be a challenge... Obviously, you can express it slightly more verbosely: (and (< a b) (>= b c))
- davidrusu 12y ago(< a (>= b c)) same number of operators but a few extra parens
- deleted 12y ago
- jfoster 12y agoI think syntactic sugar is vastly undervalued by programmers. Anything that lets me more naturally read & write code should lead to fewer bugs and more productive software development, as well as making programming more enjoyable.
- teddyh 12y agoSQL has “x BETWEEN y AND z” as a special case which does “y >= x >= z”, which in practice is quite often what you actually want to use this feature for.
- Zecc 12y agoIt's not simply syntax sugar either. def sideEffects(): print "Called." return 5 if 0 < sideEffects() < 10: # sideEffects is only called once
- retroencabulato 12y agoNice list, but I was confused by the arguments to the dict .get() example until I looked up the definition.
- ims 12y agoI think the example in #4 misses the point of using a Counter. He could have done the very same for-loop business if mycounter was a defaultdict(int). The nice thing about a Counter is that it will take a collection of things and... count them: >>> from random import randrange >>> from collections import Counter >>> mycounter = Counter(randrange(10) for _ in range(100)) >>> mycounter Counter({1: 15, 5: 14, 3: 11, 4: 11, 6: 11, 7: 11, 9: 8, 8: 7, 0: 6, 2: 6}) Docs: https://docs.python.org/2/library/collections.html#counter-objects https://docs.python.org/2/library/collections.html#counter-o...
- Shizka 12y agoI recently had a use case for this where I needed a naively created bag of words along with the frequency of words. Having Counter made this extremely easy as I could simply split the string on whitespaces and pass the result to a Counter object. Another useful feature is that you can take the intersection of two Counter objects. It's a really nice data structure to have!
- frindo 12y agoPython noob here. What does the _ mean in "...for _ in range(100))"?
- icebraining 12y agoIt's just a name of a variable. There's nothing special about it (in this context), but it's conventionally used when you don't care about the value assigned to it.
- ixwt 12y agoNormally you would place a variable there, such as x or y. But in this case, the variable doesn't matter. You aren't using it in the function call. You're telling the programming the variable for the for loop that it doesn't matter.
- danudey 12y agoTo add to specifics: _ in python is commonly used as a variable name for values you want to throw away. For example, let's say you have a log file split on newlines, with records like this: logline = "127.0.0.1 localhost.example.com GET /some/url 404 12413" You want to get all the URLs that are 404s, but you don't care about who requested them, etc. You could do this: _, _, _, url, returncode, _ = logline.split(' ') There's no special behaviour for _ in this case; in fact, normally in the interactive interpreter it's used to store the result of the last evaluated line, like so: >>> SomeModel.objects.all() [SomeModel(…), SomeModel(…), SomeModel(…), …] >>> d = _ >>> print d [SomeModel(…), SomeModel(…), SomeModel(…), …] Which I think is basically the same behaviour; you run some code, you don't assign it, so the interpreter dumps it into _ and goes on about its day.
- wodenokoto 12y agoCan someone direct me to a comparision of subprocess and os? I keep hearing subprocess is better, but have not really read any explanation as to why or when it is better. (I'm glad I'm not the only one who was thrilled to discover enumerate()!)
- buttcoinslol 12y agoThe subprocess module is meant to replace a number of functions in the os module. The python documentation lists a number of examples here: https://docs.python.org/2/library/subprocess.html#subprocess-replacements https://docs.python.org/2/library/subprocess.html#subprocess... If you google 'os vs subprocess python' there are a few stackoverflow and quora threads comparing them.
- deleted 12y ago[deleted]
- crdoconnor 12y agoAll explained here: https://www.python.org/dev/peps/pep-0324/ https://www.python.org/dev/peps/pep-0324/ tl;dr * Security (including avoiding holes like shellshock) * Unification of process handling under one core library. * Easier detection of errors in processes. * Easier control of stderr/stdout. * Universal newline support. * Elimination of race conditions with .communicate()
- willvarfar 12y agoThe OS module interacts directly with the OS rather than abstracts it, so a lot of the functions in it have the "may not be available on all platforms" apology. Subprocess uses OS under the hood but offers an abstraction that mostly works on all platforms, e.g. the way that "communicate" is implemented on Windows differs considerably from how its implemented on Unix.
- euphemize 12y agoOne of my favorites: >>> print "* "* 50 to quickly print a separator on my terminal :) Previous discussion on python idioms from 300 days ago: https://news.ycombinator.com/item?id=7151433 https://news.ycombinator.com/item?id=7151433
- Animats 12y agoThat's cute, but the result of a bad design decision. Python overloads "+" as concatenate for strings. This also applies to lists. So [1,2,3] + [4,5,6] yields [1,2,3,4,5,6] This is cute, but not what you want for numerical work. Then, viewing multiplication as repeated addition, Python gives us [1,2,3]*4 yields [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3] This is rarely what was wanted. Then there's numpy, which has its own array type, needed because Python's array math is slow. Numpy arrays have different semantics - add and multiply perform the usual numeric operations. You can mix numpy arrays and built-in lists with "+" and "*". Mixed expressions are evaluated using numpy's semantics. Since Python doesn't have type checking, it's possible to pass the wrong kind of numeric array to a function and have it treated with the wrong semantics. Moral: when designing your language, using "+" for concatenation is tempting, but generalizing that concept comes back to bite you.
- wodenokoto 12y agoThen what would you have [1,2,'q',[1,('a',2)]] + 4 yield? The reason why numpy lets you do math operation on each element in an array is because you can safely assume that each element is a number. You can assume absolutely nothing about the types of the elements in a list.
- Animats 12y ago"TypeError: cannot add 'str' and 'int' objects." Just because you can define semantics for nonsense doesn't mean you should.
- ubernostrum 12y ago
- ckuehl 12y ago> There is a solution: parentheses without commas. I don't know why this works, but I'm glad it does. It's worth mentioning that this is a somewhat controversial practice. Guido has even discussed removing C-style string literal concatenation: http://lwn.net/Articles/551438/ http://lwn.net/Articles/551438/ You may wish to consult your project's style guide and linter settings before using it.
- smoe 12y agoI personally don't like this style of using multiple strings. Makes radical changes of the text cumbersome. I think in most cases it's better to use triple quotes. And if the content of these variables isn't exclusively shown in the shell, you should use translation files anyway.
- philsnow 12y ago$ cat triple.py def foo(): print """this is a triple quoted string this is a continuation of a triple quoted string""" if __name__ == '__main__': foo() $ python triple.py this is a triple quoted string this is a continuation of a triple quoted string This is really warty. In bash you can mostly get around this with <<- for here documents (which removes leading indentation, so even if you're defining a here doc at a place that itself is indented, you don't have to align the text to the left column yourself). The man page in my version suggests it only works for leading tabs, not spaces, though. e.g. $ function usage() { cat <<-END this is a here document that spans multiple lines and is indented END } $ usage this is a here document that spans multiple lines and is indented $
- leephillips 12y agoI was grateful for the example of multilined strings, mysterious as it is. The lack of any way to do this has been an annoyance of mine for quite some time.
- rnhmjoj 12y agoThis is something I do instead of writing a long if-else: opt = {0: do_a, 1: do_b, 3: do_b, 4: do_c} opt[option]()
- baddox 12y agoDo you consider that to be idiomatic? I've been out of touch with the a Python community for a few years, but back then I wouldn't have considered that remotely idiomatic, and if I was on a team writing software, I would have argued that we shouldn't be writing code like that.
- ubernostrum 12y agoIt's not something I see all the time, but using a dictionary instead of a big if/else is something that I'd consider a Python idiom, yes. And it's something I do on occasion. For example: https://github.com/ubernostrum/webcolors/blob/master/webcolors.py#L358 https://github.com/ubernostrum/webcolors/blob/master/webcolo...
- zenciadam 12y agoWhy? It's just a jump-table. In C, you would use function pointers instead of references to functions.
- davidrusu 12y agoI've done this before, I think of it as a more powerful form of a switch statement. I'd love to hear why you think it's not ideal.
- SiVal 12y agoI don't think it's more powerful in any way than an if-elif-elif-else which, for what it's worth, I consider the Pythonic way. Having said that, just because I consider something more Pythonic doesn't mean I prefer it. I've worked in a lot of languages over the years and still work in several in addition to Python. I really enjoy Python, but I prefer techniques that are more universal in many cases. For example, I prefer the idiom of looping over a list index to using Python's "enumerate" in most cases, because index looping is a common cross-language idiom and enumerate usually doesn't offer any benefit I value more than universal obviousness. Other things such as Python's `for item in items` looping style are both VERY Pythonic and much nicer than, say, index looping, so I would almost always prefer such idioms. The above switch -> function pointerish thing is clear to me from years of C/C++, but it is both less generally applicable across languages than if-elif... and less Pythonic, so I would prefer the if-elif... approach. Obviously a matter of preferences, but since you asked....
- polemic 12y agoI work with python full time, and the last (#10 string chaining) is one of the few times the syntax had caused me grief, due to missed commas in what were supposed to be tuples of strings. The chaining rules are one of the few sources of apparent ambiguity in the syntax, especially when you include the multiline versions.
- ghshephard 12y agoWow - that's really, really great list. In particular, #7 is something that I didn't even know existed, and I've been hacking around for 2+ years. Instead of: mdict={'gordon':10,'tim':20} >>> print mdict.get('gordon',0) 10 >>> print mdict.get('tim',0) 20 >>> print mdict.get('george',0) 0 I've always done the much more verbose: class defaultdict(dict): def __init__(self, default=None): dict.__init__(self) self.default = default def __getitem__(self, key): try: return dict.__getitem__(self, key) except KeyError: return self.default mdict=defaultdict(0) mdict['gordon']=10 mdict['tim']=20 print mdict['gordon'] 10 print mdict['tim'] 20 print mdict['george'] 0 I'll be sure to make great use of the dictionary get method - I'm embarrassed to admit how many thousands of times I could have used that, and didn't know it existed.
- bhy 12y agoDo you know there's also collections.defaultdict ?
- ghshephard 12y agoI do now! I had known (and used) collections.OrderedDict, but had never used defaultdict. Millions of keyerrors later... I clearly am going to have to spend a few hours today grokking everything that collections.* has to offer. Thanks very much.
- raverbashing 12y agoRead the manuals! ;) There's a lot of hidden gems. Your idea is nice in a syntactic sugar way, also, the default being a part of the dictionary rather than the get function makes it copyable. I'll give you another gem that could be interesting: else clause in for loops
- surreal 12y agoAlso read other people's code :) You'll see this being used countless times in libraries / on GitHub.
- dllthomas 12y ago"Because I was so used to statically typed languages (where this idiom would be ambiguous), it never occurred to me to put two operators in the same expression. In many languages, 4 > 3 > 2 would return as False, because (4 > 3) would be evaluated as a boolean, and then True > 2 would be evaluated as False." The second half of this is correct, but it has nothing to do with whether the language is statically or dynamically typed. It's a tweak to the parser, mostly.
- azernik 12y agoIt's not just a tweak to the parser, and it does have to do with the type system, but you're right that it's not about static typing. The issue is that there are languages (like C) where typing is static but weak, so e.g. booleans are also integers and can have integer operations like '>' applied to them. In other words, the problem is that in C True == 1 and 1 > 2 is a valid expression. In Python, which has strong(er) types, this expression can have an unambiguous meaning if you want to add the feature to your parser.
- dllthomas 12y agoYou can implement it entirely in the parser if you can avoid name capture - it may or may not be implemented entirely as a tweak to the parser in practice, but it's fundamentally a syntactic thing. Your discussion of types here is all wrong - it's true that C treats booleans as if they were integers, but Python does, too: >>> (3 > 4) < 2 True >>> 3 > 4 < 2 False >>> 3 > (4 < 2) True It has nothing to do with types.
- rectangletangle 12y agoI'm a fan of Python's conditional expressions. foo = bar if qux is None else baz They're particularly interesting when combined with comprehensions. ['a' if i % 2 == 0 else 'b' for i in range(10)] Though this particular example can be expressed much more concisely. ['a', 'b'] * 5
- TillE 12y agoThat's a Pythonization of C's conditional operator. foo = qux == NULL ? bar : baz; Of course, C does not do list comprehensions.
- plikan13 12y agoBefore the conditional expression was introduced to Python I sometimes wrote things like foo = [baz, bar][qux is None]
- misiti3780 12y agoi didnt realize you could add the if statement in the comprehension where you did ['a' if i % 2 == 0 else 'b' for i in range(10)] very cool! I was on aware of code like ['a' for i in range(10) if i % 2 == 0]
- misiti3780 12y agoi didnt realize you could add the if statement in the comprehension where you did ['a' if i % 2 == 0 else 'b' for i in range(10)] very cool! I was only aware of code like ['a' for i in range(10) if i % 2 == 0]
- tinkerdol 12y ago"Missing from this list are some idioms such as list comprehensions and lambda functions, which are very Pythonesque and very efficient and very cool, but also very difficult to miss because they're mentioned on StackOverflow every other answer!" Can anyone link to good explanations of list comprehensions and lambda functions?
- RyanMcGreal 12y agoThis seems like an easy intro to list comprehensions: http://www.pythonforbeginners.com/basics/list-comprehensions-in-python http://www.pythonforbeginners.com/basics/list-comprehensions...
- jemfinch 12y agoMost of these idioms actually make me sad. When I first started using Python around 1999, it didn't even have list comprehensions. Code was extremely consistent across projects and programmers because there really was only one way to do things. It was refreshing, especially compared to Perl. It was radical simplicity. Over the decade and a half since then, the Python maintainers have lost sight of the language's original elegance, and instead have pursued syntactical performance optimizations and sugar. It turns out that Python has been following the very same trail blazed by C++ and Perl, just a few years behind. (At this point Python (especially with the 2 vs. 3 debacle) has become so complex, so rife with multiple ways to do even simple things that for a small increase in complexity, I can just use C++ and solve bigger problems faster.)
- stdbrouw 12y agoAre we reading the same article, though? It's certainly up for debate whether named tuples and enums, various kinds of metaprogramming and decorators might be making the language more complex for fairly little gain... but this article talks about the `enumerate` function, about string formatting and dictionary comprehensions. Simple, straightforward stuff with no downsides.
- kstrauser 12y agoBut syntax matters! When you're writing stuff all day, a = [_ * 2 for _ in range(10)] is a lot more pleasant than: a = []; for _ in range(10): a.append(_ * 2) It also gives Python a lot more information about your actual intent. Suppose "range(10)" were actually "giant_list". Hypothetically, the list comprehension could pre-allocate len(giant_list) elements instead of calling list.append that many times. That's potentially a huge performance win. You see Python as getting more complex. I see it as getting less complex by giving concise alternatives to common idioms.
- tomp 12y agoSome comments: 1. Am I the only one that really loves that `print` is a statement and not a function? Call me lazy, but I don't mind not having to type additional parentheses. 5. Dict comprehensions can be dangerous, as keys that appear twice will be silently overridden: elements = [('a', 1), ('b', 2), ('a', 3)] {key: value for key, value in elements} == {'a': 3, 'b': 2} # same happens with the dict() constructor dict(elements) == {'a': 3, 'b': 2} 7. I see D.get(key, None) way too often. 8. Unpacking works in many situations, basically whenever a new variable is introduced. for i, el in enumerate(['a', 'b']): print i, el {key: value for (key, value) in [('a', 1), ('b', 2), ('a', 3)]} map(lambda (x, y): x + y, [(1, 2), (5, -1)]) Note: the last example (`lambda`) requires parentheses in `(x, y)`, as `lambda x, y:` would declare a two-argument function, whereas `lambda (x, y):` is a one-argument function, that expects the argument to be a 2-tuple.
- kryptiskt 12y ago> 1. Am I the only one that really loves that `print` is a statement and not a function? Call me lazy, but I don't mind not having to type additional parentheses. If you think the parentheses are bad, why wouldn't you prefer a language like Ruby where you can omit them generally? Leaving them out for just one special construct seems so insufficient as a cure.
- tomp 12y agoIn general, Ruby's syntax (or rather, semantics) is ambiguous (you don't know if a statement without parentheses is calling a function or just accessing a value). I prefer unambiguous syntax. However, Python's `print` statement is (1) well-known, (2) very useful (for prototyping, debugging, in REPL), and (3) shouldn't in general be present in production code (logging should be used instead). Therefore, omitting parentheses would help ease debugging and exploring (REPL prototyping), while not making code more ambiguous in general. Yes, it's a special case, but `print` is also has very special, very specific use-case.
- DougalDC 12y agotom, get back to work!
- itmard 12y agoGood job https://twitter.com/TehPUG/status/537911745323753472 https://twitter.com/TehPUG/status/537911745323753472
- flares 12y agohaha.. in #1, the easter egg "not a chance" :) :)
- __luca 12y agoSincerely, Transforming Code into Beautiful, Idiomatic Python – by Raymond Hettinger... http://youtu.be/OSGv2VnC0go http://youtu.be/OSGv2VnC0go
- RyanMcGreal 12y agoI came across this when I was first learning Python and it has always impressed me: from random import shuffle deck = ['%s of %s' % (number, suit) for number in '2 3 4 5 6 7 8 9 10 Jack Queen King Ace'.split(' ') for suit in 'Hearts Clubs Diamonds Spades'.split(' ')] shuffle(deck)
- ajuc 12y agoI never liked how people in Python use stringWithSpaces.split instead of a list. Just feels wrong somehow. But I've seen it many times so it's probably pythonic
- TuringTest 12y agoIt makes sense when all the values in the list are text. Avoids lots of ", "
- DaFranker 12y agoI do it entirely, exclusively, only, purely because it requires less punctuation typing. It returns a list anyway. The performance hit is virtually unnoticeable in almost every use case (unless this is a function taking in input strings formatted this way many times per second, but in that case you've got way worse to worry about first...).
- icebraining 12y agoI agree; what I usually do is type the string.split() version in a REPL (I keep one open permanently) and then copy-paste the result to the file. It avoids having to do a big change if you need to add a new item with spaces in it.
- omaranto 12y agoIt's like old Perl idiom where Python's 'ab cd ef'.split(), would be written as qw(ab cd ef), which probably looks nicer --'qw' stands for 'quote words', I think.
- RyanMcGreal 12y agoAs DaFranker points out, it's just easier to type than ('Hearts', 'Diamonds', 'Spades', 'Clubs') and has less opportunity for typos and syntax errors. If I was concerned about performance I would replace it with a tuple, but it was Good Enough for a quick example.
- sherjilozair 12y agoIs there any such collection of advanced Python patterns, aimed at Python programmers with more than 2-3 years of experience?
- tomkinstinch 12y agoI'd love other answers to the same question. Here are a few I've come across in the past: http://www.rafekettler.com/magicmethods.html http://www.rafekettler.com/magicmethods.html http://sahandsaba.com/thirty-python-language-features-and-tricks-you-may-not-know.html http://sahandsaba.com/thirty-python-language-features-and-tr... http://www.siafoo.net/article/52 http://www.siafoo.net/article/52
- TheLoneWolfling 12y agoI wish there was an interval set in Python's builtins. I also wish that ranges were an actual proper set implementation - so you could, for example, take intersection and union of ranges. And I wish that Python had an explicit concatenation operator.
- Spiritus 12y agoYou mean like the built in `set` object? https://docs.python.org/2/library/stdtypes.html#set https://docs.python.org/2/library/stdtypes.html#set
- omaranto 12y agoI think he or she meant a specialized set data structure that stores sets of numbers which can be written as finite unions of interval. Typically besides set operations you'd also want (1) that only endpoints of the intervals are stored, so that the structure is compactly represented in memory, (2) to be able to recover the canonical representation of the set as a sorted union of disjoint intervals.
- TheLoneWolfling 12y agoThis is exactly what I meant, thank you.
- TheLoneWolfling 12y agoSet doesn't work for this. You need to store all possible numbers between the lower and upper bound, which isn't exactly workable for (for example) floats.
- panzi 12y agoOh, wow. I didn't know the dict comprehensions. Since when do they exist? I always used: d = dict((key(x), value(x)) for x in xs)
- shackenberg 12y agoIf you were underwhelmed by this blog post have a look at: Transforming code into Beautiful, Idiomatic Python by Raymond Hettinger at PyCon 2013 https://speakerdeck.com/pyconslides/transforming-code-into-beautiful-idiomatic-python-by-raymond-hettinger-1 https://speakerdeck.com/pyconslides/transforming-code-into-b... and https://www.youtube.com/watch?v=OSGv2VnC0go&noredirect=1 https://www.youtube.com/watch?v=OSGv2VnC0go&noredirect=1 Topics include: 'looping' with iterators to avoid creating new lists, dictionaries, named tuples and more
- Vaskivo 12y agoAnd, if possible, see every video by Raymond Hettinger. He is a great teacher.
- glitchdout 12y agoSorry for the offtopic but why did you add `&noredirect=1` to the end of that youtube url?
- shackenberg 12y agosorry, was only an accident.
- _navaneethan 12y agoThere is an sweet in India called [0] phirni when you start eating from outer layer to inner layer, you will feel like walking in heaven.Now you are in outer layer.I hope you enter to the inner level and feel the python still. :) [0] http://www.wikihow.com/Make-Phirni-%28a-Rice-and-Milk-Dish%29 http://www.wikihow.com/Make-Phirni-%28a-Rice-and-Milk-Dish%2...