7 ms·

Code Like a Pythonista: Idiomatic Python

237 points by ashishgandhi 15y ago

grammati 15y ago
I have just skimmed this so far, but it looks very good. A lot of the Python scripts that I see at work look like either C programs or glorified batch files. I'll definitely point people to this when they are ready and willing to move on to the next level. Well done. Suggestion: one thing I didn't see mentioned is switching from "if s.find(c) == -1: ..." to "if c in s: ...". I see people do that a lot.
- epenn 15y ago
  one thing I didn't see mentioned is switching from "if s.find(c) == -1: ..." to "if c in s: ...". I see people do that a lot. I think this is covered implicitly by the two "Use in where possible" sections. Although as Tim Peter's The Zen of Python states, which is also quoted in the tutorial, "explicit is better than implicit." If people are doing that despite knowledge of the in keyword then I suppose that just proves Tim's point.
  - donatzsky 15y ago
    I don't know. I'd argue that if c not in s: ... is more explicit (and readable) than if s.find(c) == -1: ...
- klenwell 15y ago
  You just made my weekend. I've been doing Python for a while and am well acquainted with the "item in list" idiom but did not know it could be used for find. That str.find syntax has always bothered me. Still, I had to test just to be sure: test_cases = [ # haystack, needle, expect ('abcdefg', 'a', True), ('abcdefg', 'b', True), ('abcdefg', 'bcd', True), ('abcdefg', 'h', False), ] for haystack,needle,expect in test_cases: # find version is_found = haystack.find(needle) != -1 # in version is_in = needle in haystack # confirm print haystack, needle, expect, '-->', (is_found, is_in) assert is_found == expect assert is_found == is_in Passed! Thanks.
sneak 15y ago
He had me up until the part where he wrote an algorithm to specifically leave out the Oxford Comma. :) Kidding aside, this was a great read.
- Luyt 15y ago
  As he remarks, he doesn't take in account the corner cases. Suprisingly, or maybe not, his concatenation statement is similar to what I regularly use: def commalist(f, endword="and"): "Concatenate a list of strings in the form of 'Red, Yellow, Green and Blue'." if len(f) == 0: return "" elif len(f) == 1: return f[0] else: return ", ".join(f[:-1]) + " " + endword + " " + f[-1]
zachallaun 15y ago
Read from the perspective of someone very much still learning to code, this was excellent. I particularly valued the idiomatic comparisons to 'naive' implementations (the kind I would initially reach for). One of the later sections on lazy generation also opened up some new doors for me, and gave me a nice solution to a problem I've been having with a piece of code.
hello_moto 15y ago
In the Java ecosystem, they call it best practices. But of course using the phrase "best practices" these days is considered taboo. Thus the use of the word "idiomatic"?
- Ixiaus 15y ago
  A "best practice" is essentially the same thing as "idiomatic python" in my opinion - I could pick apart different meanings from each but often times, it really is used to say the same thing. The phrase "best practice" isn't taboo in my experience - I'll often ask what the "best practice would be for implementing XYZ". If you rephrase that, "what is an idiomatic way to implement XYZ", it's saying the same thing. Python's community probably leans toward this diction because the culture of python revolves around the nebulous concepts of simplicity, obviousness, and "only one way", that in order to make those concepts more hylic and less "abstract and subjective" there is a lot of education (in the form of docs, PEP8, the very doc linked to in this post) to set the stage for what the expected idiom is.
- Goladus 15y ago
  A "best practice" is an optimal way to do something in a given context. An "idiom" is a commonly accepted way to express the thing you want to do. Best practices and idioms overlap, but are not identical.
- mekoka 15y ago
  http://dictionary.reference.com/browse/idiom http://dictionary.reference.com/browse/idiom An idiom is rather a preferred practice. If you take some time to read through some Python source, you'll notice that, although the code uses the exact same constructs that you are used to in Java, it does so in a manner you might find unfamiliar or unpredictable. One example is the use of try/except instead of if/else (beg for forgiveness rather than ask permission). Another is Duck-typing rather than Interface (if it quacks like a duck...) Coming from Java and looking at some Python code without forewarning or context, you might just be tempted to label the whole thing as really really bad practice, when in truth, it's just Idiomatic Python.
  - 15y ago
zmmmmm 15y ago
In case there are Pythonistas here who will indulge a question: one thing I've never figured out how to do elegantly ("idiomatically", I suppose) is create a list of predetermined size initialized with a constant(usually zeros). I always end up with something kludgey like: x = [0 for i in xrange(0..100)] which seems so roundabout it can't possibly be the right solution.
- Goladus 15y ago
  x = [0] * 100
  - zmmmmm 15y ago
    Thanks! At first I wondered about efficiency but it seems like I can allocate lists with tens of millions of elements this way without any spike in CPU so it is obviously implemented efficiently. I knew there would be a better way :-)
  - Ixiaus 15y ago
    This is much better than my own comment - I never even knew about that syntax! Is it efficient though?
    - Goladus 15y ago
      My guess is that it's optimized, but I don't know. Semantics-wise, it's consistent with list addition. >>> [0] + [0] + [0] [0, 0, 0] >>> [0] * 3 [0, 0, 0]
      - Ixiaus 15y ago
        I just read, that syntax is sugar for the list.extend() method so it's still making copies of the list - I wouldn't use it for a list containing complex objects; but that is very idiomatic for simple use cases (like this one?).
    - calpaterson 15y ago
      It isn't syntax - help(list.__mul__)
Goladus 15y ago
Just a note: the optparse module is deprecated, in python 2.7+ argparse is preferred.
- DasIch 15y ago
  Doesn't really matter though because using argparse means an additional dependency on <2.7. In practice nobody really uses argparse atm.
  - Goladus 15y ago
    It's worth knowing given the intent of the original post.
ryanklee 15y ago
Can anyone suggest any other materials or resources in the same vein as this?
- espeed 15y ago
  See http://www.quora.com/How-can-I-learn-to-program-in-Python/answer/James-Thornton http://www.quora.com/How-can-I-learn-to-program-in-Python/an...
  - ryanklee 15y ago
    Thanks for the reply -- I should have been more specific. I'm after something that similarly contrasts naive and idiomatic styles.
    - espeed 15y ago
      You mean Python specifically or for other programming languages?
      - ryanklee 15y ago
        Python, specifically.
        espeed 15y ago
        * Idioms and Anti-Idioms in Python (http://docs.python.org/howto/doanddont.html) * Python Idioms and Efficiency (http://jaynes.colorado.edu/PythonIdioms.html) * Python Style Guide (http://www.python.org/dev/peps/pep-0008/) * Google Python Style Guide (http://google-styleguide.googlecode.com/svn/trunk/pyguide.html) * Pocoo Style Guide (http://www.pocoo.org/internal/styleguide/)
arctangent 15y ago
One thing that has always bugged me about PEP8 is the advice about trying to keep line length to <80 characters. It seems a little dated. I basically just completely ignore this part of the guidance, even though I know that in theory it makes it hard to edit code via a terminal. But this is not something we ever do in practice where I work - code on servers gets there only by being checked out of SVN. I personally find it incredibly irritating when I see people who've split a function call over a couple of lines just to try and meet this criterion, e.g.: def __init__(self, first, second, third, fourth, fifth, sixth): Note that if you're creating functions with very long signatures you're probably doing something wrong. About the only time I make any kind of concession to the line length guidance is when I initialise a big dictionary, e.g.: my_dict = { ham: eggs, bar: quux, # Snip 10 lines of code rhubarb: custard } (There are often better ways to do this too, but sometimes it's necessary for clarity to build a dictionary all at once like this.) I'm perfectly happy to initialise dictionaries with a small number of keys just like this: my_dict = { ham: eggs, bar: quux, rhubarb: custard }
- gbog 15y ago
  Some people like to split windows vertically, shorter line help them, but splitting lines break greppability, and I am always wondering how to indent them properly.
- jlarocco 15y ago
  The original reason may be dated, but there are other places where limiting line length is still helpful. For example, I'll often have my editor open and docs open in a web browser right next to it. It also helps when viewing side by side diffs.
- notatoad 15y ago
  i love the 80char limit. it makes things nice in a terminal, it lets me tile lots of windows across my screen, and it prevents excessively clever but entirely unreadable one-liners.
- sixtofour 15y ago
  It's easier to read something quickly if your eyes don't have to travel far. This is why newspapers (remember them?) are written in multiple columns across the page. This may be less of an issue with code, due to its structure, but I still find it much easier to read 80 column code than code that stretches across the screen. If I'm reading code in an 80 column terminal I prefer the line breaks to come from the human that wrote it, to show and reveal structure, rather than having it arbitrarily broken at 80 columns by the editor. Another reason is that most of your code won't stretch across the screen, but you need your window sized for the small amount of code that does, losing a bunch of screen estate to empty space. But I don't worry too much about it.
Luyt 15y ago
I wrote an article about the same subject, albeit much shorter and from a slightly different perspective: http://www.michielovertoom.com/python/ugly-python/ http://www.michielovertoom.com/python/ugly-python/
kmfrk 15y ago
This might be completely trivial, but can someone explain to me how one Python idiom is not to mix tabs and spaces, and another is to align arguments over several lines when needed like this: def __init__(self, first, second, third, fourth, fifth, sixth): I see similar examples elsewhere, and I don't see how you can achieve the alignment without mixing in spaces - at least in some cases. I, too, prefer to align arguments, but it looks bad in places like GitHub and Pastebin, when my Sublime Tex-written code gets uploaded. Am I missing something here? The two just seem mutually exclusive in most cases (where the alignment position's required spaces % 4 != 0. Is the indentation done with just spaces then?
- sam 15y ago
  Yes, indentation is done just with spaces. Tabs are not used. Typically you set your text editor to have the tab key print 4 spaces.
  - kmfrk 15y ago
    Ah, there lies the rub. Thanks a bunch.
    - xtacy 15y ago
      EDIT: Emacs is pretty intelligent when it comes to indenting Python code; it does not merely insert 4 spaces for a TAB, but it automatically aligns your code to the Python expression in your previous line! I don't know which one is the "right" way, but I certainly prefer Emacs's default behaviour. Maybe I just didn't configure my Vim properly :-) Example: In Emacs: def _five(one,two, ....,,,,..three,four): In Vim, with 4 space Tabs: def _five(one,two, ....,,,,|hree,four): | => cursor position when you press ENTER.
rubergly 15y ago
Is it really idiomatic to use intrinsic truth values (eg, using "if x" instead of "if x != 0")? Here are my arguments against it: - "Explicit is better than implicit." - If there's need for a table as reference for what is intrinsically true and what is intrinsically false, then it's clearly not "elegant" enough for people to just understand from looking at the code. - Conventions vary across languages. You're basically forcing anyone who looks at your code who's not used tons of Python (and who would otherwise completely understand the code) to look up that table.
- billjings 15y ago
  - But readability counts. Having written a lot of Java and Obj-C lately, I honestly prefer the ability to write "if (foo && foo->bar && foo->bar->baz)" in the latter. - Frankly, it's not a very complicated table. "Zero, empty, or none" summarizes it pretty well. http://docs.python.org/release/2.5.2/lib/truth.html http://docs.python.org/release/2.5.2/lib/truth.html - Of course conventions vary across languages. If you're familiar with perl, awk, or ruby, some of these conventions will be familiar to you. If you're more familiar with Java, they'll feel odd and you'll feel an uncertainty using them.
prolepunk 15y ago
I really liked the part about for ... else statement, the body of else statement executes the code after the loop finishes but not if break statement was executed. I know about this construct but whenever I have a problem that requires it, I always forget and use old: if counter -1 == len(obj): ... ugh.

237 points by ashishgandhi 15y ago

I have just skimmed this so far, but it looks very good. A lot of the Python scripts that I see at work look like either C programs or glorified batch files. I'll definitely point people to this when they are ready and willing to move on to the next level. Well done. Suggestion: one thing I didn't see mentioned is switching from "if s.find(c) == -1: ..." to "if c in s: ...". I see people do that a lot.

epenn 15y ago
one thing I didn't see mentioned is switching from "if s.find(c) == -1: ..." to "if c in s: ...". I see people do that a lot. I think this is covered implicitly by the two "Use in where possible" sections. Although as Tim Peter's The Zen of Python states, which is also quoted in the tutorial, "explicit is better than implicit." If people are doing that despite knowledge of the in keyword then I suppose that just proves Tim's point.
- donatzsky 15y ago
  I don't know. I'd argue that if c not in s: ... is more explicit (and readable) than if s.find(c) == -1: ...
klenwell 15y ago
You just made my weekend. I've been doing Python for a while and am well acquainted with the "item in list" idiom but did not know it could be used for find. That str.find syntax has always bothered me. Still, I had to test just to be sure: test_cases = [ # haystack, needle, expect ('abcdefg', 'a', True), ('abcdefg', 'b', True), ('abcdefg', 'bcd', True), ('abcdefg', 'h', False), ] for haystack,needle,expect in test_cases: # find version is_found = haystack.find(needle) != -1 # in version is_in = needle in haystack # confirm print haystack, needle, expect, '-->', (is_found, is_in) assert is_found == expect assert is_found == is_in Passed! Thanks.

sneak 15y ago

He had me up until the part where he wrote an algorithm to specifically leave out the Oxford Comma. :) Kidding aside, this was a great read.

Luyt 15y ago
As he remarks, he doesn't take in account the corner cases. Suprisingly, or maybe not, his concatenation statement is similar to what I regularly use: def commalist(f, endword="and"): "Concatenate a list of strings in the form of 'Red, Yellow, Green and Blue'." if len(f) == 0: return "" elif len(f) == 1: return f[0] else: return ", ".join(f[:-1]) + " " + endword + " " + f[-1]

zachallaun 15y ago

Read from the perspective of someone very much still learning to code, this was excellent. I particularly valued the idiomatic comparisons to 'naive' implementations (the kind I would initially reach for). One of the later sections on lazy generation also opened up some new doors for me, and gave me a nice solution to a problem I've been having with a piece of code.

hello_moto 15y ago

In the Java ecosystem, they call it best practices. But of course using the phrase "best practices" these days is considered taboo. Thus the use of the word "idiomatic"?

Ixiaus 15y ago
A "best practice" is essentially the same thing as "idiomatic python" in my opinion - I could pick apart different meanings from each but often times, it really is used to say the same thing. The phrase "best practice" isn't taboo in my experience - I'll often ask what the "best practice would be for implementing XYZ". If you rephrase that, "what is an idiomatic way to implement XYZ", it's saying the same thing. Python's community probably leans toward this diction because the culture of python revolves around the nebulous concepts of simplicity, obviousness, and "only one way", that in order to make those concepts more hylic and less "abstract and subjective" there is a lot of education (in the form of docs, PEP8, the very doc linked to in this post) to set the stage for what the expected idiom is.
Goladus 15y ago
A "best practice" is an optimal way to do something in a given context. An "idiom" is a commonly accepted way to express the thing you want to do. Best practices and idioms overlap, but are not identical.
mekoka 15y ago
http://dictionary.reference.com/browse/idiom http://dictionary.reference.com/browse/idiom An idiom is rather a preferred practice. If you take some time to read through some Python source, you'll notice that, although the code uses the exact same constructs that you are used to in Java, it does so in a manner you might find unfamiliar or unpredictable. One example is the use of try/except instead of if/else (beg for forgiveness rather than ask permission). Another is Duck-typing rather than Interface (if it quacks like a duck...) Coming from Java and looking at some Python code without forewarning or context, you might just be tempted to label the whole thing as really really bad practice, when in truth, it's just Idiomatic Python.
- 15y ago

zmmmmm 15y ago

In case there are Pythonistas here who will indulge a question: one thing I've never figured out how to do elegantly ("idiomatically", I suppose) is create a list of predetermined size initialized with a constant(usually zeros). I always end up with something kludgey like: x = [0 for i in xrange(0..100)] which seems so roundabout it can't possibly be the right solution.

Goladus 15y ago
x = [0] * 100
- zmmmmm 15y ago
  Thanks! At first I wondered about efficiency but it seems like I can allocate lists with tens of millions of elements this way without any spike in CPU so it is obviously implemented efficiently. I knew there would be a better way :-)
- Ixiaus 15y ago
  This is much better than my own comment - I never even knew about that syntax! Is it efficient though?
  - Goladus 15y ago
    My guess is that it's optimized, but I don't know. Semantics-wise, it's consistent with list addition. >>> [0] + [0] + [0] [0, 0, 0] >>> [0] * 3 [0, 0, 0]
    - Ixiaus 15y ago
      I just read, that syntax is sugar for the list.extend() method so it's still making copies of the list - I wouldn't use it for a list containing complex objects; but that is very idiomatic for simple use cases (like this one?).
  - calpaterson 15y ago
    It isn't syntax - help(list.__mul__)

Goladus 15y ago

Just a note: the optparse module is deprecated, in python 2.7+ argparse is preferred.

DasIch 15y ago
Doesn't really matter though because using argparse means an additional dependency on <2.7. In practice nobody really uses argparse atm.
- Goladus 15y ago
  It's worth knowing given the intent of the original post.

ryanklee 15y ago

Can anyone suggest any other materials or resources in the same vein as this?

espeed 15y ago
See http://www.quora.com/How-can-I-learn-to-program-in-Python/answer/James-Thornton http://www.quora.com/How-can-I-learn-to-program-in-Python/an...
- ryanklee 15y ago
  Thanks for the reply -- I should have been more specific. I'm after something that similarly contrasts naive and idiomatic styles.
  - espeed 15y ago
    You mean Python specifically or for other programming languages?
    - ryanklee 15y ago
      Python, specifically.
      - espeed 15y ago
        * Idioms and Anti-Idioms in Python (http://docs.python.org/howto/doanddont.html) * Python Idioms and Efficiency (http://jaynes.colorado.edu/PythonIdioms.html) * Python Style Guide (http://www.python.org/dev/peps/pep-0008/) * Google Python Style Guide (http://google-styleguide.googlecode.com/svn/trunk/pyguide.html) * Pocoo Style Guide (http://www.pocoo.org/internal/styleguide/)

arctangent 15y ago

One thing that has always bugged me about PEP8 is the advice about trying to keep line length to <80 characters. It seems a little dated. I basically just completely ignore this part of the guidance, even though I know that in theory it makes it hard to edit code via a terminal. But this is not something we ever do in practice where I work - code on servers gets there only by being checked out of SVN. I personally find it incredibly irritating when I see people who've split a function call over a couple of lines just to try and meet this criterion, e.g.: def __init__(self, first, second, third, fourth, fifth, sixth): Note that if you're creating functions with very long signatures you're probably doing something wrong. About the only time I make any kind of concession to the line length guidance is when I initialise a big dictionary, e.g.: my_dict = { ham: eggs, bar: quux, # Snip 10 lines of code rhubarb: custard } (There are often better ways to do this too, but sometimes it's necessary for clarity to build a dictionary all at once like this.) I'm perfectly happy to initialise dictionaries with a small number of keys just like this: my_dict = { ham: eggs, bar: quux, rhubarb: custard }

gbog 15y ago
Some people like to split windows vertically, shorter line help them, but splitting lines break greppability, and I am always wondering how to indent them properly.
jlarocco 15y ago
The original reason may be dated, but there are other places where limiting line length is still helpful. For example, I'll often have my editor open and docs open in a web browser right next to it. It also helps when viewing side by side diffs.
notatoad 15y ago
i love the 80char limit. it makes things nice in a terminal, it lets me tile lots of windows across my screen, and it prevents excessively clever but entirely unreadable one-liners.
sixtofour 15y ago
It's easier to read something quickly if your eyes don't have to travel far. This is why newspapers (remember them?) are written in multiple columns across the page. This may be less of an issue with code, due to its structure, but I still find it much easier to read 80 column code than code that stretches across the screen. If I'm reading code in an 80 column terminal I prefer the line breaks to come from the human that wrote it, to show and reveal structure, rather than having it arbitrarily broken at 80 columns by the editor. Another reason is that most of your code won't stretch across the screen, but you need your window sized for the small amount of code that does, losing a bunch of screen estate to empty space. But I don't worry too much about it.

Luyt 15y ago

I wrote an article about the same subject, albeit much shorter and from a slightly different perspective: http://www.michielovertoom.com/python/ugly-python/ http://www.michielovertoom.com/python/ugly-python/

kmfrk 15y ago

This might be completely trivial, but can someone explain to me how one Python idiom is not to mix tabs and spaces, and another is to align arguments over several lines when needed like this: def __init__(self, first, second, third, fourth, fifth, sixth): I see similar examples elsewhere, and I don't see how you can achieve the alignment without mixing in spaces - at least in some cases. I, too, prefer to align arguments, but it looks bad in places like GitHub and Pastebin, when my Sublime Tex-written code gets uploaded. Am I missing something here? The two just seem mutually exclusive in most cases (where the alignment position's required spaces % 4 != 0. Is the indentation done with just spaces then?

sam 15y ago
Yes, indentation is done just with spaces. Tabs are not used. Typically you set your text editor to have the tab key print 4 spaces.
- kmfrk 15y ago
  Ah, there lies the rub. Thanks a bunch.
  - xtacy 15y ago
    EDIT: Emacs is pretty intelligent when it comes to indenting Python code; it does not merely insert 4 spaces for a TAB, but it automatically aligns your code to the Python expression in your previous line! I don't know which one is the "right" way, but I certainly prefer Emacs's default behaviour. Maybe I just didn't configure my Vim properly :-) Example: In Emacs: def _five(one,two, ....,,,,..three,four): In Vim, with 4 space Tabs: def _five(one,two, ....,,,,|hree,four): | => cursor position when you press ENTER.

rubergly 15y ago

Is it really idiomatic to use intrinsic truth values (eg, using "if x" instead of "if x != 0")? Here are my arguments against it: - "Explicit is better than implicit." - If there's need for a table as reference for what is intrinsically true and what is intrinsically false, then it's clearly not "elegant" enough for people to just understand from looking at the code. - Conventions vary across languages. You're basically forcing anyone who looks at your code who's not used tons of Python (and who would otherwise completely understand the code) to look up that table.

billjings 15y ago
- But readability counts. Having written a lot of Java and Obj-C lately, I honestly prefer the ability to write "if (foo && foo->bar && foo->bar->baz)" in the latter. - Frankly, it's not a very complicated table. "Zero, empty, or none" summarizes it pretty well. http://docs.python.org/release/2.5.2/lib/truth.html http://docs.python.org/release/2.5.2/lib/truth.html - Of course conventions vary across languages. If you're familiar with perl, awk, or ruby, some of these conventions will be familiar to you. If you're more familiar with Java, they'll feel odd and you'll feel an uncertainty using them.

prolepunk 15y ago

I really liked the part about for ... else statement, the body of else statement executes the code after the loop finishes but not if break statement was executed. I know about this construct but whenever I have a problem that requires it, I always forget and use old: if counter -1 == len(obj): ... ugh.