27 ms·

Common Python Mistakes

• ajanuary 12y ago
  The scoping rules in #4, combined with being able to reference variables before definite assignment, is what leads to the 'variable hoisting' in Javascript. Is #6 really called 'late binding'? That seems like the wrong term.
  • mguillech 12y ago
    Please check out http://en.wikipedia.org/wiki/Late_binding http://en.wikipedia.org/wiki/Late_binding to know more.
    • ajanuary 12y ago
      I've always heard it used to refer to method dispatch, which that wikipedia article also seems to. However, it seems like the Python spec does use it to refer to when variable values are resolved.
• ygra 12y ago
  And I would have thought that incorrect usage of bytestrings for text and then asking on Stack Overflow about the UnicodeDecodeErrors would be quite common as well ...
  • gtaylor 12y ago
    Yeah, bigtime. I'm not sure that would make for a quick, easy countdown list of a blog post, though.
• herge 12y ago
  My pet peeve with python is the classic: return x, I have never wanted to declare a tuple without surrounding it with (). Too bad it's not a syntax error in python 3. Also, as opposed to one of his examples, if you are using python 2.7, declare your exception blocks as: except (FooException, BarException) as e: It's forward compatible with python 3, it's easier to read and the syntax errors are clearer.
  • icebraining 12y ago
    I have never wanted to declare a tuple without surrounding it with (). No? I do that occasionally, e.g.: x, y = 5, 6
    • akx 12y ago
      Doing (x, y) = (5, 6) makes it clearer to "visually grep" that it's not a normal assignment though.
      • icebraining 12y ago
        Fair enough, that would be fine by me.
      • rnnr 12y ago
        It's the comma that determines an expression as a tuple, not the parentheseis.
        • lucian1900 12y ago
          Which is the source of the original problem, which I've also encountered myself.
        • yeukhon 12y ago
          That syntax reminds me of trying to do the following print x, print y But I generall find myself doing the , in the situation like (element1,) to create a new tuple.
        • herge 12y ago
          Not all the time. x = , causes a syntax error, while x = (,) creates an empty tuple.
          • dalke 12y ago
            Under both Python 2.7 and Python 3.3 I get: >>> x=(,) File "<stdin>", line 1 x=(,) ^ SyntaxError: invalid syntax
• leephillips 12y ago
  Is the explanation for #1 correct? "when the default value for a function argument is an expression, the expression is evaluated only once" I would explain the behavior he shows as due to the default value being mutable. I don't see an expression there, just an empty list used as a default.
  • herge 12y ago
    The default value is an expression which is executed when the module is instantiated, even if the expression results in an empty list. If I had: def f(now=datetime.datetime.now()): ... now would be the time when the module was loaded, not when f is called the first time, or when f is called after that, despite datetime objects being immutable.
    • arethuza 12y ago
      Isn't that just now being bound to a different datetime object each time f is defined?
      • mguillech 12y ago
        Right, it's (re) defined every time you call 'foo2' but that's a different scenario than the one written in the article (hadn't you had 'foo2' and only 'foo' then you'd have the same time in all your calls to the function without supplying arguments).
    • kghose 12y ago
      import datetime, time def foo2(): def foo(a=datetime.datetime.now()): time.sleep(1) return a return foo() for n in range(10): print foo2() 2014-05-08 11:23:01.642871 2014-05-08 11:23:02.644276 2014-05-08 11:23:03.644579 2014-05-08 11:23:04.645146 2014-05-08 11:23:05.646328 2014-05-08 11:23:06.647572 2014-05-08 11:23:07.647904 2014-05-08 11:23:08.648213 2014-05-08 11:23:09.648973 2014-05-08 11:23:10.649742 So, not quite module load time, but evaluation time.
      • 12y ago
• Hovertruck 12y ago
  "Thus, the bar argument is initialized to its default (i.e., an empty list) only the first time that foo() is called, but then subsequent calls to foo() (i.e., without a bar argument specified) will continue to use the same list to which bar was originally initialized." This actually happens when the function is defined, not when it's called the first time.
• kghose 12y ago
  Also, for the first example, it is a scope issue: def foo2(): def foo(a=[]): a.append('ba') return a return foo() print foo2() print foo2() print foo2() print foo2() print foo2() Gives ['ba'] ['ba'] ['ba'] ['ba'] ['ba']
  • kilink 12y ago
    I think the way he described it is pretty accurate. Default keyword arguments are only evaluated once at function definition, so supplying a mutable default keyword argument can cause issues. Your example is pretty contrived and doesn't illustrate what he was pointing out, as you're creating a new function foo every time foo2 is called, and only calling it once.
• outworlder 12y ago
  > "Python is an interpreted, object-oriented, high-level programming language with dynamic semantics." I have an issue with that statement. No languages are inherently "compiled" or "interpreted", that's a property of the implementation. If we are talking about CPython here, Python code is compiled to bytecode which is then interpreted. Not unlike Java - with the difference that the main implementation has a JIT and afaik, Python's does not. But that's CPython. What about PyPy? It has a JIT.
  • marcosdumay 12y ago
    High-level is also a relative term (with multivariable semantic, leading to not comparable languages), and the term "object-oriented" is getting less expressive by the day. Anyway, all of those terms do communicate something, even if tomorrow they may wrongly describe the language.
  • mguillech 12y ago
    That's right, in most Python implementations there's a "compiled" part (generally to bytecode) and then an "interpreted" one to run that bytecode. PyPy is a good example of a Python interpreter built on top of RPython (compiled with RPython, that is) adding a JIT to it.
  • Xophmeister 12y ago
    The compiled vs. interpreted thing is kind of a historical relic that people still cling on to. It's not dissimilar to where one draws the line for "scripting" or "nth generation" languages; it's all somewhat nebulously defined.
    • JasonFruit 12y ago
      Do people still talk about nth-generation languages outside explaining why Forth is called Forth? I thought it was mostly history by now.
      • Xophmeister 12y ago
        From my experience, the only time I've heard it nowadays is from people who were taught how to code by people who haven't coded since at least the 80s and never went on to develop the skill either professionally or as a hobby. So, yes, very much a historical relic!
• gejjaxxita 12y ago
  #6 is really confusing. Whenever I encounter something like this my first reaction is that whenever possible such obscure components of a language should be avoided and more verbose/clear code used instead. Programming languages are meant to be read as well as written, and someone relatively new to Python (and many who have used the language for a long time) is certain to get confused about the difference between: return [lambda x, i=i : i * x for i in range(5)] and return [lambda x : i * x for i in range(5)]
  • mguillech 12y ago
    Agreed 100%, this type of constructions should be avoided in the first place in favor of more "readable" ones but this happens in a fair amount of code that I've seen (and I keep seeing).
    • maxerickson 12y ago
      Some of it seems to come from people cargo-culting their knowledge of anonymous and first class functions, so they end up believing that the only way to pass a function around is to construct it anonymously.
      • nine_k 12y ago
        In the particular case of constructing several similar functions that do essentially the same thing, lambda is a rather natural choice. Being more explicit and less hacky can well be combined with staying true to functional style: from functools import partial mul = lambda x, y: x*y # could use int.__mul__, too multipliers = [partial(mul, n) for n in range(5)] It does the closure-capturing of n for you.
        • maxerickson 12y ago
          A function defined with def would work just the same there. It would have a big ugly 'return' in it and be a few characters longer, but it would work the same, so I don't see what lambda brings to it.
• mark-r 12y ago
  I've always thought that #1 is a sign of an incorrect operation altogether. If you want to always modify the passed parameter, it doesn't make sense to have a default. If you want to return a modified version of the input, you should make a copy immediately and then you don't get this problem. Doing both an in-place modification and returning a modified object at the same time is just wrong.
  • ajanuary 12y ago
    A slightly more realistic example: class Bag(object): def __init__(self, items=[]): self.items = items def add_item(self, item): # check the item is valid self.items.append(item) bag1 = Bag() bag1.add('an item') bag2 = Bag() print(bag2.items)
    • mark-r 12y ago
      Again, the problem is not what it appears. You're keeping a reference to an existing item rather than making a copy. The results would be just as bad if you passed in an initial list rather than taking the default. initial = ['first item'] bag3 = Bag(initial) bag3.add_item('second item') print(initial) I think the surprising thing to most people is that you don't automatically get a copy when you do the assignment. That's how it works in older languages like C and C++, and how it appears to behave when you use immutable objects.
• picanteverde 12y ago
  Great Article!
• cefstat 12y ago
  I have been bitten by #6 in a similar situation in the past. My solution was the analogue of the rather convoluted def create_multipliers(): def multiplier(i): return lambda x: i*x return [multiplier(i) for i in range(5)] for multiplier in create_multipliers(): print multiplier(2) I would still prefer that Python doesn't do this.
• cridenour 12y ago
  For #7, now you have a performance problem of importing every time you run that function. Rather, you can place the import at the bottom of b.py and be okay.
  • jagger27 12y ago
    The author states that Python is smart enough not to reimport a file that has already been imported. Is that not the case here?
  • mguillech 12y ago
    Python caches modules imported, you can check it out in your local shell with: import sys ; sys.modules. That's why whenever you make changes to a module which has been already loaded you won't see the changes until you load the module again, either quitting the shell or using reload(module) on Python 2.x
• lukasm 12y ago
  For #4 use nonlocal keyword (python 3.x)
• famousactress 12y ago
  This list is an excellent summary. If tasked with a #11 I'd probably add the slightly more obscure, but still super painful (when you do run into it) implicit string concatenation: >>> l = ["a", ... "b", ... "c" ... "d"] >>> l ['a', 'b', 'cd']
  • yeukhon 12y ago
    I never knew you could do that. You don't need the continuation though. >>> l = ["A", "c""d"] >>> l ['A', 'cd']
    • mark-r 12y ago
      You don't need the continuation, but that's where you're most likely to make the mistake. It'll catch you when you leave off the comma from the last element of the list, then go back later and add another element to the end.
      • yeukhon 12y ago
        Sorry, I am not followed. Can you show it instead? Thanks!
    • ColinWright 12y ago
      You need to re-format that with leading spaces to retain the indentation.
  • nine_k 12y ago
    Still hugely useful when you want to write long string constants. Imho, adding a comma after each list element is a good practice. You can easily swap them, add more, and never run into a an issue you describe: foo = [ "a", "bc", "def", # comma here, too ]
    • fragmede 12y ago
      This is preferred! (It also makes diffs more readable)
• agf 12y ago
  This is a pretty good list of gotchas, but it's important when writing something targeted at beginners to be as precise and clear as possible. Nearly every section here either uses terminology poorly, is slightly incorrect, or has difficult examples. Python supports optional function arguments and allows default values to be specified for any optional argument. No, specifying a default is what causes an argument to be optional. it can lead to some confusion when specifying an expression as the default value for an optional function argument. Anything you specify as a default value is an expression. The problem is when the default is mutable. the bar argument is initialized to its default (i.e., an empty list) only the first time that foo() is called No, rather it's when the function is defined. class variables are internally handled as dictionaries As dictionary keys, and that's still only roughly correct. In "Common Mistake #5", he uses both a lambda and array index based looping, neither of which are particularly Pythonic. A better example of where this is a problem in otherwise Pythonic code would be good. In "Common Mistake #6" he uses a lambda in a list comprehension -- for an article of mistakes mostly made by Python beginners, this is going to make it tough to follow the example. In "Common Mistake #7", he describes "recursive imports" where he means "circular imports". In "Common Mistake #8" he refers repeatedly to "stdlib" where he means the Python Standard Library. Someone is going to read that and try to "import stdlib".
  • hsinger 12y ago
    Hey, thanks for the great feedback! We agreed with (almost :-) ) all of your comments and have made corresponding mods/corrections to the post. Thanks again! [Toptal blog editor]
    • agf 12y ago
      Good changes. One more issue (I believe recently introduced): LEGB ends with "Built-in", not with "Module". It's also good to note in the blog post that it's been updated.
      • hsinger 12y ago
        Doh. You sure it doesn't stand for "Mbuilt-in"? ;-) Again, thanks for the attention to detail. We've fixed this as well.
• tomp 12y ago
  Another one: class A(): def __init__(self): self._x = 0 @property def x(self): return self._x @x.setter def x(self, new_value): self._x = new_value Using it: a = A() print a._x # 0 print a.x # 0 a.x = 4 print a.x # 4 print a._x # 0 wait WTF?! The bug was not having `A` inherit from `object`. With old-style classes, properties do not work correctly.
  • Redoubts 12y ago
    is def x(self, new_value): self._x = x really what you mean? not _x = new_value?
    • tomp 12y ago
      You're right, I fixed it.
  • Spittie 12y ago
    For anyone wondering, this works just fine in Python 3 :)
    • shotwell 12y ago
      Yes, but just because all classes in python 3 are new-style.
    • tomp 12y ago
      That's correct, but the post was targeted to Python 2(.7, I assume), so I thought it would fit :)
• worklogin 12y ago
  Even the first argument doesn't make sense to me. The optional argument is within the scope of the function; why is the temporary optional argument getting carried over?
• yeukhon 12y ago
  Circular import is definitely one of the things I absolutely hate to deal with again and again.
• Goosey 12y ago
  Slightly off topic, but does anyone know of a resource that has 'most common mistakes' for different languages all in one place? It's certainly possible to google for blog posts and stack overflow questions to assemble such a list, but it would be handy to have them all in one place. My use case is when interviewing candidates I often ask them to rate themselves on a scale of 1-5 in the languages they know, and then ask them increasingly 'tricky' questions in each language to get a feel for how their "personal" scale aligns to their real knowledge. This works fine if we have an overlap of several languages, but in the case where I know nothing or very little of one of the languages they know I lose that data point. I find it valuable to know what a "I am a 1 at X" vs "I am a 3 X" vs "I am a 5 at X" means to them, since I've found little correlation between how harshly someone rates themselves and their true ability. Sometimes self-rated 5s are really 5s by my book, sometimes self-rated 3s are really 5s by my book, and sometimes self-rated 5s are really 2s by my book. So I want to know how "my scale" translates to "their scale". If it was more formalized I'd go as far as to get a "confidence quotient" for a person as self-critical and self-confident people can be fantastic engineers or horrible engineers. Does anyone else do this process when interviewing?
  • mark-r 12y ago
    While such a resource would make your job easier, it would make the interviewee's job easier still. They'd just have to memorize all the points in the reference.
    • timdierks 12y ago
      At which point they will no longer make these mistakes, and thus are more expert. Everyone wins!
      • mark-r 12y ago
        Except that you're risking hiring people that know all the trivia but still can't program FizzBuzz properly. The test loses its predictive value.
        • 12y ago
• andreif 12y ago
  I have just been asked by a colleague about yet another gotcha when you have for example: # mypackage/__init__.py from .settings import settings and when trying to import settings from mypackage from mypackage import settings you get module instead of settings object.
  • kyro 12y ago
    from mypackage.settings import settings
• dmritard96 12y ago
  There are more than a few interesting points in here but this is funny to me coming from someone who is seemingly well versed in python: Mistake #5 numbers = [n for n in range(10)] this should be: range(10)
  • abaschin 12y ago
    not in Python 3, where range() replaces xrange()
    • dmritard96 12y ago
      interesting, haven't written in python 3 yet
  • dragonwriter 12y ago
    Unless its Python 3.x. Though, really, even there, while the list comprehension works, its kind of an awkward construction to use that instead of: numbers = list(range(10))
• deleted 12y ago
  [deleted]
• evincarofautumn 12y ago
  Here’s a thought experiment for you—think of “common mistakes in language X” as “design flaws in language X” or “ways in which language X is surprising” and what could have been done to mitigate that.
  • pekk 12y ago
    Whether you choose 0-based indexing or 1-based indexing, somebody is going to be confused about indexing sometime.
    • JadeNB 12y ago
      That's why I always use 7-based indexing, to avoid any possibility of confusion.
• vram22 12y ago
  In "Common Mistake #2", I'd say that the mistake is fairly obvious to anyone who understands even a little bit about OOP and inheritance. Since class C doesn't define its own variable x, it has to be that it inherits the x in class A, so there's no reason to be surprised that C.x changes when A.x does.
  • radiowave 12y ago
    While I agree that the "problem" case can be seen as obvious when considered in isolation, really it's the behaviour of the two cases taken together that can seem inconsistent. Nothing about understanding OOP or inheritance will prepare a person for that.
    • vram22 12y ago
      >really it's the behaviour of the two cases taken together that can seem inconsistent. Why do you think so? I think that both cases seem consistent, or rather, correct (and therefore this example should not be treated as a common Python mistake), because x is not assigned a value anywhere in class C, and C inherits from A, so it should be clear to anyone knowing OOP and inheritance, that C's x is the same as A's x. (And the same holds true for inherited methods.) Even the OP says that in the post: >In other words, C doesn’t have its own x property, independent of A.
      • radiowave 12y ago
        What's happening here is that a variable is inheriting its value from the superclass, except for when it doesn't. And when it doesn't, why is that? Well presumably it's because something's been overridden - OO tells us that's how we change the properties that are inherited from the superclass. No wait, that's not it; nothing's been overridden here. All that's happened is we've assigned a value to B.x, and doing so seems to have changed the inheritance of our class. So this variable is neither completely shared across classes and their subclasses (per Smalltalk class variables), nor completely independent across classes and their subclasses (per Smalltalk class instance variable), but instead its [in]dependence alters based upon whether (and where) you assign values to it. While I can understand that in terms of the dictionary mechanism used to implement it, from my point of view it's just weird behaviour.
• codezero 12y ago
  OT but this page hard crashes Safari on iPhone.
  • brown9-2 12y ago
    Very reproducibly too.
  • jordigh 12y ago
    File a bug report with Apple? No input should crash a program.
    • codezero 12y ago
      I'll do that. In general Safari doesn't crash so I was letting the OP know for their own benefit as they are in a better position to isolate the cause and work around it.
  • peter_l_downs 12y ago
    Also crashes Safari on my OS 10.5 Mac, and is unusably laggy in Firefox on the same computer. All sorts of thrashy javascript nonsense seems to be going on.
    • rmrfrmrf 12y ago
      I use JavaScript Blocker for Safari, which is sort of like a less paranoid (and more convenient) version of NoScript. Looks like this site attempts to load 45 JavaScript files over 12 iframes, 17 of which JS Blocker blocked. Mildly excessive? /s
  • japaget 12y ago
    It crashes with Chrome also, but I was able to read the page with Opera Mini. iPhone 5, iOS 7.1.
  • mctx 12y ago
    Also crashes chrome on ios 7.1.1 on an iPhone 5S every time
• andrewdon 12y ago
  good stuff
• euske 12y ago
  Some of the mistakes mentioned in OP (#1, #3, and #4) can be automatically caught by tools like PyLint (and to lesser extent, Pyflakes), as well as good unittests.
• baby 12y ago
  Does someone knows if the first mistake is also present in Ruby? I ran into this code: def get_analytics_data(options = {}) options = options.merge({'ids' => GAReadonly.configuration.id }) and I wonder if I should fix it.
  • ludamad 12y ago
    It's fine, Ruby takes the more sensible semantics here (IMO)
  • yxhuvud 12y ago
    No need. You can even refer to earlier arguments in the list like def foo a, b=a.succ b end foo(1) => 2
• adidash 12y ago
  Bookmarked it. Might not be the most complete list but appreciate the OP is taking feedback and updating it. Kudos to hsinger!
• heyjonboy 12y ago
  Does this link crash anyone else's Safari on iPhone 5s?
  • deleted 12y ago
    [deleted]
• michaelmior 12y ago
  For the first gotcha, using None as a default argument solves the problem, but checking `if not bar` instead of `if bar is None` can produce different results if bar evaluates to None in a boolean context. >>> def foo(bar=None): ... if not bar: ... bar = [] ... bar.append("baz") ... return bar ... >>> bar = [] >>> foo(bar) ["baz"] >>> bar []
  • icebraining 12y ago
    True, but foo shouldn't be modifying bar anyway; make a copy instead: bar = list(bar) if bar else []
• warriar 12y ago
  Common Mistake: ceate a blog that brings mobile Safari to crash reliably on every visit!
• ronaldx 12y ago
  I now feel dumb, since Python most often burns me with "Assignment Creates References, Not Copies".
• skywhopper 12y ago
  Hmm, "Common Mistake #1" appears to be an error on the side of the Python interpreter, not the programmer.
• udioron 12y ago
  Regarding common mistake #5, when using enumerate(mylist), mylist can be modified:: for i, x in enumerate(numbers): if odd(x): del numbers[i]
• tom_jones 12y ago
  Any reason why you're using a slice here? >>> numbers[:] = [n for n in numbers if not odd(n)] I'm thinking that doing >>> numbers = [n for n in numbers if not odd(n)] wouldn't be a problem since the assignment is executed after the computation of the list comprehension.
• udioron 12y ago
  Regarding circular imports and #7: The main problem in arises when using the from mymodule import mysymbol notion. The example solved this by properly using import mymodule, although this might cause some more problem if your design is wrong, as see in the example. Calling f() from the module ("library") code itself is a very bad idea. Instead one should do this: a.py: import b def f(): return b.x b.py: import a x = 1 def g(): print a.f() main.py: import a a.f()