4 ms·

I didn't explain the second one well. Here's some exact code. group_keys = ... if not isinstance(group_keys, list): groups_keys = [ group_keys ] So r

I didn't explain the second one well. Here's some exact code. group_keys = ... if not isinstance(group_keys, list): groups_keys = [ group_keys ] So rather than listifying the non-list variable, it was creating a new variable. The cause of this bug is that Python doesn't distinguish between declaring new variables and overwriting existing ones.

sreque 7y ago
I still don't think you quite understand what's going on here. Python wouldn't create a new variable in this case. I It would re-assign the value represented by the variable you already assigned once. I agree that it would have been better if Python had explicit variable declarations (this is one of the few things I think Perl got right.). On the other hand, Ruby made this same mistake. If you wrote this code in Javascript you wouldn't get an error, but you would in fact have two different variables. For instance, this code runs for me using node 8: var fun = (bool) => { var x = 1; if (bool) { var x = 2; x += 1; } else { x += 1; } console.log("x=" + x); } fun(1); fun(0);
- CydeWeys 7y ago
  I don't understand why you're accusing me of being irrational. These seem like very rational problems to have with Python. They literally caused me bugs that cost me time to deal with that I wouldn't have faced in other languages. You're also assuming that I don't have the same problems with Ruby or JavaScript. I do. The exact same critique could be made of them as well, but they're not the subject of this thread; Python is.
  - sreque 7y ago
    My bad. I didn't realize you were against all dynamic languages in particular. FWIW I prefer Java and static types as well, but as far as scripting languages go, I think Python is pretty great.
  - rohan1024 7y ago
    You can't argue with someone who has chosen to overlook your viewpoint. I've ran into the same issues while writing python code. People who are newly picking up python are especially prone to these kind of bugs. Also, with python I have to spend lot of time to figure out what went wrong in my code as compared to other languages. People who have been using python for long have wired there brain to avoid such pitfalls and now they happily defend it.
    - sreque 7y ago
      I don't think what you're saying is true. I already said I think it would have been better if Python and Ruby had explicit variable declarations. But, if this is your biggest issue with a language and it's ecosystem, then IMO that language is doing pretty well. I would rather, for instance, have to deal with implicit variable declarations in Python that the gigantic mess of Java frameworks that have been invented to "reduce boilerplate", such as Spring/Guice, AspectJ, Hibernate, etc.
- winstonewert 7y ago
  > I still don't think you quite understand what's going on here. Python wouldn't create a new variable in this case. I It would re-assign the value represented by the variable you already assigned once. Uh. no. he typoed the reassignment, so it wouldn't re-assign the value. > So, of three of the most popular dynamic languages, Python, Ruby, and Javascript, none of them would have helped you catch this kind of error at script-parsing time. So again, it seems like you have an irrational dislike for Python, all things considered. Sure, but he's made it clear he likes Java. Fundamentally he's against dynamic typing, so of course he doesn't like any of the dynamic languages.
  - sreque 7y ago
    Ah my bad. I think I would have understood it more if he included the typo in his example.
    - winstonewert 7y ago
      He did. But that just illustrates why this kind of bug is so annoying, its hard to spot.
jldugger 7y ago
Well, this should have been caught as an unused assignment in static analysis. A whole ton of languages allow this situation, so I'm not gonna ding Python too hard for that one. However, here's a related but different python gotcha: if foo(a): v = list(bar(a)) for i in v: print i In this example, v is only defined inside the if. Due to python's limited scopes, v is also valid outside the if, but only has an assignment when foo(a) is True. When foo(a) is false, the for loop throws an NameError. And yes, a coworker wrote code that accidentally implemented this antipattern, albeit much more spread out in the code base. This is clearly a bug in the code, yet no static analysis tools I've tried have successfully discovered it. There's a bug in pylint that's been marked WONTFIX because it requires a wildly different implementation. At a language level, it feels weird that if blocks aren't a scope level for new variables. If you want to reference v outside the if loop, declare / assign it outside the loop first.
- takeda 7y ago
  mypy was able to detect these kinds of issues for me
- joshuamorton 7y ago
  Indeed, as another user mentions, mypy will detect this issue, as will pytype, even without any annotations.
  - jldugger 7y ago
    Interesting, I had not looked at these because I'm not interested in volunteering to add type notations to our code base.
- marmaduke 7y ago
  I'm retty sure PyCharm catches that 2nd one with a warning
takeda 7y ago
I see this often and it is a bad pattern that people do. Typically type checked languages wouldn't even allow you to do. If you would use mypy for type checking it wouldn't like it because you're redefining the type of a variable. Best practices would suggest you use a different variable for the conversion if you must, but ideally you should just make the function accept list as an argument. If you're really worried about passing something else than a list, you should use type annotations to tell type checker what it is. If you want to add extra runtime check then do: assert isinstance(group_keys, list) You can complain that Python allowed you to something dangerous, but you have tools to help you avoid it and this flexibility is what makes tools like SQLAlchemy so powerful.
joshuamorton 7y ago
FYI, the google style guide (or maybe the internal only version) suggests to avoid initialize-then-assign in favor of single assignment form: unclear_type_thing = ... if isinstance(unclear_type_thing, list): group_keys = unclear_type_thing else: group_keys = [unclear_type_thing] statically avoids this problem. In general, prefer immutable variables where possible. Single-assignment form is nice for a lot of reasons, not the least of which is that it avoids this particular gotcha. And I should add that the "right" way to do this would be to factor this out to a function: group_keys = coerce_to_list(...) is much clearer than either block, and avoids the possibility of the issue.
- CydeWeys 7y ago
  All of these things are true, but they require a non-trivial level of experience and discipline to avoid most potential gotchas. Your average Python project on the Web isn't written to this level of quality, and when people are learning programming using Python in school they certainly aren't there yet, and are gonna hit all kinds of problems related to this stuff. But is there a way to force immutable variables in Python? You can easily still end up in the same situation when you typo something (easy to do when plurals are involved), and then end up reassigning something when you meant to create a new variable.
  - _coveredInBees 7y ago
    I don't think that's fair to be honest. If you had simply used Pycharm with default settings you would have easily caught the first bug due to the linting. It's a fair complaint, but this specific bug is easy to catch using any modern Python IDE.
    - CydeWeys 7y ago
      I've never found the "Use this specific IDE" defense particularly valid, considering that many IDEs don't have these features and that in other languages the compiler itself protects you. Needless to say, I was not using Pycharm for this development, nor am I likely to install an entire IDE just for a small change I'm making on a random project. It's a non-trivial burden to configure and learn an entire IDE, vs just using what I already know (which is often just emacs).

sreque 7y ago

I still don't think you quite understand what's going on here. Python wouldn't create a new variable in this case. I It would re-assign the value represented by the variable you already assigned once. I agree that it would have been better if Python had explicit variable declarations (this is one of the few things I think Perl got right.). On the other hand, Ruby made this same mistake. If you wrote this code in Javascript you wouldn't get an error, but you would in fact have two different variables. For instance, this code runs for me using node 8: var fun = (bool) => { var x = 1; if (bool) { var x = 2; x += 1; } else { x += 1; } console.log("x=" + x); } fun(1); fun(0);

CydeWeys 7y ago
I don't understand why you're accusing me of being irrational. These seem like very rational problems to have with Python. They literally caused me bugs that cost me time to deal with that I wouldn't have faced in other languages. You're also assuming that I don't have the same problems with Ruby or JavaScript. I do. The exact same critique could be made of them as well, but they're not the subject of this thread; Python is.
- sreque 7y ago
  My bad. I didn't realize you were against all dynamic languages in particular. FWIW I prefer Java and static types as well, but as far as scripting languages go, I think Python is pretty great.
- rohan1024 7y ago
  You can't argue with someone who has chosen to overlook your viewpoint. I've ran into the same issues while writing python code. People who are newly picking up python are especially prone to these kind of bugs. Also, with python I have to spend lot of time to figure out what went wrong in my code as compared to other languages. People who have been using python for long have wired there brain to avoid such pitfalls and now they happily defend it.
  - sreque 7y ago
    I don't think what you're saying is true. I already said I think it would have been better if Python and Ruby had explicit variable declarations. But, if this is your biggest issue with a language and it's ecosystem, then IMO that language is doing pretty well. I would rather, for instance, have to deal with implicit variable declarations in Python that the gigantic mess of Java frameworks that have been invented to "reduce boilerplate", such as Spring/Guice, AspectJ, Hibernate, etc.
winstonewert 7y ago
> I still don't think you quite understand what's going on here. Python wouldn't create a new variable in this case. I It would re-assign the value represented by the variable you already assigned once. Uh. no. he typoed the reassignment, so it wouldn't re-assign the value. > So, of three of the most popular dynamic languages, Python, Ruby, and Javascript, none of them would have helped you catch this kind of error at script-parsing time. So again, it seems like you have an irrational dislike for Python, all things considered. Sure, but he's made it clear he likes Java. Fundamentally he's against dynamic typing, so of course he doesn't like any of the dynamic languages.
- sreque 7y ago
  Ah my bad. I think I would have understood it more if he included the typo in his example.
  - winstonewert 7y ago
    He did. But that just illustrates why this kind of bug is so annoying, its hard to spot.

jldugger 7y ago

Well, this should have been caught as an unused assignment in static analysis. A whole ton of languages allow this situation, so I'm not gonna ding Python too hard for that one. However, here's a related but different python gotcha: if foo(a): v = list(bar(a)) for i in v: print i In this example, v is only defined inside the if. Due to python's limited scopes, v is also valid outside the if, but only has an assignment when foo(a) is True. When foo(a) is false, the for loop throws an NameError. And yes, a coworker wrote code that accidentally implemented this antipattern, albeit much more spread out in the code base. This is clearly a bug in the code, yet no static analysis tools I've tried have successfully discovered it. There's a bug in pylint that's been marked WONTFIX because it requires a wildly different implementation. At a language level, it feels weird that if blocks aren't a scope level for new variables. If you want to reference v outside the if loop, declare / assign it outside the loop first.

takeda 7y ago
mypy was able to detect these kinds of issues for me
joshuamorton 7y ago
Indeed, as another user mentions, mypy will detect this issue, as will pytype, even without any annotations.
- jldugger 7y ago
  Interesting, I had not looked at these because I'm not interested in volunteering to add type notations to our code base.
marmaduke 7y ago
I'm retty sure PyCharm catches that 2nd one with a warning

takeda 7y ago

I see this often and it is a bad pattern that people do. Typically type checked languages wouldn't even allow you to do. If you would use mypy for type checking it wouldn't like it because you're redefining the type of a variable. Best practices would suggest you use a different variable for the conversion if you must, but ideally you should just make the function accept list as an argument. If you're really worried about passing something else than a list, you should use type annotations to tell type checker what it is. If you want to add extra runtime check then do: assert isinstance(group_keys, list) You can complain that Python allowed you to something dangerous, but you have tools to help you avoid it and this flexibility is what makes tools like SQLAlchemy so powerful.

joshuamorton 7y ago

FYI, the google style guide (or maybe the internal only version) suggests to avoid initialize-then-assign in favor of single assignment form: unclear_type_thing = ... if isinstance(unclear_type_thing, list): group_keys = unclear_type_thing else: group_keys = [unclear_type_thing] statically avoids this problem. In general, prefer immutable variables where possible. Single-assignment form is nice for a lot of reasons, not the least of which is that it avoids this particular gotcha. And I should add that the "right" way to do this would be to factor this out to a function: group_keys = coerce_to_list(...) is much clearer than either block, and avoids the possibility of the issue.

CydeWeys 7y ago
All of these things are true, but they require a non-trivial level of experience and discipline to avoid most potential gotchas. Your average Python project on the Web isn't written to this level of quality, and when people are learning programming using Python in school they certainly aren't there yet, and are gonna hit all kinds of problems related to this stuff. But is there a way to force immutable variables in Python? You can easily still end up in the same situation when you typo something (easy to do when plurals are involved), and then end up reassigning something when you meant to create a new variable.
- _coveredInBees 7y ago
  I don't think that's fair to be honest. If you had simply used Pycharm with default settings you would have easily caught the first bug due to the linting. It's a fair complaint, but this specific bug is easy to catch using any modern Python IDE.
  - CydeWeys 7y ago
    I've never found the "Use this specific IDE" defense particularly valid, considering that many IDEs don't have these features and that in other languages the compiler itself protects you. Needless to say, I was not using Pycharm for this development, nor am I likely to install an entire IDE just for a small change I'm making on a random project. It's a non-trivial burden to configure and learn an entire IDE, vs just using what I already know (which is often just emacs).