5 ms·

>> in Python the number of times something like this happens Anec-data but I’ve been writing python for 15+ years, I’ve contributed to various popular open sou

>> in Python the number of times something like this happens Anec-data but I’ve been writing python for 15+ years, I’ve contributed to various popular open source projects. I’ve never seen this in a code review. I’ve certainly never seen this kind of mistake released.

NavinF 5y ago
I've seen a variation of this bug several times in just the last year: myVariable = 1 if not something_unusual(): myVariablr = 2 return myVariablr This code will work just fine until something_unusual() returns True and then it crashes with UnboundLocalError: local variable 'myVariablr' referenced before assignment This really sucks when it happens during a long-running job. Say for example you're looping through an array in chunks and distributing each chunk across GPUs. The last chunk will have fewer elements than all the others, maybe even 1 element. If you don't pad the chunk, it will go through a different (and likely underutilized) code path downstream and crash like this.
- formerly_proven 5y ago
  Dynamic languages often benefit from some tool assistance. Both your and GPs experience can happen at the same time: GP is probably using a linter or an IDE (like Pycharm) which will point out that "myVariablr" may be undefined.
  - rich_sasha 5y ago
    If you're returning myVariable instead, that's "fine" - it is always defined. You might get a warning that myVariabl_r_ is unused, but that's also not a given. The bar for being 'used' is pretty low. For example, in this def main(): x = 10 l = [] l.append(x) return 5 pyflakes believes both x and l are "used", even though neither affects function return value.
    - formerly_proven 5y ago
      In Python even the trivial definition of "used" (i.e. "bound name which is referred to afterwards") is not that easy (e.g. getattr), linters go for the literal definition instead ("bound name, which is statically referred to afterwards"). This works for 99 % of code. Your definition of "used" (~"influences the observable result") is of course fully within the dominion of Rice's theorem, though this particular example would be caught by any data-flow analyzer.
- fractalb 5y ago
  Those type of errors will be caught by the compiler. It's definitely a problem for python-like languages (but not for compiled languages) because there's no compilation step involved before running the code.
  - NavinF 5y ago
    It’s a compiled language in the same sense that Java is compiled. Most people use the CPython implementation which compiles .py files to .pyc files before executing them. That compiler can’t catch bugs of this form because it’s not always a bug. That’s just how dynamic languages work. But if Python had a different syntax like “let var = value” for creating new variables rather than mutating existing ones, we wouldn’t have this specific problem. That’s what this thread is about. If C++ inferred “auto” for all assignments/initializations, it would have a similar problem where typos in assignments effectively lead to that line of code getting skipped.
- CraigJPerry 5y ago
  VSCode (pylance) & PyCharm both immediately identify myVariable as unused - this is inline in the editor, before any compilation or testing. Assuming somehow this code made it into a pull request (although as above, there would be no reason for that) - then flake8 flags myVariable as unused, your CI pipeline wouldn't get as far as notifying a peer that the PR is ready for review.

NavinF 5y ago

I've seen a variation of this bug several times in just the last year: myVariable = 1 if not something_unusual(): myVariablr = 2 return myVariablr This code will work just fine until something_unusual() returns True and then it crashes with UnboundLocalError: local variable 'myVariablr' referenced before assignment This really sucks when it happens during a long-running job. Say for example you're looping through an array in chunks and distributing each chunk across GPUs. The last chunk will have fewer elements than all the others, maybe even 1 element. If you don't pad the chunk, it will go through a different (and likely underutilized) code path downstream and crash like this.

formerly_proven 5y ago
Dynamic languages often benefit from some tool assistance. Both your and GPs experience can happen at the same time: GP is probably using a linter or an IDE (like Pycharm) which will point out that "myVariablr" may be undefined.
- rich_sasha 5y ago
  If you're returning myVariable instead, that's "fine" - it is always defined. You might get a warning that myVariabl_r_ is unused, but that's also not a given. The bar for being 'used' is pretty low. For example, in this def main(): x = 10 l = [] l.append(x) return 5 pyflakes believes both x and l are "used", even though neither affects function return value.
  - formerly_proven 5y ago
    In Python even the trivial definition of "used" (i.e. "bound name which is referred to afterwards") is not that easy (e.g. getattr), linters go for the literal definition instead ("bound name, which is statically referred to afterwards"). This works for 99 % of code. Your definition of "used" (~"influences the observable result") is of course fully within the dominion of Rice's theorem, though this particular example would be caught by any data-flow analyzer.
fractalb 5y ago
Those type of errors will be caught by the compiler. It's definitely a problem for python-like languages (but not for compiled languages) because there's no compilation step involved before running the code.
- NavinF 5y ago
  It’s a compiled language in the same sense that Java is compiled. Most people use the CPython implementation which compiles .py files to .pyc files before executing them. That compiler can’t catch bugs of this form because it’s not always a bug. That’s just how dynamic languages work. But if Python had a different syntax like “let var = value” for creating new variables rather than mutating existing ones, we wouldn’t have this specific problem. That’s what this thread is about. If C++ inferred “auto” for all assignments/initializations, it would have a similar problem where typos in assignments effectively lead to that line of code getting skipped.
CraigJPerry 5y ago
VSCode (pylance) & PyCharm both immediately identify myVariable as unused - this is inline in the editor, before any compilation or testing. Assuming somehow this code made it into a pull request (although as above, there would be no reason for that) - then flake8 flags myVariable as unused, your CI pipeline wouldn't get as far as notifying a peer that the PR is ready for review.