5 ms·

A tiny Python exception oddity

89 points by luord 7y ago

lilyball 7y ago
I'm curious why this is reported as a SyntaxError. This doesn't seem like a syntactical problem to me, but a semantical one.
- rwbhn 7y ago
  It's not a valid production for 'atom' in the grammar[1]. https://docs.python.org/3/reference/grammar.html https://docs.python.org/3/reference/grammar.html
  - deathanatos 7y ago
    I presume your parent is referring to the double nonlocal/global statements, which are never valid productions for the atom grammar even when they are validly produced. Here, the production is func_body_suite → stmt+ → (stmt, stmt) → (simple_stmt, simple_stmt) → (small_stmt, small_stmt) → (global_stmt, nonlocal_stmt) ; syntactically, I'm inclided to agree that the code sample seems valid. (It is, of course, semantically malformed.) I suspect that there just never was a separate exception type for this sort of thing.
- dragonwriter 7y ago
  > I'm curious why this is reported as a SyntaxError. "Names listed in a global statement must not be used in the same code block textually preceding that global statement"; the nonlocal declaration is a use, and so that syntax rule is violated. https://docs.python.org/3/reference/simple_stmts.html#grammar-token-global-stmt https://docs.python.org/3/reference/simple_stmts.html#gramma...
  - lilyball 7y ago
    If you can construct an AST, you’ve moved past the syntax stage.
    - dbcurtis 7y ago
      Well, in a pure sense, you are correct. But for practical reasons, it can sometimes be easier to define a parsing grammar that is a little loose, and pick up the stragglers by analyzing the AST. Sometimes the “correct” grammar is needlessly complex and you can reduce the headaches with a quick sanitation pass over the results of a less pure parser.
      - lilyball 7y ago
        Even so, I wouldn't consider this a syntax error, because the error isn't with the syntax but with using an identifier in a context where that particular identifier isn't allowed (but others are). This would make it more akin to a NameError.
- lidHanteyk 7y ago
  Python's grammar is so very rich. Many programs are rejected for "syntactic" reasons if they do not fit into the abstract syntax [0]; I suppose that this could be some sort of "AbstractSyntaxError", if Python had such a thing. For some examples: x == not y Rejected by the parser itself. not x = y Accepted by the parser and AST builder, and only rejected by the bytecode compiler. f(*xs,) Rejected by the parser itself. ... Rejected somewhere between the parser and the AST builder. Valid uses of Ellipsis are actually special-cased; it is not a standard part of the abstract syntax. [0] https://github.com/python/cpython/blob/master/Parser/Python.asdl https://github.com/python/cpython/blob/master/Parser/Python....
  - kccqzy 7y ago
    Why is x == not x rejected by the parser? In many other languages this is perfectly valid and, depending on the semantics, may allow the compiler that into false. (Of course this can't generally be the case where operators are user-overloadable.)
    - js2 7y ago
      Because that's how the grammar is defined: https://docs.python.org/3.8/reference/grammar.html https://docs.python.org/3.8/reference/grammar.html The RHS of the == (comp_op) has to be an expr. You can force "not x" to be an expr by parenthesizing it: x == (not x)
      - kccqzy 7y ago
        That interesting. On the other hand, not x == x is totally valid.
    - lidHanteyk 7y ago
      I don't know man, I didn't do it.
- chirau 7y ago
  The nonlocal declaration is considered usage if you use a global statement in the same textual code block preceding the global itself. That is what makes it a syntax error.
bratao 7y ago
I was impressed by the quality of PyPy errors!
mherdeg 7y ago
I'm impressed by the humility here from someone who (a) is writing a library to replace core traceback behavior in pypi; (b) has doubtless read both cpython and pypi code; (c) nevertheless writes "While I have some general idea of how the CPython interpreter works, I absolutely do not understand well enough to claim with absolute certainty how this situation arise". Jeepers, who WOULD be qualified to explain for sure how this works?
- jedberg 7y ago
  > Jeepers, who WOULD be qualified to explain for sure how this works? Guido?
_ZeD_ 7y ago
I'm wondering if this case will be handled differently by the new pegen https://github.com/gvanrossum/pegen https://github.com/gvanrossum/pegen
underdeserver 7y ago
If 99.999% of Python programmers won't care about this, and there are approximately 8.2M Python programmers in the world [1], then only 82 people will care about this. Given the number of upvotes you've covered them all. [1] https://www.google.com/search?q=how+many+python+programmers+are+there+in+the+world https://www.google.com/search?q=how+many+python+programmers+...
- aroberge 7y ago
  Thank you. I actually thought I was overestimated the interest. Then again, I did not think that my blog post would attract the interest of the HackerNews readers.
Wheaties466 7y ago
I seem like i'm alone in this but I think I would prefer the CPython handling of this "Syntax Error". I am not a "Programmer" so having feedback on the original problem line would be more beneficial than returning the second.

89 points by luord 7y ago

I'm curious why this is reported as a SyntaxError. This doesn't seem like a syntactical problem to me, but a semantical one.

rwbhn 7y ago
It's not a valid production for 'atom' in the grammar[1]. https://docs.python.org/3/reference/grammar.html https://docs.python.org/3/reference/grammar.html
- deathanatos 7y ago
  I presume your parent is referring to the double nonlocal/global statements, which are never valid productions for the atom grammar even when they are validly produced. Here, the production is func_body_suite → stmt+ → (stmt, stmt) → (simple_stmt, simple_stmt) → (small_stmt, small_stmt) → (global_stmt, nonlocal_stmt) ; syntactically, I'm inclided to agree that the code sample seems valid. (It is, of course, semantically malformed.) I suspect that there just never was a separate exception type for this sort of thing.
dragonwriter 7y ago
> I'm curious why this is reported as a SyntaxError. "Names listed in a global statement must not be used in the same code block textually preceding that global statement"; the nonlocal declaration is a use, and so that syntax rule is violated. https://docs.python.org/3/reference/simple_stmts.html#grammar-token-global-stmt https://docs.python.org/3/reference/simple_stmts.html#gramma...
- lilyball 7y ago
  If you can construct an AST, you’ve moved past the syntax stage.
  - dbcurtis 7y ago
    Well, in a pure sense, you are correct. But for practical reasons, it can sometimes be easier to define a parsing grammar that is a little loose, and pick up the stragglers by analyzing the AST. Sometimes the “correct” grammar is needlessly complex and you can reduce the headaches with a quick sanitation pass over the results of a less pure parser.
    - lilyball 7y ago
      Even so, I wouldn't consider this a syntax error, because the error isn't with the syntax but with using an identifier in a context where that particular identifier isn't allowed (but others are). This would make it more akin to a NameError.
lidHanteyk 7y ago
Python's grammar is so very rich. Many programs are rejected for "syntactic" reasons if they do not fit into the abstract syntax [0]; I suppose that this could be some sort of "AbstractSyntaxError", if Python had such a thing. For some examples: x == not y Rejected by the parser itself. not x = y Accepted by the parser and AST builder, and only rejected by the bytecode compiler. f(*xs,) Rejected by the parser itself. ... Rejected somewhere between the parser and the AST builder. Valid uses of Ellipsis are actually special-cased; it is not a standard part of the abstract syntax. [0] https://github.com/python/cpython/blob/master/Parser/Python.asdl https://github.com/python/cpython/blob/master/Parser/Python....
- kccqzy 7y ago
  Why is x == not x rejected by the parser? In many other languages this is perfectly valid and, depending on the semantics, may allow the compiler that into false. (Of course this can't generally be the case where operators are user-overloadable.)
  - js2 7y ago
    Because that's how the grammar is defined: https://docs.python.org/3.8/reference/grammar.html https://docs.python.org/3.8/reference/grammar.html The RHS of the == (comp_op) has to be an expr. You can force "not x" to be an expr by parenthesizing it: x == (not x)
    - kccqzy 7y ago
      That interesting. On the other hand, not x == x is totally valid.
  - lidHanteyk 7y ago
    I don't know man, I didn't do it.
chirau 7y ago
The nonlocal declaration is considered usage if you use a global statement in the same textual code block preceding the global itself. That is what makes it a syntax error.

bratao 7y ago

I was impressed by the quality of PyPy errors!

mherdeg 7y ago

I'm impressed by the humility here from someone who (a) is writing a library to replace core traceback behavior in pypi; (b) has doubtless read both cpython and pypi code; (c) nevertheless writes "While I have some general idea of how the CPython interpreter works, I absolutely do not understand well enough to claim with absolute certainty how this situation arise". Jeepers, who WOULD be qualified to explain for sure how this works?

jedberg 7y ago
> Jeepers, who WOULD be qualified to explain for sure how this works? Guido?

_ZeD_ 7y ago

I'm wondering if this case will be handled differently by the new pegen https://github.com/gvanrossum/pegen https://github.com/gvanrossum/pegen

underdeserver 7y ago

If 99.999% of Python programmers won't care about this, and there are approximately 8.2M Python programmers in the world [1], then only 82 people will care about this. Given the number of upvotes you've covered them all. [1] https://www.google.com/search?q=how+many+python+programmers+are+there+in+the+world https://www.google.com/search?q=how+many+python+programmers+...

aroberge 7y ago
Thank you. I actually thought I was overestimated the interest. Then again, I did not think that my blog post would attract the interest of the HackerNews readers.

Wheaties466 7y ago

I seem like i'm alone in this but I think I would prefer the CPython handling of this "Syntax Error". I am not a "Programmer" so having feedback on the original problem line would be more beneficial than returning the second.