7 ms·
Interpreted as: [0xf or x in (1,2,3)] 0xf = Hex for 15 Expression is evaluated as a boolean
by adpirz 6y ago
Interpreted as:
[0xf or x in (1,2,3)]
0xf = Hex for 15
Expression is evaluated as a boolean
- kristaps 6y agoWhy is it parsed like that and not as invalid integer literal "0xfor"? edit: 'cause it's a bug! https://bugs.python.org/issue43833 https://bugs.python.org/issue43833
- pdw 6y agoFor pretty much the same reason that "1+2" is not parsed as an invalid integer.
- _kst_ 6y ago"1+2" is a valid expression consisting of the three tokens "1", "+", and "2". It's permitted to omit whitespace between those tokens. The question is whether "0xfor" is a valid way to represent the two tokens "0xf", "or". My reading of the rules is that it isn't, and that whitespace is required.
- _kst_ 6y agoI read the rules incorrectly. It appears that "0xfor" is a valid way to represent the two tokens "0xf" and "or". The rule, which I initially misinterpreted, is: > Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token (e.g., ab is one token, but a b is two tokens). Since "0xfor" cannot be interpreted as a different (single) token, whitespace is not needed. (I'm not a fan of this particular consequence of that rule, and I suspect it was not intended, but it says what it says.)
- mjs7231 6y agoShouldn't this be considered a bug in Python? Why does it even try to evaluate 0xfor without the space? Trying a few other things.. * 0xfor1 evaluates. * 1or 2 evaluates. * 1or2 doesn't. * ''or'foo' evaluates. This is gross.
- kristaps 6y agoNot by design, fortunately: https://bugs.python.org/issue43833 https://bugs.python.org/issue43833
- xxpor 6y agoThat hasn't been confirmed. Are we sure that it's not an inherent ambiguity in the grammar?
- joshuamorton 6y agoThat's not totally clear. A bug being filed doesn't mean it's accepted. And this has been (ab)used for quite some time in various python codegolf. See https://codegolf.stackexchange.com/a/56 https://codegolf.stackexchange.com/a/56 from 2011.
- Sohcahtoa82 6y agoIt is 100% by design. It's even documented. https://docs.python.org/3/reference/lexical_analysis.html#whitespace-between-tokens https://docs.python.org/3/reference/lexical_analysis.html#wh... This is not a bug.
- _kst_ 6y agoThe cited documentation says: > Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token (e.g., ab is one token, but a b is two tokens). The two tokens in this case are "0xf" and "or". Their concatenation cannot be interpreted as a different token, because "0xfor" is not a valid token. Therefore, if I'm reading the rule correctly, whitespace is needed in this case. "0xffor" is another interesting case. It's also not a valid token, but it could be interpreted as two tokens in two different ways: "0xf" "for" or "0xff" "or". (Python does the latter. I presume it uses something like C's "maximal munch" rule.)
- Sohcahtoa82 6y agoYour conclusion is the exact opposite of what the documentation explicitly states. > Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token (e.g., ab is one token, but a b is two tokens). Because the concatenation of "0xf" and "or" can't be interpreted as a different token, the whitespace is not needed.
- deleted 6y ago[deleted]
- TrackerFF 6y agoAnyone know why it's legal to use logic operators like that in Python? That is, 0or, 0xor, 0and,etc. in my view, this should clearly be a syntax error. edit: as noted by the parent comment, 0xor != 0 xor, but 0x or. But this seems to hold true for all data types.
- deleted 6y ago[deleted]
- mywittyname 6y agoOther languages don't enforce whitespace around logical operators. Like `0xf||1+2` is perfectly valid javascript. I can see how this could slip by, it's possible that original python used a non-alphabetical token as the logical operator, then it was swapped out at some later time for or and and.
- wyldfire 6y agoHere's the AST: >>> ast.dump(ast.parse('[0xfor x in (1,2,3)]')) "Module(body=[Expr(value=List(elts=[BoolOp(op=Or(), values=[Num(n=15), Compare(left=Name(id='x', ctx=Load()), ops=[In()], comparators=[Tuple(elts=[Num(n=1), Num(n=2), Num(n=3)], ctx=Load())])])], ctx=Load()))])"
- tgv 6y agoA bit of a miss for a language that puts whitespace central in the syntax.
- goto11 6y ago"2+2" and "2 + 2" is the same in Python, so whitespace is not significant between otherwise unambigous tokens in Python. Whitespace in Python is significant to indicate statements and block.
- commandlinefan 6y agoI initially assumed the missing space was a typo in the headline, so I tried [0x for x in (1,2,3)] and got "invalid hexadecimal literal" as I would have expected. Took me a few minutes to realize the typo was intentional. I still can't figure out what the author was trying to do when he stumbled onto that, though...
- tsimionescu 6y agoMy guess is that they were trying to do soemthing like [x for x in some_list], but had some unfortunate typos.
- commandlinefan 6y agoYeah maybe he was converting a tuple to a list, but that still seems like a weird way to do it.
- Dylan16807 6y agoThink of it as a simplified version to demonstrate the issue.
- wyldfire 6y agoThis is the kind of thing you could end up with from an automated test case reducer. Start with a typo that causes odd behavior (which might be interpreted as a CPython 'bug') and use something like creduce to whittle it down to the minimal case.
- nickysielicki 6y agoJust to elaborate, it short circuits, so x need not be defined. [sielicki@dogfruit ~]$ python3.10 -c 'print(0xf or some_undefined_variable in some_undefined_set)' 15
- wyldfire 6y agoThis is the kinda thing that helps illustrate just how powerfully dynamic Python is. Almost nothing is statically bound. Super cool if you want to write code that inspects some dynamic state and has special behavior. But the cost is that there's very little static checking capability. This code "should" result in a NameError or something similar. But resolving that identifier to a name won't happen until we decide to evaluate the right-hand side of the `or`. Thankfully, most python static checkers make some mostly-sane assumptions and will flag this code as an error, despite the subtle possibility of legally creating this name using some other Python/CPython features. Type annotations are a great idea but in practice I don't see them used much. Should be great for dev teams working on bigger Python projects.
- layer8 6y agoThis illustrates why lexer tokens should preferably not be defined as exactly what the language allows. Instead the internal lexer definition should include invalid tokens (like “0xfor”) that are only sorted out in a later step (in this case, when actually converting it to an integer value).
- fanf2 6y agoYes, that is basically how C works, though the C standard describes it in a weird way. There is the notion of a “preprocessor number” which starts with a digit and allows any characters that can appear in a number or identifier. At a later step in parsing it is converted to a strict number, which would fail in a case like 0xfor.
- brundolf 6y agoThere's no ambiguity though, right? [0x for x in (1,2,3)] is nonsense (one typo off from something legitimate, but nonsense regardless). I can't think of another way someone might expect this to parse.
- Skunkleton 6y agoI would have expected the english boolean operators to require whitespace to parse correctly, personally. "0xf||x" makes sense. "0xforx" makes no sense.
- Sohcahtoa82 6y ago"0xforx" would be parsed as "0xf orx" which would be a syntax error.
- Skunkleton 6y agoYeah, I realized after my edit window had expired. Should have been "0xf|| x" and "0xfor x".
- moefh 6y agoIf you want confusion with a valid expression: [0x1for x in (1,2,3)] looks like it should parse as [0x1 for x in (1,2,3)] but instead parses as [0x1f or x in (1,2,3)]
- nickysielicki 6y agoThis is better than the original post.