3 ms·

I am having some problems trying to understand the implementation. What would the AST from evaluating "f'{a+1}'" look like? Will there be a special AST node for

by dalke 11y ago

I am having some problems trying to understand the implementation. What would the AST from evaluating "f'{a+1}'" look like? Will there be a special AST node for f-strings, or will it be pre-structured into the AST? If it's a special node, is it the responsibility of the byte code generator to parse the string? My belief is that it's part of the parser's job, so the AST will never contain an f-string. What does a syntax error report look like? Or traceback? Will it be able to narrow down the part of the string which causes a problem? Can f-strings include f-strings, like: f"{a + (f' and {b+1}')}" I assume the answer is 'yes, and you shouldn't do that', which I can accept. Support for arbitrary expressions inside of an f-string means that the following is allowed, def f(a, b): return f"{yield a} = {b}" print(f(1, 2)) and will work, and will print something, but it won't be "1 = 2". Nor will any but heavy-weight analysis tools be able to figure out that this 'f' is a generator. I am less happy accepting that a magic string can turn a function into a generator. Take for example this code from around line 438 of https://searchcode.com/codesearch/view/18830026/ https://searchcode.com/codesearch/view/18830026/ : if attr=='yields' : yield_unit = self._grab_attr_(obj,'yield_unit') if yield_unit: ret = '%s %s'%(ret,yield_unit) # FIXME: i18n? return ret The penultimate line could be rewritten, validly, as: ret = f'{ret} {yield_unit}' # FIXME: i18n? The introduction of a typo, from 'yield_unit' to 'yield unit', would drastically change the function, and be very hard to spot. ret = f'{ret} {yield unit}' # FIXME: i18n? Yes, Don't Do That, but we know that people use things like syntax highlighters to help understand the code and identify mistakes like this. EDIT: the PEP says that the expression are "parsed with the equivalent of ast.parse(expression, '<fstring>', 'eval')". That means that 'yield' is not allowed.

TazeTSchnitzel 11y ago
Presumably it'd be handled similarly to how PHP handles {} and $ in strings. As soon as possible, you swap "{foo} {bar}" for (foo.__format__() + " " + bar.__format__())
- dalke 11y ago
  It's not so simple. The full implementation has to do something like insert an AST into the right place, because {foo} can be an expression like f"{__import__('math').cos(len(s))}". The Python tokenizer will pass an f-string to the AST builder, which has to pass the string to another tokenizer to generate the new AST. Because f-strings can contain f-strings, this process is recursive. The end result is a new AST that replaces the original f-string.

by dalke 11y ago

TazeTSchnitzel 11y ago

Presumably it'd be handled similarly to how PHP handles {} and $ in strings. As soon as possible, you swap "{foo} {bar}" for (foo.__format__() + " " + bar.__format__())

dalke 11y ago
It's not so simple. The full implementation has to do something like insert an AST into the right place, because {foo} can be an expression like f"{__import__('math').cos(len(s))}". The Python tokenizer will pass an f-string to the AST builder, which has to pass the string to another tokenizer to generate the new AST. Because f-strings can contain f-strings, this process is recursive. The end result is a new AST that replaces the original f-string.