4 ms·

Go(lang)'s rejection makes sense. A format function that arbitrarily executes code from within a format string sounds like a complete nightmare. Log4j as an e

Go(lang)'s rejection makes sense. A format function that arbitrarily executes code from within a format string sounds like a complete nightmare. Log4j as an example. The rejection's example shows how that arbitrary code within the string could instead be fixed functions outside of a string. Safer, easier for compilers and programmers; unless an 'eval' for strings is what was desired. (Offhand I've only seen eval in /scripted/ languages; go makes binaries.)

NoTeslaThrow 2y ago
> A format function that arbitrarily executes code from within a format string So, a template? I certainly ain't gonna be using go for its mustache support.
miki123211 2y ago
In many languages, f-strings (or f-string like constructs) are only supported for string literals, not user-supplied strings. When compiling, those can be lowered to simple string concatenation, just like any for loop can be lowered to and represented as a while.
- NoTeslaThrow 2y ago
  What's the risk of user supplied strings? Surely you know their size. What else is there to worry about?
- mjevans 2y ago
  My reply was to the parent post's SPECIFIC example of Golang's rejected feature request. Please go read that proposal. It is NOT about the possibility of referencing existing / future (lazy / deferred evaluation) string literals from within the string, but about a format string that would literally evaluate arbitrary functions within a string.
  - unscaled 2y ago
    The proposal doesn't say anything about executing code in user-supplied strings. It only talks about a string literal that is processed by the compiler (at which point no user-supplied string can be available). On the other hand, the current solution offered by Go (fmt.Sprintf) is the one who supports a user-supplied format String. Admittedly, there is a limited amount of damage that could be done this well, but you can at the very least cause a program to panic. The reason for declining this feature[1] has nothing to do with what you stated. Ian Lance Taylor simply said: "This doesn't seem to have a big advantage over calling fmt.Sprintf" and "You can a similar effect using fmt.Sprint". He conceded that there are performance advantages to string interpolation, but he doesn't believe there are any gains in usability over fmt.Sprintf/fmt.Sprint and as is usual with Go (compared to other languages), they're loathe to add new features to the compiler[2]. [1] https://github.com/golang/go/issues/34174#issuecomment-1450932232 https://github.com/golang/go/issues/34174#issuecomment-14509... [2] https://github.com/golang/go/issues/34174#issuecomment-530131536 https://github.com/golang/go/issues/34174#issuecomment-53013...
- zahlman 2y ago
  In case there was confusion: Python's f-string functionality in particular is specific to string literals. The f prefix doesn't create a different data type; instead, the contents of the literal are parsed at compile time and the entire thing is rewritten into equivalent string concatenation code (although IIRC it uses dedicated bytecodes, in at least some versions). The t-string proposal involves using new data types to abstract the concatenation and formatting process, but it's still a compile-time process - and the parts between the braces still involve code that executes first - and there's still no separate type for the overall t-string literal, and no way to end up eval'ing code from user-supplied data except by explicitly requesting to do so.
  - the_clarence 2y ago
    There is no compile time in python
    - pansa2 2y ago
      Yes there is, when it compiles source code to bytecode.
    - zahlman 2y ago
      Yes, there is. Python source code is translated into bytecode for a VM just like in Java or C#, and by default it's cached in .pyc files. It's only different in that you can ask to execute a source code file and the compilation happens automatically before the bytecode-interpretation. `SyntaxError` is fundamentally different from other exceptions because it can occur during compilation, and only occurs at run-time if explicitly raised (or via explicit invocation of another code compilation, such as with `exec`/`eval`, or importing a module). This is also why you can't catch a `SyntaxError` caused by the invalid syntax of your own code, but only from such an explicit `raise` or a request to compile a source code string (see https://stackoverflow.com/questions/1856408 https://stackoverflow.com/questions/1856408 ).
paulddraper 2y ago
No, the format function doesn't "arbitrarily execute code." An f/t string is syntax not runtime. Instead of "Hello " + subject + "!" you write f"Hello {subject}!" That subject is simple an normal code expression, but one that occurs after the opening quote of the literal and before the ending quote of the literal. And instead of query(["SELECT * FROM account WHERE id = ", " AND active"], [id]) you write query(t"SELECT * FROM account WHERE id = {id} AND active") It's a way of writing string literals that if anything makes injection less likely.
- mjevans 2y ago
  Please read the context of my reply again. The Rejected Golang proposal cited by the post I'm replying to. NOT Python's present PEP or any other string that might resolve magic variables (just not literally eval / exec functions!).
  - zahlman 2y ago
    As far as I can tell from the linked proposal, it wouldn't have involved such evaluation either. It seems like it was intended to work fundamentally the same way as it currently does in Python: by analyzing the string literal ahead of time and translating into equivalent explicit formatting code, as syntactic sugar. There seem to have been many misunderstandings in the GitHub discussion.
    - mjevans 2y ago
      In that case, I might have misunderstood the intent of those examples. However the difficulty of understanding also illustrates the increased maintenance burden and language complexity.
      - eviks 2y ago
        Unless workarounds to a missing feature have a higher maintenance burden like in this case, and you can't avoid it via learning
bcoates 2y ago
No, it's exactly the opposite--f-strings are, roughly, eval (that is, unsanitary string concatenation that is presumptively an error in any nontrivial use) to t-strings which are just an alternative expression syntax, and do not even dereference their arguments.
- rowanG077 2y ago
  f-strings are not eval. It's not dynamic. It's simply an expression that is ran just like every other expression.
  - bcoates 2y ago
    Right, and then if you do literally anything with the output other than print() to a tty, it’s an escaping/injection attack. any_func(f"{attacker_provided}") <=> eval(attacker_provided), from a security/correctness perspective
    - saagarjha 2y ago
      How is this any different from any_func(attacker_provided)
    - rowanG077 1y ago
      Shooting any unsanitized input into your application is bad. template strings don't make this worse. any_func(attacker_provided) is even worse then any_func(t"{attacker_provided}") since in the later case you actually have reduced the attack surface to just strings.

NoTeslaThrow 2y ago

> A format function that arbitrarily executes code from within a format string So, a template? I certainly ain't gonna be using go for its mustache support.

miki123211 2y ago

In many languages, f-strings (or f-string like constructs) are only supported for string literals, not user-supplied strings. When compiling, those can be lowered to simple string concatenation, just like any for loop can be lowered to and represented as a while.

NoTeslaThrow 2y ago
What's the risk of user supplied strings? Surely you know their size. What else is there to worry about?
mjevans 2y ago
My reply was to the parent post's SPECIFIC example of Golang's rejected feature request. Please go read that proposal. It is NOT about the possibility of referencing existing / future (lazy / deferred evaluation) string literals from within the string, but about a format string that would literally evaluate arbitrary functions within a string.
- unscaled 2y ago
  The proposal doesn't say anything about executing code in user-supplied strings. It only talks about a string literal that is processed by the compiler (at which point no user-supplied string can be available). On the other hand, the current solution offered by Go (fmt.Sprintf) is the one who supports a user-supplied format String. Admittedly, there is a limited amount of damage that could be done this well, but you can at the very least cause a program to panic. The reason for declining this feature[1] has nothing to do with what you stated. Ian Lance Taylor simply said: "This doesn't seem to have a big advantage over calling fmt.Sprintf" and "You can a similar effect using fmt.Sprint". He conceded that there are performance advantages to string interpolation, but he doesn't believe there are any gains in usability over fmt.Sprintf/fmt.Sprint and as is usual with Go (compared to other languages), they're loathe to add new features to the compiler[2]. [1] https://github.com/golang/go/issues/34174#issuecomment-1450932232 https://github.com/golang/go/issues/34174#issuecomment-14509... [2] https://github.com/golang/go/issues/34174#issuecomment-530131536 https://github.com/golang/go/issues/34174#issuecomment-53013...
zahlman 2y ago
In case there was confusion: Python's f-string functionality in particular is specific to string literals. The f prefix doesn't create a different data type; instead, the contents of the literal are parsed at compile time and the entire thing is rewritten into equivalent string concatenation code (although IIRC it uses dedicated bytecodes, in at least some versions). The t-string proposal involves using new data types to abstract the concatenation and formatting process, but it's still a compile-time process - and the parts between the braces still involve code that executes first - and there's still no separate type for the overall t-string literal, and no way to end up eval'ing code from user-supplied data except by explicitly requesting to do so.
- the_clarence 2y ago
  There is no compile time in python
  - pansa2 2y ago
    Yes there is, when it compiles source code to bytecode.
  - zahlman 2y ago
    Yes, there is. Python source code is translated into bytecode for a VM just like in Java or C#, and by default it's cached in .pyc files. It's only different in that you can ask to execute a source code file and the compilation happens automatically before the bytecode-interpretation. `SyntaxError` is fundamentally different from other exceptions because it can occur during compilation, and only occurs at run-time if explicitly raised (or via explicit invocation of another code compilation, such as with `exec`/`eval`, or importing a module). This is also why you can't catch a `SyntaxError` caused by the invalid syntax of your own code, but only from such an explicit `raise` or a request to compile a source code string (see https://stackoverflow.com/questions/1856408 https://stackoverflow.com/questions/1856408 ).

paulddraper 2y ago

No, the format function doesn't "arbitrarily execute code." An f/t string is syntax not runtime. Instead of "Hello " + subject + "!" you write f"Hello {subject}!" That subject is simple an normal code expression, but one that occurs after the opening quote of the literal and before the ending quote of the literal. And instead of query(["SELECT * FROM account WHERE id = ", " AND active"], [id]) you write query(t"SELECT * FROM account WHERE id = {id} AND active") It's a way of writing string literals that if anything makes injection less likely.

mjevans 2y ago
Please read the context of my reply again. The Rejected Golang proposal cited by the post I'm replying to. NOT Python's present PEP or any other string that might resolve magic variables (just not literally eval / exec functions!).
- zahlman 2y ago
  As far as I can tell from the linked proposal, it wouldn't have involved such evaluation either. It seems like it was intended to work fundamentally the same way as it currently does in Python: by analyzing the string literal ahead of time and translating into equivalent explicit formatting code, as syntactic sugar. There seem to have been many misunderstandings in the GitHub discussion.
  - mjevans 2y ago
    In that case, I might have misunderstood the intent of those examples. However the difficulty of understanding also illustrates the increased maintenance burden and language complexity.
    - eviks 2y ago
      Unless workarounds to a missing feature have a higher maintenance burden like in this case, and you can't avoid it via learning

bcoates 2y ago

No, it's exactly the opposite--f-strings are, roughly, eval (that is, unsanitary string concatenation that is presumptively an error in any nontrivial use) to t-strings which are just an alternative expression syntax, and do not even dereference their arguments.

rowanG077 2y ago
f-strings are not eval. It's not dynamic. It's simply an expression that is ran just like every other expression.
- bcoates 2y ago
  Right, and then if you do literally anything with the output other than print() to a tty, it’s an escaping/injection attack. any_func(f"{attacker_provided}") <=> eval(attacker_provided), from a security/correctness perspective
  - saagarjha 2y ago
    How is this any different from any_func(attacker_provided)
  - rowanG077 1y ago
    Shooting any unsanitized input into your application is bad. template strings don't make this worse. any_func(attacker_provided) is even worse then any_func(t"{attacker_provided}") since in the later case you actually have reduced the attack surface to just strings.