3 ms·

> actual code being embedded inside strings My view on this is that it shouldn’t be interpreted as code being embedded inside strings, but as a special form of

by layer8 2y ago

> actual code being embedded inside strings My view on this is that it shouldn’t be interpreted as code being embedded inside strings, but as a special form of string concatenation syntax. In turn, this would mean that you can nest the syntax, for example: "foo { toUpper("bar { x + y } bar") } foo" The individual tokens being (one per line): "foo { toUpper ( "bar { x + y } bar" ) } foo" If `+` does string concatenation, the above would effectively be equivalent to: "foo " + toUpper("bar " + (x + y) + " bar") + " foo" I don’t know if there is a language that actually works that way.

panzi 2y ago
Indeed in some of the listed languages you can nest it like that, but in others (e.g. Python) you can't. I would guess they deliberately don't want to enable that and it's not a problem in their parser or something.
- layer8 2y ago
  Even when nesting is disallowed, my point is that I find it preferable to not view it (and syntax-highlight it) as a “special string” with embedded magic, but as multiple string literals with just different delimiters that allow omitting the explicit concatenation operator, and normal expressions interspersed in between. I think it’s important to realize that it is really just very simple syntactic sugar for normal string concatenation.
  - Timwi 2y ago
    While you're conceptually right, in practice I think it bears mentioning that in C# the two syntaxes compile differently. This is because C#’s target platform, the .NET Framework, has always had a function called `string.Format` that lets you write this: var str = string.Format("{0} is {1} years old.", name, age); When interpolated strings were introduced later, it was natural to have them compile to this instead of concatenation.
    - layer8 2y ago
      There's no reason in principle why name + " is " + age + " years old." couldn't compile to exactly the same. (Other than maybe `string.Format` having some additional customizable behavior, I don't know C# that well.)
      - epcoa 2y ago
        Like python, and Rust with the format! macro (which doesn't even support arbitrary expressions), C# the full syntax for interpolated/formatted strings is this: {<interpolationExpression>[,<alignment>][:<formatString>]}, ie there is more going on then just a simple wrapper around concat or StringBuilder.
      - ygra 2y ago
        When not using the format specifiers or alignment it will indeed compile to just string.Concat (which is also what the + operator for strings compiles to). Similar to C compilers choosing to call pits instead of printf if there is nothing to be formatted.
  - epcoa 2y ago
    If it’s treated strictly as simple concatenation syntactic sugar then you are allowing something like print(“foo { func() ); Which seems janky af. > just very simple syntactic sugar for normal string concatenation. Maybe. There’s also possibly a string conversion. It seems reasonable to want to disallow implicit string conversion in a concatenation operator context (especially if overloading +) while allowing it in the interpolation case.
    - layer8 2y ago
      I failed to mention the balancing requirement, that should of course remain. But it's an artificial requirement, so to speak, that is merely there to double-check the programmer's intent. The compiler/parser wouldn't actually care (unlike for an arithmetic expression with unbalanced parentheses, or scope blocks with unbalanced braces), the condition is only checked for the programmer's benefit. > here’s also possibly a string conversion. It seems reasonable to want to disallow implicit string conversion in a concatenation operator context (especially if overloading +) while allowing it in the interpolation case. Many languages have a string contenation operator that does implicit conversion to string, while still having a string interpolation syntax like the above. It's kind of my point that both are much more similar to each other than many people seem to realize.
- Tarean 2y ago
  As of python 3.6 you can nest fstrings. Not all formatters and highlighters have caught up, though. Which is fun, because correct highlighting depends on language version. Haskell has similar problems where different compiler flags require different parsers. Close enough is sufficient for syntax highlighting, though. Python is also a bit weird because it calls the format methods, so objects can intercept and react to the format specifiers in the f-string while being formatted.
  - panzi 2y ago
    I didn't mean nested f-strings. I mean this is a syntax error: >>> print(f"foo {"bar"}") SyntaxError: f-string: expecting '}' Only this works: >>> print(f"foo {'bar'}") foo bar
    - pdw 2y ago
      You're using an old Python version. On recent versions, it's perfectly fine: Python 3.12.7 (main, Oct 3 2024, 15:15:22) [GCC 14.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print(f"foo {"bar"}") foo bar
epcoa 2y ago
> "foo { … That should probably not be one token. > My view on this is that it shouldn’t be interpreted as code being embedded inside strings I’m not sure exactly what you’re proposing and how it is different. You still can’t parse it as a regular lexical grammar. How does this change how you highlight either? Whatever you call it, to the lexer it is a special string, it has to know how to match it, the delimiters are materially different than concatenation. I might be being dense but I’m not sure what’s formally distinct.
- layer8 2y ago
  > > "foo { … > That should probably not be one token. It's exactly the point that this is one token. It's a string literal with opening delimiter `"` and closing delimiter `{`, and that whole token itself serves as a kind of opening "brace". Alternatively, you can see `{` as a contraction of `" +`. Meaning, aside from the brace balancing requirement, `"foo {` does the same a `"foo " +` would. Still alternatively, you could imagine a language that concatenates around string literals by default, similar to how C behaves for sequences of string literals. In C, "foo" "bar" "baz" is equivalent to "foobarbaz" Similarly, you could imagine a language where "foo" some_variable "bar" would perform implicit concatenation, without needing an explicit operator (as in `"foo" + x + "bar"`). And then people might write it without the inner whitespace, as: "foo"some_variable"bar" My point is that "foo{some_variable}bar" is really just that (plus a condition requiring balanced pairs of braces). You can also re-insert the spaces for emphasis: "foo{ some_variable }bar" The fact that people tend to think of `{some_variable}` as an entity is sort-of an illusion. > How does this change how you highlight either? You would highlight the `"...{`, `}...{`, and `}..."` parts like normal string literals (they just use curly braces instead of double quotes at one or both ends), and highlight the inner expressions the same as if they weren't surrounded by such literals.
  - epcoa 2y ago
    > It's exactly the point that this is one token. Fair enough. The point, as you have acknowledged, being that unlike + you have to treat { specially for balancing (and separately from the “). > The fact that people tend to think of `{some_variable}` as an entity is sort-of an illusion. I guess. I just don’t know what being an illusion means formally. It’s not an illusion to the person that has to implement the state machine that balances the delimiters. > You would highlight the `"...{`, `}...{`, and `}..."` parts like normal string literals (they just use curly braces instead of double quotes at one or both ends), and highlight the inner expressions the same as if they weren't surrounded by such literals Emacs does it this way FWIW. But I’m not sure how important it is to dictate that the brace can’t be a different color. In any event, I can agree your design is valid (Kotlin works this way), but I don’t necessarily agree it is any more valid than say how Python does it where there can format specifiers, implicit conversion to string is performed whereas not with concatenation. I’m not seeing the clear definitive advantage of interpolated strings being an equivalent to concatenation vs some other type of method call. The other detail is order of evaluation or sequencing. String concat may behave differently. Not sure I agree it is wrong, because at the end of the day it is distinct looking syntax. Illusion or not, it looks like a neatly enclosed expression, and concatenation looks like something else. That they might parse, evaluate or behave different isn't unreasonable.

by layer8 2y ago

panzi 2y ago

Indeed in some of the listed languages you can nest it like that, but in others (e.g. Python) you can't. I would guess they deliberately don't want to enable that and it's not a problem in their parser or something.

layer8 2y ago
Even when nesting is disallowed, my point is that I find it preferable to not view it (and syntax-highlight it) as a “special string” with embedded magic, but as multiple string literals with just different delimiters that allow omitting the explicit concatenation operator, and normal expressions interspersed in between. I think it’s important to realize that it is really just very simple syntactic sugar for normal string concatenation.
- Timwi 2y ago
  While you're conceptually right, in practice I think it bears mentioning that in C# the two syntaxes compile differently. This is because C#’s target platform, the .NET Framework, has always had a function called `string.Format` that lets you write this: var str = string.Format("{0} is {1} years old.", name, age); When interpolated strings were introduced later, it was natural to have them compile to this instead of concatenation.
  - layer8 2y ago
    There's no reason in principle why name + " is " + age + " years old." couldn't compile to exactly the same. (Other than maybe `string.Format` having some additional customizable behavior, I don't know C# that well.)
    - epcoa 2y ago
      Like python, and Rust with the format! macro (which doesn't even support arbitrary expressions), C# the full syntax for interpolated/formatted strings is this: {<interpolationExpression>[,<alignment>][:<formatString>]}, ie there is more going on then just a simple wrapper around concat or StringBuilder.
    - ygra 2y ago
      When not using the format specifiers or alignment it will indeed compile to just string.Concat (which is also what the + operator for strings compiles to). Similar to C compilers choosing to call pits instead of printf if there is nothing to be formatted.
- epcoa 2y ago
  If it’s treated strictly as simple concatenation syntactic sugar then you are allowing something like print(“foo { func() ); Which seems janky af. > just very simple syntactic sugar for normal string concatenation. Maybe. There’s also possibly a string conversion. It seems reasonable to want to disallow implicit string conversion in a concatenation operator context (especially if overloading +) while allowing it in the interpolation case.
  - layer8 2y ago
    I failed to mention the balancing requirement, that should of course remain. But it's an artificial requirement, so to speak, that is merely there to double-check the programmer's intent. The compiler/parser wouldn't actually care (unlike for an arithmetic expression with unbalanced parentheses, or scope blocks with unbalanced braces), the condition is only checked for the programmer's benefit. > here’s also possibly a string conversion. It seems reasonable to want to disallow implicit string conversion in a concatenation operator context (especially if overloading +) while allowing it in the interpolation case. Many languages have a string contenation operator that does implicit conversion to string, while still having a string interpolation syntax like the above. It's kind of my point that both are much more similar to each other than many people seem to realize.
Tarean 2y ago
As of python 3.6 you can nest fstrings. Not all formatters and highlighters have caught up, though. Which is fun, because correct highlighting depends on language version. Haskell has similar problems where different compiler flags require different parsers. Close enough is sufficient for syntax highlighting, though. Python is also a bit weird because it calls the format methods, so objects can intercept and react to the format specifiers in the f-string while being formatted.
- panzi 2y ago
  I didn't mean nested f-strings. I mean this is a syntax error: >>> print(f"foo {"bar"}") SyntaxError: f-string: expecting '}' Only this works: >>> print(f"foo {'bar'}") foo bar
  - pdw 2y ago
    You're using an old Python version. On recent versions, it's perfectly fine: Python 3.12.7 (main, Oct 3 2024, 15:15:22) [GCC 14.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print(f"foo {"bar"}") foo bar

epcoa 2y ago

> "foo { … That should probably not be one token. > My view on this is that it shouldn’t be interpreted as code being embedded inside strings I’m not sure exactly what you’re proposing and how it is different. You still can’t parse it as a regular lexical grammar. How does this change how you highlight either? Whatever you call it, to the lexer it is a special string, it has to know how to match it, the delimiters are materially different than concatenation. I might be being dense but I’m not sure what’s formally distinct.

layer8 2y ago
> > "foo { … > That should probably not be one token. It's exactly the point that this is one token. It's a string literal with opening delimiter `"` and closing delimiter `{`, and that whole token itself serves as a kind of opening "brace". Alternatively, you can see `{` as a contraction of `" +`. Meaning, aside from the brace balancing requirement, `"foo {` does the same a `"foo " +` would. Still alternatively, you could imagine a language that concatenates around string literals by default, similar to how C behaves for sequences of string literals. In C, "foo" "bar" "baz" is equivalent to "foobarbaz" Similarly, you could imagine a language where "foo" some_variable "bar" would perform implicit concatenation, without needing an explicit operator (as in `"foo" + x + "bar"`). And then people might write it without the inner whitespace, as: "foo"some_variable"bar" My point is that "foo{some_variable}bar" is really just that (plus a condition requiring balanced pairs of braces). You can also re-insert the spaces for emphasis: "foo{ some_variable }bar" The fact that people tend to think of `{some_variable}` as an entity is sort-of an illusion. > How does this change how you highlight either? You would highlight the `"...{`, `}...{`, and `}..."` parts like normal string literals (they just use curly braces instead of double quotes at one or both ends), and highlight the inner expressions the same as if they weren't surrounded by such literals.
- epcoa 2y ago
  > It's exactly the point that this is one token. Fair enough. The point, as you have acknowledged, being that unlike + you have to treat { specially for balancing (and separately from the “). > The fact that people tend to think of `{some_variable}` as an entity is sort-of an illusion. I guess. I just don’t know what being an illusion means formally. It’s not an illusion to the person that has to implement the state machine that balances the delimiters. > You would highlight the `"...{`, `}...{`, and `}..."` parts like normal string literals (they just use curly braces instead of double quotes at one or both ends), and highlight the inner expressions the same as if they weren't surrounded by such literals Emacs does it this way FWIW. But I’m not sure how important it is to dictate that the brace can’t be a different color. In any event, I can agree your design is valid (Kotlin works this way), but I don’t necessarily agree it is any more valid than say how Python does it where there can format specifiers, implicit conversion to string is performed whereas not with concatenation. I’m not seeing the clear definitive advantage of interpolated strings being an equivalent to concatenation vs some other type of method call. The other detail is order of evaluation or sequencing. String concat may behave differently. Not sure I agree it is wrong, because at the end of the day it is distinct looking syntax. Illusion or not, it looks like a neatly enclosed expression, and concatenation looks like something else. That they might parse, evaluate or behave different isn't unreasonable.