3 ms·

I'm not the GP, but personally I'd consider syntax highlighting perfectly solved if it was handled in the language server. That way we could have highlighting p

I'm not the GP, but personally I'd consider syntax highlighting perfectly solved if it was handled in the language server. That way we could have highlighting performed by 100% accurate parsers instead of approximations. It wouldn't be as fast as in-editor simple tree-sitter highlighting, but I don't see any issue with my text appearing uncolored at first and gaining color as my editor polls the language server as I type. Currently language servers support semantic highlighting which is close, but as far as I can tell, it seems meant to be supplemental to in-editor local highlighting rather than a full replacement.

pawelmurias 3y ago
You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code
- craftkiller 3y ago
  That's sort-of already the case whenever we start a string in any language that supports multi-line strings. You type the first " and then the rest of your file gets re-interpreted as a string. It is annoying but I still think its a worthy trade-off, because I spend a lot more time reading code than being mid-edit with invalid syntax.
  - troupo 3y ago
    Syntax errors are not limited to just typing a quote
    - craftkiller 3y ago
      I never said they were. I'm just pointing out that we already experience the same problem in the case of quotes without the world ending, and I don't think expanding the problem to all syntax errors will make it much worse.
      - troupo 3y ago
        It would make it significantly much worse. Because every incomplete statement that you're just writing would break all syntax highlighting right after your cursor. And if you use any modern editor or IDE this doesn't happen.
- _flux 3y ago
  If the code was valid at some point (e.g. when opening it) it could remember the old colorings of the rest of the code. Though handling opening and closing a string and still ending up in an illegal state would need some more logic.. Maybe not worth it.
  - morelisp 3y ago
    > If the code was valid at some point (e.g. when opening it) it could remember the old colorings of the rest of the code. If you continue down this line of thinking a few more steps, eventually you produce tree-sitter.
- lelanthran 3y ago
  > You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code Sounds like a feature - don't need to scan for tiny squiggly red lines, or examine some tiny font output in a status bar somewhere. I'd love it.
- G3rn0ti 3y ago
  > You would only get the top half of your code highlighted because a compiler Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception. Almost all LSP severs don't actually use the compiler backend of the language they are servicing. All the LSP server implementations I've seen at least just use a static parsing approach (or even worse i.e. just a tokenizer) similar to what Treesitter does. It is just not limited to a single file. That's why I still think Treesitter has the potential to not only improve syntax highlighting but also to simplify language servers or even -- in the long run with some extensions of the the language modes -- replace it altogether.
  - gpderetta 3y ago
    But some do. Most notably clangd. Still clangd tries parse broken code using recovery heuristics (which can still be useful in a compiler to produce decent error messages).
  - morelisp 3y ago
    > Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception. But it is a misconception which framed the discussion, not a misconception of the answerer. The claim was that LSP was > performed by 100% accurate parsers instead of approximations
morelisp 3y ago
But you want syntax highlighting performed by approximations, because when editing your code is only approximately valid.
- globular-toast 3y ago
  It would be even better if the code was always valid because we only edit the AST itself, like what paredit does for Lisp modes.
  - morelisp 3y ago
    No. (The code being valid s-sexps doesn’t even guarantee it’s syntactically valid after macro expansion, but even if it does this is a bad idea.)
reuben364 3y ago
One interesting instance of semantic highlighting occurs in the Lean 4 language. It has extensible syntax whose parsers can be expressed in the language itself. Since parsing and evaluation are intertwined, highlighting can only really be effectively achieved in an LSP lest you re-implement the entirety of Lean.

pawelmurias 3y ago

You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code

craftkiller 3y ago
That's sort-of already the case whenever we start a string in any language that supports multi-line strings. You type the first " and then the rest of your file gets re-interpreted as a string. It is annoying but I still think its a worthy trade-off, because I spend a lot more time reading code than being mid-edit with invalid syntax.
- troupo 3y ago
  Syntax errors are not limited to just typing a quote
  - craftkiller 3y ago
    I never said they were. I'm just pointing out that we already experience the same problem in the case of quotes without the world ending, and I don't think expanding the problem to all syntax errors will make it much worse.
    - troupo 3y ago
      It would make it significantly much worse. Because every incomplete statement that you're just writing would break all syntax highlighting right after your cursor. And if you use any modern editor or IDE this doesn't happen.
_flux 3y ago
If the code was valid at some point (e.g. when opening it) it could remember the old colorings of the rest of the code. Though handling opening and closing a string and still ending up in an illegal state would need some more logic.. Maybe not worth it.
- morelisp 3y ago
  > If the code was valid at some point (e.g. when opening it) it could remember the old colorings of the rest of the code. If you continue down this line of thinking a few more steps, eventually you produce tree-sitter.
lelanthran 3y ago
> You would only get the top half of your code highlighted because a compiler doesnt usually continue when encountering syntax errors due to partial code Sounds like a feature - don't need to scan for tiny squiggly red lines, or examine some tiny font output in a status bar somewhere. I'd love it.
G3rn0ti 3y ago
> You would only get the top half of your code highlighted because a compiler Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception. Almost all LSP severs don't actually use the compiler backend of the language they are servicing. All the LSP server implementations I've seen at least just use a static parsing approach (or even worse i.e. just a tokenizer) similar to what Treesitter does. It is just not limited to a single file. That's why I still think Treesitter has the potential to not only improve syntax highlighting but also to simplify language servers or even -- in the long run with some extensions of the the language modes -- replace it altogether.
- gpderetta 3y ago
  But some do. Most notably clangd. Still clangd tries parse broken code using recovery heuristics (which can still be useful in a compiler to produce decent error messages).
- morelisp 3y ago
  > Your comment implies a language server taps into the language's compiler/interpreter. That is a popular misconception. But it is a misconception which framed the discussion, not a misconception of the answerer. The claim was that LSP was > performed by 100% accurate parsers instead of approximations

morelisp 3y ago

But you want syntax highlighting performed by approximations, because when editing your code is only approximately valid.

globular-toast 3y ago
It would be even better if the code was always valid because we only edit the AST itself, like what paredit does for Lisp modes.
- morelisp 3y ago
  No. (The code being valid s-sexps doesn’t even guarantee it’s syntactically valid after macro expansion, but even if it does this is a bad idea.)

reuben364 3y ago

One interesting instance of semantic highlighting occurs in the Lean 4 language. It has extensible syntax whose parsers can be expressed in the language itself. Since parsing and evaluation are intertwined, highlighting can only really be effectively achieved in an LSP lest you re-implement the entirety of Lean.