4 ms·
The author may have missed that lexing C is actually context-sensitive, i.e. you need a symbol table: https://en.wikipedia.org/wiki/Lexer_hack https://en.wikipe
by layer8 2y ago
The author may have missed that lexing C is actually context-sensitive, i.e. you need a symbol table: https://en.wikipedia.org/wiki/Lexer_hack https://en.wikipedia.org/wiki/Lexer_hack
Of course, for syntax highlighting this is only relevant if you want to highlight the multiplication operator differently from the dereferencing operator, or declarations differently from expressions.
More generally, however, I find it useful to highlight (say) types differently from variables or functions, which in some (most?) popular languages requires full parsing and symbol table information. Some IDEs therefore implement two levels of syntax highlighting, a basic one that only requires lexical information, and an extended one that kicks in when full grammar and type information becomes available.
- legobmw99 2y agoI’d be shocked if jart didn’t know this, but it seems unlikely that an LLM would generate one of these most vexing parses, unless explicitly asked
- layer8 2y agoGiven all the things that were new to the author in the article, I wouldn’t be shocked at all. There’s just a huge number of things to know, or to have come across.
- jraph 2y agoJustine is proficient in C, she is the author of a libc (cosmopolitan) among other things, like Actually Portable Executables [1]. I would expect her to know C quite well, and that's probably an understatement. [1] https://justine.lol/ape.html https://justine.lol/ape.html
- quietbritishjim 2y agoI think you're thinking of something different to the issue in the parent comment. The most vexing parse is, as the name suggests, a problem at the parsing stage rather than the earlier lexing phase. Unlike the referenced lexing problem, it does't require any hack for compilers to deal with it. That's because it's not really a problem for the compiler; it's humans that find it surprising.
- alekratz 2y agoI don't think the lexer hack is relevant in this instance. The lexer hack just refers to the ambiguity of `A * B` and whether that should be parsed as a variable declaration or an expression. If you're building a syntax tree, then this matters, but AFAICT all the author needs is a sequence of tokens and not a syntax tree. Maybe "parser hack" would be a better name for it.
- teo_zero 2y ago> this is only relevant if you want to highlight the multiplication operator differently from the dereferencing operator Can you mention one editor which does that?
- quietbritishjim 2y agoI don't think they implied there is. The sentence you quoted is essentially "this is relevant for their article about weird lexical syntax, but (almost definitely) not relevant to their original problem of syntax highlighting".
- mdaniel 2y agoI could be stretching the definition of "does" but the newfound(?) tree-sitter support in Emacs[1] I believe would allow that since it for sure understands the distinction but I don't possess enough font-lock ninjary to actually, for real, bind a different color to the distinct usages /* given foo.c */ int main() { int a, *b; a = 5 * 10; b = &a; printf("a is %d\n", *b); } and then M-x c-ts-mode followed by navigating to each * and invoking M-x treesit-inspect-node-at-point in turn produces, respectively: (declaration declarator: (pointer_declarator "*")) right: (binary_expression operator: "*") arguments: (argument_list (pointer_expression operator: "*")) 1: https://www.emacswiki.org/emacs/Tree-sitter https://www.emacswiki.org/emacs/Tree-sitter
- teo_zero 2y agoThese examples are unambiguous. Try with something more spicy like return (A)*(B); which depends on A being a type or a variable.
- dummy7777 2y agohey