3 ms·
Very cool. I suppose these days tree-sitter would be one way to support a lot of languages at once ( https://tree-sitter.github.io/tree-sitter/#available-parser
by chriswarbo 6y ago
Very cool. I suppose these days tree-sitter would be one way to support a lot of languages at once ( https://tree-sitter.github.io/tree-sitter/#available-parsers https://tree-sitter.github.io/tree-sitter/#available-parsers )
- WorldMaker 6y agoStill a tiny fraction compared to available tokenizers in Pygments: https://pygments.org/languages/ https://pygments.org/languages/ (Or the vast number of TextMate grammars in the wild, such as editors like VS Code have standardized on.) The "universality" of tokenizers is tough to beat, and though the tokenizers seem "dumber" than parser brethren, it's at a nice maxima for "good enough" for things like character-based diffs.