3 ms·

Cryptography yes, but are you sure about parsers? As far as I can tell, there's some kind of U-curve there. Beginners code them by hand, intermediate-level prog

by pkkm 3y ago

Cryptography yes, but are you sure about parsers? As far as I can tell, there's some kind of U-curve there. Beginners code them by hand, intermediate-level programmers and intermediate-scope projects use parser generators, and people maintaining the most sophisticated parsers prefer to code them by hand too. For example, GCC used to have a bison parser, but they switched to a hand-coded recursive descent one because that let them produce more helpful error messages. Clang uses recursive descent too.

mdaniel 3y ago
I offer, again, my JetBrains GrammarKit counterpoint from the last time that assertion came up <https://news.ycombinator.com/item?id=38192427 https://news.ycombinator.com/item?id=38192427> >>> I consider the JetBrains parsing system to be world class and they seem to hand-write very few (instead building on this system: https://github.com/JetBrains/Grammar-Kit#readme https://github.com/JetBrains/Grammar-Kit#readme ) - https://github.com/JetBrains/intellij-community/blob/idea/233.11555.11/java/java-frontback-psi-impl/src/com/intellij/lang/java/lexer/_JavaLexer.flex https://github.com/JetBrains/intellij-community/blob/idea/23... (the parser I'll concede, as they do seem to be hand-rolling that part) - https://github.com/JetBrains/intellij-community/blob/idea/233.11555.11/python/python-parser/src/com/jetbrains/python/lexer/Python.flex https://github.com/JetBrains/intellij-community/blob/idea/23... (same for its parser) - https://github.com/JetBrains/intellij-community/blob/idea/233.11555.11/plugins/sh/core/src/com/intellij/sh/lexer/Sh.flex https://github.com/JetBrains/intellij-community/blob/idea/23... and https://github.com/JetBrains/intellij-community/blob/idea/233.11555.11/plugins/sh/core/grammar/sh.bnf https://github.com/JetBrains/intellij-community/blob/idea/23... - https://github.com/JetBrains/intellij-plugins/blob/idea/233.11555.11/makefile/grammars/MakefileLexer.flex https://github.com/JetBrains/intellij-plugins/blob/idea/233.... and https://github.com/JetBrains/intellij-plugins/blob/idea/233.11555.11/makefile/grammars/Makefile.bnf https://github.com/JetBrains/intellij-plugins/blob/idea/233....
- Xeamek 3y ago
  To be fair though, jetbrains use case is fairly unique, as they basically want to implement parsing for as many languages as possible, all while doing it in a verry structured and consistent way, with having many other parts of their infrastructure being dependent on that parsing API. I think it's fair to say that those requirements are outside of the norm
  - mdaniel 3y ago
    I think that's a fine observation, but I'll also add that since their cases are almost always consumed in an editor context, they need them to be performant as well as have strong support for error recovery, since (in my mental model) the editor spends 90% of its time in a bad state. If I understand tree-sitter correctly, those are some of its goals, too, for the same reason

by pkkm 3y ago

mdaniel 3y ago

I offer, again, my JetBrains GrammarKit counterpoint from the last time that assertion came up <https://news.ycombinator.com/item?id=38192427 https://news.ycombinator.com/item?id=38192427> >>> I consider the JetBrains parsing system to be world class and they seem to hand-write very few (instead building on this system: https://github.com/JetBrains/Grammar-Kit#readme https://github.com/JetBrains/Grammar-Kit#readme ) - https://github.com/JetBrains/intellij-community/blob/idea/233.11555.11/java/java-frontback-psi-impl/src/com/intellij/lang/java/lexer/_JavaLexer.flex https://github.com/JetBrains/intellij-community/blob/idea/23... (the parser I'll concede, as they do seem to be hand-rolling that part) - https://github.com/JetBrains/intellij-community/blob/idea/233.11555.11/python/python-parser/src/com/jetbrains/python/lexer/Python.flex https://github.com/JetBrains/intellij-community/blob/idea/23... (same for its parser) - https://github.com/JetBrains/intellij-community/blob/idea/233.11555.11/plugins/sh/core/src/com/intellij/sh/lexer/Sh.flex https://github.com/JetBrains/intellij-community/blob/idea/23... and https://github.com/JetBrains/intellij-community/blob/idea/233.11555.11/plugins/sh/core/grammar/sh.bnf https://github.com/JetBrains/intellij-community/blob/idea/23... - https://github.com/JetBrains/intellij-plugins/blob/idea/233.11555.11/makefile/grammars/MakefileLexer.flex https://github.com/JetBrains/intellij-plugins/blob/idea/233.... and https://github.com/JetBrains/intellij-plugins/blob/idea/233.11555.11/makefile/grammars/Makefile.bnf https://github.com/JetBrains/intellij-plugins/blob/idea/233....

Xeamek 3y ago
To be fair though, jetbrains use case is fairly unique, as they basically want to implement parsing for as many languages as possible, all while doing it in a verry structured and consistent way, with having many other parts of their infrastructure being dependent on that parsing API. I think it's fair to say that those requirements are outside of the norm
- mdaniel 3y ago
  I think that's a fine observation, but I'll also add that since their cases are almost always consumed in an editor context, they need them to be performant as well as have strong support for error recovery, since (in my mental model) the editor spends 90% of its time in a bad state. If I understand tree-sitter correctly, those are some of its goals, too, for the same reason