3 ms·

I have seen multiple projects that run into maintenance problems due to the large grammar with thousands lines and antlr 3/4 incompatibility. Generally parser

by brutuz 7y ago

I have seen multiple projects that run into maintenance problems due to the large grammar with thousands lines and antlr 3/4 incompatibility. Generally parser generator adds a layer of complexity/constraint which may be significant if you need full control of lexing, parsing, semantic processing, error recovery of your language. After working on a few language projects (including core language, web-based editor with syntax coloring and auto-completion) myself I would strongly suggest hand-rolled parser for any serous language endeavour(general purpose or DSL).

grandinj 7y ago
Second this. Recursive descent parsers are much easier in the long run, even if you can't immediately see your grammar after they have grown for a while.
- raverbashing 7y ago
  I would agree but... From trying to understand parsing and RDPs I think I don't have the part of the brain required to understand it. Not that it isn't simple, it is. But it seems examples (as usual) overexplain the simple things then overlook something that seems obvious but isn't. The only time I managed to write a parser for simple math that wasn't an example was through the use of 'reverse production' parsing. Yes, it's the worse way, but it worked for me (this was a long time ago though)
  - unionpivo 7y ago
    Try this from Crenshaw. Its old but just assumes basic programming skills (even if you olnly know javascript its close enough) I manged to follow it when i was 14, when i found it on one of the newsgroups. https://compilers.iecc.com/crenshaw/ https://compilers.iecc.com/crenshaw/ here is port of code to C https://github.com/lotabout/Let-s-build-a-compiler https://github.com/lotabout/Let-s-build-a-compiler
ftomassetti 7y ago
I am a bit surprised by this, as I had the opposite experience. I find ANTLR grammars much more maintanable than the hand-written parsers I encountered. Indeed I was asked to port hand-written parsers to ANTLR for maintanability. Also, ANTLR4 seems to me to produce grammars which are clearer than ANTLR3. The weak point is error recovering, in my opinion. While ANTLR offers a sort-of-decent error recovery strategy for free, one has to customize it if they want to get great error messages. I think that many maintainability issues are due to poor usage of ANTLR. What we do is to: 1) Limit semantic actions 2) For complex languages do tree-transformations, after parsing With this approach we got pretty decent results. Personally I find hand-rolled parsers too costly to build and harder to maintain, but I have limited experience with them. I guess it also depends on the context: if you are designing a DSL while writing the parser you want to be able to evolve it very quickly and in that context I think that a parser generator is very useful. If instead I would need to build an industrial grade parser for a general purpose language, having a large budget, then I would go for an hand-rolled parser
- mxz3000 7y ago
  My experience has been similar to yours. The company I work at has very complex business logic written as a custom DSL parsed/executed by a crazy pile of Perl code. Recently we had requirements to come up with a way of providing on-the-fly validation in a web editor. This was only possible by ditching the old implementation and re-writing the grammar using ANTLR. While the old implementation is unmaintainable and (probably, who knows?) buggy, the ANTLR implementation is trivial to work on, test and add new features to. If you're working with a limited time budget, which is common when your main job isn't to maintain the language, then parser generators such as ANTLR are a godsend. ANTLR even enables you to generate parsers in different languages depending on where it needs to be executed. Need something to run client-side, in the users browser? Generate a JavaScript parser and you're done. Need it to run in the Java backend? Generate it in Java and call it a day. While it's true that error handling isn't the best, it's already better than nothing, and, as you say, can probably be improved with a bit of customization.

by brutuz 7y ago

grandinj 7y ago

Second this. Recursive descent parsers are much easier in the long run, even if you can't immediately see your grammar after they have grown for a while.

raverbashing 7y ago
I would agree but... From trying to understand parsing and RDPs I think I don't have the part of the brain required to understand it. Not that it isn't simple, it is. But it seems examples (as usual) overexplain the simple things then overlook something that seems obvious but isn't. The only time I managed to write a parser for simple math that wasn't an example was through the use of 'reverse production' parsing. Yes, it's the worse way, but it worked for me (this was a long time ago though)
- unionpivo 7y ago
  Try this from Crenshaw. Its old but just assumes basic programming skills (even if you olnly know javascript its close enough) I manged to follow it when i was 14, when i found it on one of the newsgroups. https://compilers.iecc.com/crenshaw/ https://compilers.iecc.com/crenshaw/ here is port of code to C https://github.com/lotabout/Let-s-build-a-compiler https://github.com/lotabout/Let-s-build-a-compiler

ftomassetti 7y ago

I am a bit surprised by this, as I had the opposite experience. I find ANTLR grammars much more maintanable than the hand-written parsers I encountered. Indeed I was asked to port hand-written parsers to ANTLR for maintanability. Also, ANTLR4 seems to me to produce grammars which are clearer than ANTLR3. The weak point is error recovering, in my opinion. While ANTLR offers a sort-of-decent error recovery strategy for free, one has to customize it if they want to get great error messages. I think that many maintainability issues are due to poor usage of ANTLR. What we do is to: 1) Limit semantic actions 2) For complex languages do tree-transformations, after parsing With this approach we got pretty decent results. Personally I find hand-rolled parsers too costly to build and harder to maintain, but I have limited experience with them. I guess it also depends on the context: if you are designing a DSL while writing the parser you want to be able to evolve it very quickly and in that context I think that a parser generator is very useful. If instead I would need to build an industrial grade parser for a general purpose language, having a large budget, then I would go for an hand-rolled parser

mxz3000 7y ago
My experience has been similar to yours. The company I work at has very complex business logic written as a custom DSL parsed/executed by a crazy pile of Perl code. Recently we had requirements to come up with a way of providing on-the-fly validation in a web editor. This was only possible by ditching the old implementation and re-writing the grammar using ANTLR. While the old implementation is unmaintainable and (probably, who knows?) buggy, the ANTLR implementation is trivial to work on, test and add new features to. If you're working with a limited time budget, which is common when your main job isn't to maintain the language, then parser generators such as ANTLR are a godsend. ANTLR even enables you to generate parsers in different languages depending on where it needs to be executed. Need something to run client-side, in the users browser? Generate a JavaScript parser and you're done. Need it to run in the Java backend? Generate it in Java and call it a day. While it's true that error handling isn't the best, it's already better than nothing, and, as you say, can probably be improved with a bit of customization.