3 ms·

It is nice to see someone summarizing this kind of information. However, really this is a continuation of the academic attitude toward parsers making them MUCH

by jblow 12y ago

It is nice to see someone summarizing this kind of information. However, really this is a continuation of the academic attitude toward parsers making them MUCH harder than they have to be. If you want to study grammars in an abstract sense, then think of them this way, and that's fine. If you want to build a parser for a programming language, don't use any of this stuff. Just write code to parse the language in a straightforward way. You'll get a lot more done and the resulting system will be much nicer for you and your users.

wodenokoto 12y ago
That's horrible advice. Some problems are really complex and can't be solved just by working from A to B. Your parser will probably not end up being able to handle the problems outlined in the article unless you take that theory into account before starting to program.
- jblow 12y ago
  You never know, you might be talking to someone with actual experience.
l_dopa 12y ago
This approach might be why several ubiquitous languages have needlessly ambiguous grammars. Decades of writing tools to be bug-compatible with the original implementation is much harder than learning a tiny bit of theory.
chriswarbo 12y ago
> Just write code to parse the language in a straightforward way. This approach is why many consider parsing to be a solved problem, so it's certainly a valid approach. However, it's not the only valid approach. For example, "straightforward" parsers often give terrible error messages: when the intended branch (eg. if/then/else) fails, the parser will backtrack and try a more general alternative (eg. a function call). Not only does this give an incorrect error (eg. "no such function 'esle'"), but it might actually succeed! In which case, the parser will be in the wrong state to parse the following text, and gives a non-sensical message (eg. "unexpected '('" several lines later). This is an important problem, since these messages can only be decyphered by those who know enough about the syntax to avoid hitting them very often! Inexperienced users will see error messages over and over again, and have no idea that they're being asked to fix non-existent errors in incorrect positions.
lvh 12y ago
Which tools in particular do you feel are acceptable? "Just writing the code to parse the language" is almost definitional of shotgun parsers, so it sounds like you are advocating an ad-hoc parser (probably something recursive descent-ish, or perhaps with even less structure than that), but I wouldn't want to put words in your mouth. ADDENDUM: In particular, the article mentions PEGs. Not in an unambiguously positive light, but nonetheless: do you count PEGs as being a useless overly academic tool? The LANGSEC project has done (in my opinion) a pretty good job arguing that this behavior causes many security issues in software. Perhaps only to some extent for programming languages in general, but in particular for data format languages and network protocols. The issue is slightly mitigated for programming languages because they typically don't have many wildly varying parsers; but (programming) languages without clearly specified grammars certainly have had problems with this in the past. Are you familiar with their work? Do you just disagree with their conclusions? [LANGSEC]: http://langsec.org http://langsec.org

by jblow 12y ago

wodenokoto 12y ago

That's horrible advice. Some problems are really complex and can't be solved just by working from A to B. Your parser will probably not end up being able to handle the problems outlined in the article unless you take that theory into account before starting to program.

jblow 12y ago
You never know, you might be talking to someone with actual experience.

l_dopa 12y ago

This approach might be why several ubiquitous languages have needlessly ambiguous grammars. Decades of writing tools to be bug-compatible with the original implementation is much harder than learning a tiny bit of theory.

chriswarbo 12y ago

> Just write code to parse the language in a straightforward way. This approach is why many consider parsing to be a solved problem, so it's certainly a valid approach. However, it's not the only valid approach. For example, "straightforward" parsers often give terrible error messages: when the intended branch (eg. if/then/else) fails, the parser will backtrack and try a more general alternative (eg. a function call). Not only does this give an incorrect error (eg. "no such function 'esle'"), but it might actually succeed! In which case, the parser will be in the wrong state to parse the following text, and gives a non-sensical message (eg. "unexpected '('" several lines later). This is an important problem, since these messages can only be decyphered by those who know enough about the syntax to avoid hitting them very often! Inexperienced users will see error messages over and over again, and have no idea that they're being asked to fix non-existent errors in incorrect positions.

lvh 12y ago

Which tools in particular do you feel are acceptable? "Just writing the code to parse the language" is almost definitional of shotgun parsers, so it sounds like you are advocating an ad-hoc parser (probably something recursive descent-ish, or perhaps with even less structure than that), but I wouldn't want to put words in your mouth. ADDENDUM: In particular, the article mentions PEGs. Not in an unambiguously positive light, but nonetheless: do you count PEGs as being a useless overly academic tool? The LANGSEC project has done (in my opinion) a pretty good job arguing that this behavior causes many security issues in software. Perhaps only to some extent for programming languages in general, but in particular for data format languages and network protocols. The issue is slightly mitigated for programming languages because they typically don't have many wildly varying parsers; but (programming) languages without clearly specified grammars certainly have had problems with this in the past. Are you familiar with their work? Do you just disagree with their conclusions? [LANGSEC]: http://langsec.org http://langsec.org