4 ms·
I learned about the existence of Lex, Yacc, and Bison a bit more than 20 years while discussing the difficulties of debugging handwritten parsers with a friend.
by stephc_int13 5y ago
I learned about the existence of Lex, Yacc, and Bison a bit more than 20 years while discussing the difficulties of debugging handwritten parsers with a friend.
But I was quickly discouraged by the complexity of using these tools, they are more like a new language that you have to learn.
Today I prefer writing parsers by hand, in C, like Fabrice Bellard, it is not super easy but manageable and I never encountered major issues.
I think that Lex/Yacc and similar tools are a good illustration of the power and shortcomings of metaprogramming.
There are some obvious use cases, but it is not always the best choice.
- 1vuio0pswjnm7 5y agoHow easy is it to make quick changes to hand-written parsers. Imagine, for example, that the input being parsed is not under the authors control and it suddenly changes. The parser must be changed.
- mateo411 5y agoThe input changing shouldn't matter. Maybe you could see a performance issue if you had to parse a really large file/files. The issue would be if the grammar to what you are parsing changes. If that changes, then no parser will work, correctly until it supports the new grammar. As a practical matter the grammar can act like a contract between the two systems. You don't have control of the input, but the input must follow the same grammar.
- 1vuio0pswjnm7 5y agoDoes the separation of the grammar into its own file, like a .y file for yacc, make it easier to modify the grammar at a later point. Similarly, is it easier to modify a parser later on, if the input changes, if the patterns to be searched in the input and the state machine are stored in a separate file, like an .l file for flex.
- mateo411 5y agoI would think so. I don't work on parsers and grammar day to day, so I might not be the best person to ask about this. I am interested in the subject though. I just don't get to work on this in my current day job.
- mbrodersen 5y agoIt is super easy to change. It is just normal code.
- 1vuio0pswjnm7 5y agoWhy then do people use programs like ANTLR, Ragel, Lemon/re2c, etc.
- mbrodersen 5y agoProbably because they don’t know how to write parsers by hand and assume it is hard?
- dilap 5y agoSame experience here; recursive descent parser advantages: - It's just normal code; you don't have to go learn a whole 'nother thing - Easy and fun to write (ymmv) - Does not add extra dependencies to your build system (the time I've wasted getting the right version of yacc installed just to handle software that parses some trivial grammar...) - You can easily deliver really good error messages to your users (they love this!) Disadvantages: If you're not careful, you can accidentally end up w/ some sort of weird frankenstein grammar
- mbrodersen 5y agoYep same here. It is really easy to write parsers by hand if you know what you are doing.