4 ms·

I say this all the time. As someone who wrote many many parsers before, flex/bison is not worth it. Parsing is a solved problem and -- depending on your style -

I say this all the time. As someone who wrote many many parsers before, flex/bison is not worth it. Parsing is a solved problem and -- depending on your style -- nothing other than recursive descent or parsing combinators is worth it, imho. Always hand write your parsers. Also, ideally you should never write parsers at all!

jcranmer 3y ago
Flex and bison don't really give you much advantage over hand-writing your parser, and they have an interface which is actually kind of painful to use (they assume that your program will only have a single parser, ever, and everything is global variables with names like yytable). Virtually the entire infrastructure you have to provide yourself--you don't even have the ability to get autogenerated token enums or AST trees, so if your lexer is pretty close to trivial, it's more work to get it working with flex. Note that some other parser frameworks do provide some more functionality that may tip the calculation for the worth in other directions--I've used the Python-based ply parser generator a couple of times, though I'd still prefer a framework that really did location tracking (i.e., file:line:col information) for you out of the box.
- crabbone 3y ago
  Are you kidding me "not much advantage"? You write mountains of C code where you manipulate arrays of characters every now and then interpreting them as strings. This is like one of the most dangerous things you could do in C, and will take you forever to write and to iron out all the cases where you accidentally read past the missing null terminator. And you are liberated from this madness by using a generator. It's not even about the quality of the resulting parser (it's also rare that parsing is the bottleneck in larger programs that need to parse something, so the question of shift-reduces vs recursive etc. aren't worth answering really). It's more about confidence that the program won't crash on a different computer with different memory allocator. > everything is global variables This is like information from the 90s... That's not the case for a long time. You can still do it like this, but re-entrant parsers have been a thing for a long time... I've written multithreaded parsers using Flex / Bison. Again, don't get me wrong, I'm not saying that Flex / Bison are the best combo. I prefer DCGs because it's easier for me to think in that way, for example. But the benefit of having popular tools with decades of track record and easily available outweighs the benefits of hand-written parser, unless you are somehow very good and into writing parsers. The code in question doesn't show any such ability / intention. To the point that I was questioning if the code may have been generated and later "prettyfied" by hand. I mean, it's not awful, but it's not like it's doing it in some sort of a remarkable way... It just looks like a lot of tedious labor.

jcranmer 3y ago

Flex and bison don't really give you much advantage over hand-writing your parser, and they have an interface which is actually kind of painful to use (they assume that your program will only have a single parser, ever, and everything is global variables with names like yytable). Virtually the entire infrastructure you have to provide yourself--you don't even have the ability to get autogenerated token enums or AST trees, so if your lexer is pretty close to trivial, it's more work to get it working with flex. Note that some other parser frameworks do provide some more functionality that may tip the calculation for the worth in other directions--I've used the Python-based ply parser generator a couple of times, though I'd still prefer a framework that really did location tracking (i.e., file:line:col information) for you out of the box.

crabbone 3y ago
Are you kidding me "not much advantage"? You write mountains of C code where you manipulate arrays of characters every now and then interpreting them as strings. This is like one of the most dangerous things you could do in C, and will take you forever to write and to iron out all the cases where you accidentally read past the missing null terminator. And you are liberated from this madness by using a generator. It's not even about the quality of the resulting parser (it's also rare that parsing is the bottleneck in larger programs that need to parse something, so the question of shift-reduces vs recursive etc. aren't worth answering really). It's more about confidence that the program won't crash on a different computer with different memory allocator. > everything is global variables This is like information from the 90s... That's not the case for a long time. You can still do it like this, but re-entrant parsers have been a thing for a long time... I've written multithreaded parsers using Flex / Bison. Again, don't get me wrong, I'm not saying that Flex / Bison are the best combo. I prefer DCGs because it's easier for me to think in that way, for example. But the benefit of having popular tools with decades of track record and easily available outweighs the benefits of hand-written parser, unless you are somehow very good and into writing parsers. The code in question doesn't show any such ability / intention. To the point that I was questioning if the code may have been generated and later "prettyfied" by hand. I mean, it's not awful, but it's not like it's doing it in some sort of a remarkable way... It just looks like a lot of tedious labor.