3 ms·

Author here. I'm going to be super shameless here and quote myself in that chapter: > [...] we are here to learn, we want to understand how parsers work. And i

Author here. I'm going to be super shameless here and quote myself in that chapter: > [...] we are here to learn, we want to understand how parsers work. And it’s my opinion that the best way to do that is by getting our hands dirty and writing a parser ourselves. Also, I think it’s immense fun. In other words: use a parser generator when you need a working parser, quick. But if you want to learn how parsers work, what ASTs are, what "top-down parsing" means, etc. then I recommend you write your own. It's not that hard once the concept "clicked" and, again, it's a ton of fun :)

alexgmcm 7y ago
Yeah, I did the Nand2Tetris course which culminates in writing a toy compiler and the philosophy there is the same as yours. I really like that philosophy - as Feynman said: "What I cannot build I do not understand"
- tom_mellior 7y ago
  > "What I cannot build I do not understand" This is true, but it provides no guidance on the question whether you need to understand it. If you don't want to build a web browser, you don't need to understand how web browsers are built. That doesn't mean that it is dishonorable for you to use a web browser. So do you need to understand how generated parsers work? I think that's a matter of preference that Feynman doesn't help you decide.
  - macintux 7y ago
    If you’re designing a language, it’s not a stretch to assume that understanding how parsers work would be useful.
    - tom_mellior 7y ago
      I think "you plug a grammar into a generator, and you get back a function that maps character sequences to a syntax tree" can be enough understanding for a lot of use cases. Maybe you're more interested in any of the many other aspects that go into designing and implementing an interpreted language -- type checking, garbage collection, an efficient bytecode format and interpreter for example. But maybe you aren't, in which case I agree that case a hand-written parser might be right up your alley! There's no one size fits all.
      - greenshackle2 7y ago
        You need a little bit more than that, cause if that is the limit of your understanding, you will have a bad time when the generator spits back an error message about reduce/reduce conflicts or whatever instead of a function.
pizza234 7y ago
> use a parser generator when you need a working parser, quick. Not necessarily. Use it also when you need a more correct one :-) The way the book presents the parsing subject leaves the impression that a handcrafted parser is generally preferrable. I have experience with a project that was based on that book, and there were (and probably, are) a lot of bugs in the parsing code, that wouldn't have been there if a parser generator was used. I don't camp for using parser generator at all costs, however, presenting both sides of the coin would allow readers to make a (more) informed choice.
- misternugget 7y ago
  > I have experience with a project that was based on that book, and there were (and probably, are) a lot of bugs in the parsing code, that wouldn't have been there if a parser generator was used. I think I need to a add a big "do. not. use. in. production!" disclaimer then :) The parser we build in the book is, of course, not a battle-tested, industrial-grade parser that can survive every fuzzer you throw at it. Far from it, but that's fine, since that was never the goal. The goal was always to learn. So, yes, I agree. If you do need a production-ready, stable parser: use a mature parser generator or, maybe, invest more time in getting good at writing parsers than it takes to read and work through the ~100 pages I wrote on the subject.
  - bigtunacan 7y ago
    Since you have written a book in the subject you obviously are much more of an expert than I am. After reading your book what materials would you recommend for someone wanting to reach the stage is bring able to implement their on "production grade" parsers?

alexgmcm 7y ago

Yeah, I did the Nand2Tetris course which culminates in writing a toy compiler and the philosophy there is the same as yours. I really like that philosophy - as Feynman said: "What I cannot build I do not understand"

tom_mellior 7y ago
> "What I cannot build I do not understand" This is true, but it provides no guidance on the question whether you need to understand it. If you don't want to build a web browser, you don't need to understand how web browsers are built. That doesn't mean that it is dishonorable for you to use a web browser. So do you need to understand how generated parsers work? I think that's a matter of preference that Feynman doesn't help you decide.
- macintux 7y ago
  If you’re designing a language, it’s not a stretch to assume that understanding how parsers work would be useful.
  - tom_mellior 7y ago
    I think "you plug a grammar into a generator, and you get back a function that maps character sequences to a syntax tree" can be enough understanding for a lot of use cases. Maybe you're more interested in any of the many other aspects that go into designing and implementing an interpreted language -- type checking, garbage collection, an efficient bytecode format and interpreter for example. But maybe you aren't, in which case I agree that case a hand-written parser might be right up your alley! There's no one size fits all.
    - greenshackle2 7y ago
      You need a little bit more than that, cause if that is the limit of your understanding, you will have a bad time when the generator spits back an error message about reduce/reduce conflicts or whatever instead of a function.

pizza234 7y ago

> use a parser generator when you need a working parser, quick. Not necessarily. Use it also when you need a more correct one :-) The way the book presents the parsing subject leaves the impression that a handcrafted parser is generally preferrable. I have experience with a project that was based on that book, and there were (and probably, are) a lot of bugs in the parsing code, that wouldn't have been there if a parser generator was used. I don't camp for using parser generator at all costs, however, presenting both sides of the coin would allow readers to make a (more) informed choice.

misternugget 7y ago
> I have experience with a project that was based on that book, and there were (and probably, are) a lot of bugs in the parsing code, that wouldn't have been there if a parser generator was used. I think I need to a add a big "do. not. use. in. production!" disclaimer then :) The parser we build in the book is, of course, not a battle-tested, industrial-grade parser that can survive every fuzzer you throw at it. Far from it, but that's fine, since that was never the goal. The goal was always to learn. So, yes, I agree. If you do need a production-ready, stable parser: use a mature parser generator or, maybe, invest more time in getting good at writing parsers than it takes to read and work through the ~100 pages I wrote on the subject.
- bigtunacan 7y ago
  Since you have written a book in the subject you obviously are much more of an expert than I am. After reading your book what materials would you recommend for someone wanting to reach the stage is bring able to implement their on "production grade" parsers?