5 ms·

How do we usually construct compilers in industry today? I saw others mentioning parser generators -- is that the main difference?

chrisseaton 7y ago
This whole thread has been about parser generators, but that's not really what I meant! I think the biggest difference is the rich intermediate representations we use these days. These classic texts don't really address intermediate representations and are usually syntax-directed instead. These days a program will spent almost all of its time deconstructed into an IR.
- vector_spaces 7y ago
  I actually started reading Wirth because I was frustrated at how little coverage there seems to be of syntax related matters (lexing and parsing) in modern introductory books (with a couple of exceptions, notably the Dragon Book, but I've been having a hard time wrapping my head around it). I emailed an author of a book that specifically calls parsing a "distraction" (Shriram Krishnamurthi, in PLAI) and he explained that he and other textbook authors of recent texts are reacting to a sort of over-emphasis on parsing in older books and tools (for example, YACC standing for Yet Another Compiler Constructor despite being simply a parser generator). So it's interesting what you're saying about IRs -- I've definitely noticed Shriram's book and a few other recent ones seem to focus really heavily on them. I wonder if there's also a relationship between a more IR-focused approach and the reduced emphasis on parsing?
  - chrisseaton 7y ago
    There might be a bit of a backlash yes. But I think for example the Dragon book is comically unbalanced towards parsing. It's just ludicrous how much it drones on about this tiny tiny part of writing a compiler. I've been working professionally on one compiler continuously since 2013. I've spent like two or three days working on the parser. That's it. I've probably spent longer tweaking the Markdown of the README. For me the issue is that parsing is basically a solved problem. I don't see how you can make it much better for compilers. If you give me a language I can write you a parser for it. It won't be interesting, fun, or be better or worse than any other parser. It's a purely mechanical job. (Of course it could be a nicer parser, or a faster parser, or give better error messages, or work better for IDEs or tooling, but stick with me.) But the transformations that happen after that point? Well there a whole world opens up to you! You can do so much, in so many different ways, and achieve incredibly different results for different requirements.
  - pjmlp 7y ago
    The tiger books are relatively alright, with ML, Java and C flavours. They have a couple of typos, and one can feel that ML one was the first variant, but they are relatively good, covering a lot, specially if one does the exercises. There is a 2nd edition for the Java one, where the tiger language gets replaced by a Java subset.
  - emilsoman 7y ago
    Which are some books focused on IRs you’d recommend?
    - cellularmitosis 7y ago
      “Engineering a compiler” deals with IR
- badsectoracula 7y ago
  Keep in mind that Wirth is a "simplificator": given any task, he'll try to take the simplest approach (as an example see his talk for his 80th birthday and his remark on the complexity he saw at Xerox PARC which drove him to design Oberon). His languages are even designed so that they are easy to implement. Intermediate representations might be what many compilers use nowadays, but they are not necessary to write a compiler, thus they're out - sacrificed at the altar of simplicity :-).
  - pjmlp 7y ago
    To be fair, he just adopted the Xerox PARC approach regarding code generation, where bytecode gets generated, with multiple possible execution paths. The Xerox PARC workstations were microcoded, and the one needed to load the microcode on boot for the specific environment (XDE, Smalltalk, Interlisp-D,...). Likewise the P-Code as means of portability for Pascal, M-Code on Lillith, the Ceres and latter FPGA RISC for Oberon.
slavapestov 7y ago
I think the main difference is that in today’s compilers the parser is such a small part that whether you use a parser generator or roll your own recursive descent parser is mostly just a footnote.

chrisseaton 7y ago

This whole thread has been about parser generators, but that's not really what I meant! I think the biggest difference is the rich intermediate representations we use these days. These classic texts don't really address intermediate representations and are usually syntax-directed instead. These days a program will spent almost all of its time deconstructed into an IR.

vector_spaces 7y ago
I actually started reading Wirth because I was frustrated at how little coverage there seems to be of syntax related matters (lexing and parsing) in modern introductory books (with a couple of exceptions, notably the Dragon Book, but I've been having a hard time wrapping my head around it). I emailed an author of a book that specifically calls parsing a "distraction" (Shriram Krishnamurthi, in PLAI) and he explained that he and other textbook authors of recent texts are reacting to a sort of over-emphasis on parsing in older books and tools (for example, YACC standing for Yet Another Compiler Constructor despite being simply a parser generator). So it's interesting what you're saying about IRs -- I've definitely noticed Shriram's book and a few other recent ones seem to focus really heavily on them. I wonder if there's also a relationship between a more IR-focused approach and the reduced emphasis on parsing?
- chrisseaton 7y ago
  There might be a bit of a backlash yes. But I think for example the Dragon book is comically unbalanced towards parsing. It's just ludicrous how much it drones on about this tiny tiny part of writing a compiler. I've been working professionally on one compiler continuously since 2013. I've spent like two or three days working on the parser. That's it. I've probably spent longer tweaking the Markdown of the README. For me the issue is that parsing is basically a solved problem. I don't see how you can make it much better for compilers. If you give me a language I can write you a parser for it. It won't be interesting, fun, or be better or worse than any other parser. It's a purely mechanical job. (Of course it could be a nicer parser, or a faster parser, or give better error messages, or work better for IDEs or tooling, but stick with me.) But the transformations that happen after that point? Well there a whole world opens up to you! You can do so much, in so many different ways, and achieve incredibly different results for different requirements.
- pjmlp 7y ago
  The tiger books are relatively alright, with ML, Java and C flavours. They have a couple of typos, and one can feel that ML one was the first variant, but they are relatively good, covering a lot, specially if one does the exercises. There is a 2nd edition for the Java one, where the tiger language gets replaced by a Java subset.
- emilsoman 7y ago
  Which are some books focused on IRs you’d recommend?
  - cellularmitosis 7y ago
    “Engineering a compiler” deals with IR
badsectoracula 7y ago
Keep in mind that Wirth is a "simplificator": given any task, he'll try to take the simplest approach (as an example see his talk for his 80th birthday and his remark on the complexity he saw at Xerox PARC which drove him to design Oberon). His languages are even designed so that they are easy to implement. Intermediate representations might be what many compilers use nowadays, but they are not necessary to write a compiler, thus they're out - sacrificed at the altar of simplicity :-).
- pjmlp 7y ago
  To be fair, he just adopted the Xerox PARC approach regarding code generation, where bytecode gets generated, with multiple possible execution paths. The Xerox PARC workstations were microcoded, and the one needed to load the microcode on boot for the specific environment (XDE, Smalltalk, Interlisp-D,...). Likewise the P-Code as means of portability for Pascal, M-Code on Lillith, the Ceres and latter FPGA RISC for Oberon.

slavapestov 7y ago

I think the main difference is that in today’s compilers the parser is such a small part that whether you use a parser generator or roll your own recursive descent parser is mostly just a footnote.