3 ms·
Contrary to the opinion of many HN posters, I actually believe the front-end (lexing, parsing, semantic analysis) is more important to most software engineers.
by markus_zhang 6y ago
Contrary to the opinion of many HN posters, I actually believe the front-end (lexing, parsing, semantic analysis) is more important to most software engineers.
Consider: You don't get to work on the back-end for most of the careers out there. But Lexing and parsing are very valuable if you want to parse something, say a configuration file. These kinds of things are more often seen in day-in-day work than code generation and optimization.
- chrisseaton 6y agoI think they should split university courses - a course about parsing (no need to even mention compilers) and a course about compilers that starts from an AST. Many the former a requirement and the latter an elective.
- cestith 6y agounder 40 days ago someone posted to HN a SIGPLAN story about teaching compiler from the final phase to the first phase of processing the translations. https://news.ycombinator.com/item?id=26237368 https://news.ycombinator.com/item?id=26237368 It gives you an AST you can assume is valid and has you do the work to rename variables and methods. Then it gives you an AST and has you develop your target code (in the case of the specific class, LLVM IR). Then the AST is checked for valid semantics. Then the course has you check a source program for syntactic validity and generate an AST. It gives people the advantage of understanding what the AST is for on a deeper level before deciding how to represent it. I think this sort of class and a separate parsing class could be stellar together.
- kingaillas 6y agoMy university did that... sort of. There was a required "Theory of Computation" course using the Ullman & Hopcroft textbook "Introduction to Automata Theory, Languages, and Computation". A newer textbook could also be Sipser's "Introduction to the Theory of Computation".
- Koshkin 6y agoAs a side note, "lexing" is jargon that has never been in use by authors of books on compilers.
- chrisseaton 6y agoIt's just short for lexically analysing. The Dragon book has almost a hundred pages on lexical analysis.
- Turing_Machine 6y ago"Jargon", perhaps. "Never been in use by authors"? Not so. The current "international economy edition" of Aho, et al uses "lexing" 29 times, according to the search function on Amazon. Flex & Bison by John Levine (O'Reilly and Associates) uses "lexing" 38 times. And so on.
- mhh__ 6y agoThe people who write compilers people actually use say lexing quite regularly.
- jonstewart 6y agoAgreed. Lexing and parsing might not be the most critical aspects of a compiler these days, but they’re more generally useful.
- ufo 6y agoI agree, partially. I also think that the front end of a compiler is more broadly useful skill than the back end stuff. However, I believe that most people would be better served knowing how to work with a recursive descent parser instead of a bottom up parser generator, which is something that the dragon book spends a lot of time on. For this reason, one of my favorite compiler books is munificent's Crafting Interpreters.
- markus_zhang 6y agoI'm actually reading that book. I'm going back to university to study CS and I have a course about Java which is exactly the language he uses in the first part of the book. I think this would be a good exercise for learning the language as well as learning lexing/parsing. I was blocked by the BNF expression in section 6.1. Not exactly sure what I didn't understand, but overall I felt like I could only copy what he says but not grasp the underlying rules. Maybe I need some primer on BNF first.
- cjohansson 6y agoAho and Ullman has a great book about Parsing, Translating and Compiling that I'm currently reading, it's not a either or thing - you can learn about both and they are related to each other
- wuuwoo 6y agoI wholeheartedly disagree. Compiler engineering is ultimately about taking data in one representation and converting to another, in practice much of the job is figuring out what data you need to have access to in order to perform a transformation and test that it is sound - the plumbing is usally uninteresting and straightforward. Code generation and optimization are just the task you're trying to complete, the actual implementation of it is not so different from most SWE grunt work. But many of the abstractions you build when trying to do it are useful to understand and use in practice everywhere you right software. If you've ever worked on an application that ingests data and spits out meaningful output, you've written a compiler. Failing to understand the shared abstractions is not good engineering.
- mhh__ 6y ago> front-end (lexing, parsing, semantic analysis lexing, parsing, SEMANTIC ANALYSIS (Emphasis added!)
- izgzhen 6y agoI would recommend most software engineers to avoid inventing a new configuration or other DSL and the parser/lexer for it. This can easily lead to hard-to-debug program and long-term technical debt. Always research for existing and well-tested solutions first (even JSON!). Even if you don't invent your language, you can avoid writing a low-level parser/lexer by using a higher-level format, like context-free grammar (see Lark https://github.com/lark-parser/lark https://github.com/lark-parser/lark). Define and maintain a grammar is much easier.
- m463 6y agoif json only allowed comments...
- neilparikh 6y agoI agree that lexing and parsing is probably more useful day to day, but for those use cases, I think a approach like parser combinators will be much easier than the formal approach taken in a compilers course or book.