4 ms·
> Going from a grammar to an AST is the first and one of the hardest step IMO I can give some concrete data about this. I've been working professionally on a s
by chrisseaton 6y ago
> Going from a grammar to an AST is the first and one of the hardest step IMO
I can give some concrete data about this. I've been working professionally on a single compiler full-time for seven years.
I think out of all that time I think I've spent probably less than a week dealing with lexer and parser issues all together. I'd say in practice parsing is vanishingly insignificant.
- zabzonk 6y ago> I've been working professionally on a single compiler full-time for seven years. Which one?
- chrisseaton 6y agoTruffleRuby (says in my profile.)
- MaxBarraclough 6y agoThings would be different if you were in the business of, say, keeping g++ up to date with the latest C++ spec, right?
- chrisseaton 6y agoI have to keep up with Ruby changes. But even for example the parser in clang doesn't seem to be touched very often. https://github.com/llvm/llvm-project/commits/main/clang/lib/Parse/Parser.cpp https://github.com/llvm/llvm-project/commits/main/clang/lib/... Take a look at the commits there and see how much each changes the parser vs the rest of the compiler. Also, at the bottom of page 2 for parser changes you're already all the way back in 2017!
- jcranmer 6y agoIt's somewhat disingenuous to link only clang/lib/Parse/Parser.cpp--a lot of the parsing code isn't in Parser.cpp, but other files in lib/Parser. Look at the entire directory, and you get about 1 commit per few days (somewhere between daily and weekly).
- chrisseaton 6y agoI think a lot of the commits you're seeing there are really more semantic analysis and tooling rather than actual parsing - they just interact with the parser. I can only offer my experience - and I think most of my colleagues in the field seem to say the same thing.
- jabl 6y agoI guess it's a difference who the course audience is? I'd wager most people who take a compilers 101 class at uni won't end up working on compilers for their careers. For them, some basic knowledge of parsing will likely serve them well, since designing and parsing file formats and network protocols etc. is something quite a lot of people need to do now and then (perhaps less so nowadays that you can find a library or framework to do almost anything you might want to, but still). These people will, again, be well served by some basic intuition how optimizing compilers work, but most likely they will never need to implement such a thing themselves. OTOH, if working on optimizing compilers is your career, sure, the parser is likely something that occasionally needs some minor updating, but the vast majority of time you spend on the middle/back-end optimization passes. Very different focus from the average Joe programmer taking a compilers 101 class at uni.
- chrisseaton 6y agoI think it could possibly be a good idea to have separate parsing and compiler courses, and teach parsing as a more general technique, and have it as a prerequisite for compilers.
- hvidgaard 6y agoIt's in the context of a compiler course. When I was TA I've never had students that found it easy. Semantic analysis and optimizations are as easy or hard as the course material dictates. I believe the focus was on the first part because for the most part, writing a small DSL and converting that into an AST is relevant for most CS professionals, where as I think semantic analysis and optimization are only day-to-day relevant for the small subset of people that work on compilers.
- vidarh 6y agoCouldn't agree more. You've spent a lot more time on it than me, but when I started writing my blog posts about writing a compiler this was the reason I started writing about code-generation first -- far too few resources focuses on that part, despite it being far harder than the lexing and parsing.