5 ms·

Ask HN: Resources for learning to write compilers?

44 points by jason_slack 9y ago

I'm very interested in learning to write compilers, starting with just a simple made up language. Can anyone share resources? I prefer bound books, but anything is helpful! I've been reading the source of TempleOS for tips as well.

peter_d_sherman 9y ago
Tiny C Compiler https://bellard.org/tcc https://bellard.org/tcc ANTLR http://www.antlr.org http://www.antlr.org “Compiler Construction” by Niklaus Wirth (2014) [pdf] https://news.ycombinator.com/item?id=10764672 https://news.ycombinator.com/item?id=10764672
zerr 9y ago
Relevant question: how to write a debugger?
aalhour 9y ago
Hey, I have been in your place like last year, and as a result of researching the field of Compilers and Interpreters construction I wrote a curated list of awesome resources on Compilers, Interpreters and Runtimes, feel free to check it out at: https://github.com/aalhour/awesome-compilers https://github.com/aalhour/awesome-compilers. The list is huge, so I recommend you start with the Compilers course by Stanford and go through one of the two following books as a parallel reading: 1. Engineering a Compiler, K. Cooper, L. Torczon. 2. Modern Compiler Implementation in ML, by A. Appel. If you want a smaller book, the I recommend the Basics of Compiler Design by T. Mogensen, it is a nice and short one. All of these resources are linked to in the list. Happy tinkering!
sigjuice 9y ago
I am in the middle of reading Write Your Own Compiler by Nils M Holm. See http://t3x.org/t3x/book.html http://t3x.org/t3x/book.html for the book and the accompanying code. I like it so far. T3X is a tiny (made up!?) procedural language.
techjuice 9y ago
The following that I have on my bookshelf have helped: Compilers Second Edition [0] Engineering a Compiler, Second Edition [1] Computer Systems: A Programmer's Perspective (3rd Edition) [2] [0]https://www.amazon.com/Compilers-Principles-Techniques-Alfred-Aho-ebook/dp/B009TGD06W/ https://www.amazon.com/Compilers-Principles-Techniques-Alfre... [1]https://www.amazon.com/Engineering-Compiler-Second-Keith-Cooper/dp/012088478X https://www.amazon.com/Engineering-Compiler-Second-Keith-Coo... [2]https://www.amazon.com/Computer-Systems-Programmers-Perspective-3rd/dp/013409266X/ https://www.amazon.com/Computer-Systems-Programmers-Perspect...
- jason_slack 9y ago
  Thank you, these resources look great.
- DonaldPShimoda 9y ago
  CS:APP covers compilers? My school runs a clone of that course where we do a selection of the labs (malloc, shell, bomb, etc), but we didn't touch anything like compilers. Maybe I'll have to go look through that book again!
  - spenuke 9y ago
    Any chance your school makes its exercises available? I've been trying to go through CS:APP myself, but the exercises aren't included and I believe I'd need a .edu email address to get them.
    - jason_slack 9y ago
      I have a .edu address....
    - DonaldPShimoda 9y ago
      No, the authors guard that stuff pretty jealously, and instructors are forbidden from allowing the exercises or solutions from being published online. It's a well-known course used at a large number of schools outside of CMU (150+ I think?), so they're concerned about people distributing solutions or something. Maybe you can try emailing the authors about it or something?
connorjan 9y ago
If you're not looking to implement one totally from scratch, this could be a good start: http://llvm.org/docs/tutorial/ http://llvm.org/docs/tutorial/
- m-j-fox 9y ago
  Yeah, I've been working on LLVM lately and it seems like they've solved all the problems. Somehow, they solved them in standard C++. I've been through a bunch of compilers and I could barely hang on. In LLVM, everywhere I go, I'm like: oh, so that's how you do that. One thing that helped get started was slides from a CMU class. Hang on: https://www.cs.cmu.edu/~15745/syllabus.html https://www.cs.cmu.edu/~15745/syllabus.html Excellent work and so nice of them to post it for free.
harrisreynolds 9y ago
This recent HN post could be useful to you: https://jaseemabid.github.io/2017/07/04/compiler.html https://jaseemabid.github.io/2017/07/04/compiler.html The "dragon" books is one of the classics. (Compilers: Principles, Techniques, and Tools) ... linked to in another comment.
kqr2 9y ago
Sources of various compilers, assemblers, and linkers: http://www.exmortis.narod.ru/index_eng.html http://www.exmortis.narod.ru/index_eng.html
e19293001 9y ago
I bet this is what you are looking for: https://www.amazon.com/Compiler-Construction-Using-Java-Java.. https://www.amazon.com/Compiler-Construction-Using-Java-Java.... This book taught me how to write a compiler. Here is its description from its website: * Comprehensive treatment of compiler construction. * JavaCC and Yacc coverage optional. * Entire book is Java oriented. * Powerful software package available to students that tests and evaluates their compilers. * Fully defines many projects so students can learn how to put the theory into practice. * Includes supplements on theory so that the book can be used in a course that combines compiler construction with formal languages, automata theory, and computability theory. If you already know C or C++ or Java then this book is for you. In my opinion, you can learn many computer science concepts and be able to apply to your field. The book will teach you how to write a grammar then write a parser from it then eventually be able to improve it as you go on reading and doing the exercises. It was a great moment when I feel comfortable writing recursive functions since grammars are composed of recursive functions. You'll also learn a nice way on how you can get your compiler to generate assembly code. Another feature of the book is the chapter on Finite Automata wherein you'll learn how to convert between regular expressions, regular grammars and finite automata and eventually write your own 'grep' which was for me is a mind-blowing experience. There are lots of other stuffs in this book that you could learn.
gkuan 9y ago
I would recommend Appel's Modern Compiler Implementation series. It is good starting place for building up compiler fundamentals and progressively implementing a reasonable compiler for a reasonably rich language without getting bogged down from too many idiosyncrasies.
cweagans 9y ago
Start with an interpreter. Many of the concepts are the same, but you don't have to mess with code generation at the end. I enjoyed working through https://interpreterbook.com/ https://interpreterbook.com/ I have no relation to the author. I'm just a happy customer.
plinkplonk 9y ago
If you are fluent in one of the functional languages (Haskell, SML, OCaml, F#) Modern Compiler Implementation in ML (I'm not a fan of the C/Java versions) https://www.amazon.com/Modern-Compiler-Implementation-Andrew-Appel/dp/0521607647 https://www.amazon.com/Modern-Compiler-Implementation-Andrew... If you are fluent in a mainstream OO language - Java, C#, Ruby etc Programming Language Processors in Java: Compilers and Interpreters ( the code is in Java but can be trivially ported into any OO language ) https://www.amazon.com/Programming-Language-Processors-Java-Interpreters/dp/0130257869/ https://www.amazon.com/Programming-Language-Processors-Java-... Most compiler programming books use lex/yacc versions for lexing and parsing. Imo, this isn't a good way to learn lexing/parsing, and using recursive descent or combinator parsing approaches is (imho) the right way to begin. If you want to know how tools like lex and yacc are built, then Holub's "Compiler Construction in C" is very comprehensive and goes into great detail about the required CS theory- (automata DFA, NFA etc). The book seems to be out of print, but used copies are worth buying (imho) https://www.amazon.com/Compiler-Design-C-Prentice-Hall-software/dp/0131550454/ https://www.amazon.com/Compiler-Design-C-Prentice-Hall-softw...
- ksherlock 9y ago
  The Holub book (and source code) is available for download from the author - http://holub.com/compiler/ http://holub.com/compiler/
madhadron 9y ago
Structure and Interpretation of Computer Programs has a nice section going through building an interpreter and then a compiler. That might be enough to slake your interest.
mhh__ 9y ago
Modern compiler implementation in ML and Engineering a compiler are the two best all round books I've seen. The former also contains so useful information (not much, but useful) about implementation the runtime (e.g. Intro to garbage collection) Personally, I only find the backend (Optimization) interesting: Steven Muchnick's Advanced compiler design and implementation, is the only book that I know of that is mainly focused on optimization. It is literally a encyclopedia of optimizations.
ftomassetti 9y ago
I wrote an article on everything you need to create your own lanague, hope it helps https://tomassetti.me/resources-create-programming-languages/ https://tomassetti.me/resources-create-programming-languages...
sn9 9y ago
You can take Stanford's Compilers course from Lagunita: https://lagunita.stanford.edu/courses/Engineering/Compilers/Fall2014/about https://lagunita.stanford.edu/courses/Engineering/Compilers/...