4 ms·

Resources for Amateur Compiler Writers

100 points by andrewchambers 11y ago

vu3rdd 11y ago
That's a great list, all in one place! Thanks to the author of that web page.
- andrewchambers 11y ago
  check out some of the other pages under http://c9x.me/ http://c9x.me/. He has some cool tutorials and other things.
sklogic 11y ago
A good start. But a lot of really important and approachable stuff is missing from the list. On SSA, for example, the best introduction is S. Pop thesis: http://www.cri.ensmp.fr/people/pop/papers/2006-12-thesis.pdf http://www.cri.ensmp.fr/people/pop/papers/2006-12-thesis.pdf
- andrewchambers 11y ago
  Please post as many good resources as you can, i'm sure they can be added.
  - sklogic 11y ago
    Far from a comprehensive list, just the stuff I could remember straight away: Nanopass: http://www.cs.indiana.edu/~dyb/pubs/nano-jfp.pdf http://www.cs.indiana.edu/~dyb/pubs/nano-jfp.pdf http://andykeep.com/pubs/np-preprint.pdf http://andykeep.com/pubs/np-preprint.pdf An extremely useful technique, somewhat related to what Sebastian Pop was doing with Prolog in his thesis: http://people.csail.mit.edu/mcarbin/papers/aplas05.pdf http://people.csail.mit.edu/mcarbin/papers/aplas05.pdf also http://www.cs.cmu.edu/~aldrich/courses/654/tools/bierhoff-bddbddb-05.pdf http://www.cs.cmu.edu/~aldrich/courses/654/tools/bierhoff-bd... All such things may be inappropriate for the top notch commercial compilers due to a limited performance, but are great for DSLs and for an amateur work, because they're so much easier than an ad hoc hand-coded approach. On LLVM specifically, can be used as a nice introduction into how backend works: http://eli.thegreenplace.net/2012/11/24/life-of-an-instruction-in-llvm http://eli.thegreenplace.net/2012/11/24/life-of-an-instructi...
userbinator 11y ago
Also worth studying some actual tiny compilers, just to get an idea that not all compilers have to be extremely complex: https://news.ycombinator.com/item?id=9125912 https://news.ycombinator.com/item?id=9125912 https://news.ycombinator.com/item?id=8576068 https://news.ycombinator.com/item?id=8576068 https://news.ycombinator.com/item?id=8558822 https://news.ycombinator.com/item?id=8558822 http://bellard.org/otcc/ http://bellard.org/otcc/
- sklogic 11y ago
  Tiny does not mean "simple", and this is very true in compilers. There are ways of making a compiler trivial and extremely comprehensible, but it involves somewhat lengthy code. A tiny compiler which fuses multiple transforms into one may be much harder to comprehend than a larger compiler made of a long sequence of much simpler passes. So your suggestion may be quite intimidating to the beginners. I'd rather recommend to start with the Racket-style language extensions and DSLs first.
  - nils-m-holm 11y ago
    http://t3x.org/subc/ http://t3x.org/subc/ is tiny and simple. But I'm supposed to say that!
    - sitkack 11y ago
      If you ported one of your excellent compiler books to Rust, I would buy it again, twice. Nils Holm, master compiler educator http://www.lulu.com/shop/search.ep?contributorId=740596 http://www.lulu.com/shop/search.ep?contributorId=740596
    - sklogic 11y ago
      It is nice and simple indeed but does not really show anything instructive in terms of backend, which is the focus of this thread. And once you go into SSA and proper isel, there is no way to do it both easily and compactly at the same time, passes must be split for simplicity or fused into a mess for a compactness. Of course, I am only talking about a number of lines of code, besides this you can keep your compiler totally trivial for any degree of optimisations complexity.
melling 11y ago
I've got a small collection of links on compilers: https://github.com/melling/ComputerLanguages/blob/master/compilers.org https://github.com/melling/ComputerLanguages/blob/master/com...
JustSomeNobody 11y ago
Also worth looking int pl/0 and the p-code machine. Edit: Also, Thomas W Christopher used to have some pdfs on the tools of computing web page that dealt with compilers. Had an interesting one about using matrices to computer first and follow sets. I wish I could find those again.
piinbinary 11y ago
Does anyone know of a good starting point for someone who wants to build a language targeting the JVM? (The JVM spec isn't an easy read)
- cottonseed 11y ago
  Try ASM: http://asm.ow2.org/ http://asm.ow2.org/ It is lightweight, well-documented and easy to use. It has an IntelliJ plugin that wills how you the sequence of API calls you'd need to make to create a given compiled Java class. I still recommend reading the spec. You'll need to understand the JVM execution model and the meaning of the byte codes to successfully target it.
- jerven 11y ago
  The truffle tutorial is the easiest way to get started. Will give you very good performance.
qznc 11y ago
An old tutorial: http://compilers.iecc.com/crenshaw/ http://compilers.iecc.com/crenshaw/ If you want to think about memory models, then I recently wrote an introduction: http://beza1e1.tuxen.de/articles/memory_models.html http://beza1e1.tuxen.de/articles/memory_models.html

100 points by andrewchambers 11y ago

vu3rdd 11y ago

That's a great list, all in one place! Thanks to the author of that web page.

andrewchambers 11y ago
check out some of the other pages under http://c9x.me/ http://c9x.me/. He has some cool tutorials and other things.

sklogic 11y ago

A good start. But a lot of really important and approachable stuff is missing from the list. On SSA, for example, the best introduction is S. Pop thesis: http://www.cri.ensmp.fr/people/pop/papers/2006-12-thesis.pdf http://www.cri.ensmp.fr/people/pop/papers/2006-12-thesis.pdf

andrewchambers 11y ago
Please post as many good resources as you can, i'm sure they can be added.
- sklogic 11y ago
  Far from a comprehensive list, just the stuff I could remember straight away: Nanopass: http://www.cs.indiana.edu/~dyb/pubs/nano-jfp.pdf http://www.cs.indiana.edu/~dyb/pubs/nano-jfp.pdf http://andykeep.com/pubs/np-preprint.pdf http://andykeep.com/pubs/np-preprint.pdf An extremely useful technique, somewhat related to what Sebastian Pop was doing with Prolog in his thesis: http://people.csail.mit.edu/mcarbin/papers/aplas05.pdf http://people.csail.mit.edu/mcarbin/papers/aplas05.pdf also http://www.cs.cmu.edu/~aldrich/courses/654/tools/bierhoff-bddbddb-05.pdf http://www.cs.cmu.edu/~aldrich/courses/654/tools/bierhoff-bd... All such things may be inappropriate for the top notch commercial compilers due to a limited performance, but are great for DSLs and for an amateur work, because they're so much easier than an ad hoc hand-coded approach. On LLVM specifically, can be used as a nice introduction into how backend works: http://eli.thegreenplace.net/2012/11/24/life-of-an-instruction-in-llvm http://eli.thegreenplace.net/2012/11/24/life-of-an-instructi...

userbinator 11y ago

Also worth studying some actual tiny compilers, just to get an idea that not all compilers have to be extremely complex: https://news.ycombinator.com/item?id=9125912 https://news.ycombinator.com/item?id=9125912 https://news.ycombinator.com/item?id=8576068 https://news.ycombinator.com/item?id=8576068 https://news.ycombinator.com/item?id=8558822 https://news.ycombinator.com/item?id=8558822 http://bellard.org/otcc/ http://bellard.org/otcc/

sklogic 11y ago
Tiny does not mean "simple", and this is very true in compilers. There are ways of making a compiler trivial and extremely comprehensible, but it involves somewhat lengthy code. A tiny compiler which fuses multiple transforms into one may be much harder to comprehend than a larger compiler made of a long sequence of much simpler passes. So your suggestion may be quite intimidating to the beginners. I'd rather recommend to start with the Racket-style language extensions and DSLs first.
- nils-m-holm 11y ago
  http://t3x.org/subc/ http://t3x.org/subc/ is tiny and simple. But I'm supposed to say that!
  - sitkack 11y ago
    If you ported one of your excellent compiler books to Rust, I would buy it again, twice. Nils Holm, master compiler educator http://www.lulu.com/shop/search.ep?contributorId=740596 http://www.lulu.com/shop/search.ep?contributorId=740596
  - sklogic 11y ago
    It is nice and simple indeed but does not really show anything instructive in terms of backend, which is the focus of this thread. And once you go into SSA and proper isel, there is no way to do it both easily and compactly at the same time, passes must be split for simplicity or fused into a mess for a compactness. Of course, I am only talking about a number of lines of code, besides this you can keep your compiler totally trivial for any degree of optimisations complexity.

melling 11y ago

I've got a small collection of links on compilers: https://github.com/melling/ComputerLanguages/blob/master/compilers.org https://github.com/melling/ComputerLanguages/blob/master/com...

JustSomeNobody 11y ago

Also worth looking int pl/0 and the p-code machine. Edit: Also, Thomas W Christopher used to have some pdfs on the tools of computing web page that dealt with compilers. Had an interesting one about using matrices to computer first and follow sets. I wish I could find those again.

piinbinary 11y ago

Does anyone know of a good starting point for someone who wants to build a language targeting the JVM? (The JVM spec isn't an easy read)

cottonseed 11y ago
Try ASM: http://asm.ow2.org/ http://asm.ow2.org/ It is lightweight, well-documented and easy to use. It has an IntelliJ plugin that wills how you the sequence of API calls you'd need to make to create a given compiled Java class. I still recommend reading the spec. You'll need to understand the JVM execution model and the meaning of the byte codes to successfully target it.
jerven 11y ago
The truffle tutorial is the easiest way to get started. Will give you very good performance.

qznc 11y ago

An old tutorial: http://compilers.iecc.com/crenshaw/ http://compilers.iecc.com/crenshaw/ If you want to think about memory models, then I recently wrote an introduction: http://beza1e1.tuxen.de/articles/memory_models.html http://beza1e1.tuxen.de/articles/memory_models.html