5 ms·
Are there any courses on creating a new language, interpreter, and compiler from scratch? Like not using LLVM or similar to generate intermediate representation
by satvikpendem 4y ago
Are there any courses on creating a new language, interpreter, and compiler from scratch? Like not using LLVM or similar to generate intermediate representation but to literally pick an ISA and, well, compile for it.
- bgar 4y agoCheck out https://mitpress.mit.edu/9780262047760/essentials-of-compilation/ https://mitpress.mit.edu/9780262047760/essentials-of-compila... Author has made complete draft PDFs for both the Racket and Python versions available here. https://wphomes.soic.indiana.edu/jsiek/ https://wphomes.soic.indiana.edu/jsiek/
- _the_inflator 4y agoThanks, highly appreciated. I tend to be biased towards what Jeremy Siek himself markets as "Proven in the classroom" when it comes to book authors in CS. Many book authors lack this experience and simply write for themselves, which is ok, but can result in bad didactics. Good teachers and authors from academia are invaluable.
- abecedarius 4y agoThat's a great point about experience teaching the topic. (I've written a few articles where I certainly wished for that experience.) I just wish textbooks didn't have their own bad tendencies: they have pablum as an attractor, because on average students just want to get through the class, not doing too much worse than average among the other students. Even without this problem, there's a more basic one: like with enterprise software, the decision to buy the book is not typically up to the user. "Will people actually want to read this on their own time?" is a strong driver of quality, even though it has pitfalls too.
- mrkeen 4y agoHave a look at https://keleshev.com/compiling-to-assembly-from-scratch/ https://keleshev.com/compiling-to-assembly-from-scratch/
- ly3xqhl8g9 4y agoPrecisely made for "creating a new language, interpreter, and compiler from scratch": http://craftinginterpreters.com http://craftinginterpreters.com
- satvikpendem 4y agoThis does not cover compilers. I know because I specifically asked munificent (the author) about this before and he mentioned some other books to look at.
- koonsolo 4y agoWhat do you mean exactly by "compilers"? Because the book features a chapter on how to compile source code to virtual machine byte code.
- sondr3 4y agoMost of the time when people talk about compiled languages/compilers, they mean languages that compile to native machine code, not languages that compile to a bytecode/run on a virtual machine.
- vsnf 4y agoIs that true? I think of C# and Java as compiled languages and they run on the CLR/JVM which are both virtual machines.
- sondr3 4y agoIt's just the nomenclature that I'm used to, compiled languages are so called because there is no VM, it's just machine code that is run while C# and Java still requires the JVM to do JIT compilation at runtime. But it's mostly splitting hairs, there is no agreed upon definition of what makes a languages compiled/interpreted.
- orthoxerox 4y ago
- cyber1 4y agohttps://compilerbook.com/ https://compilerbook.com/
- delta_p_delta_x 4y agoUPenn's CIS341[1] is pretty nice. It uses a bottom-up approach, where assignments start with a 64-bit integer-only x64 simulator, then goes up to compiling LLVM IR to x64, then compiling the lecturer's language ('Oat') into LLVM IR, and then optimisations like dead code elimination, register allocation, variable substitution, etc. [1]: https://www.seas.upenn.edu/~cis3410/current/ https://www.seas.upenn.edu/~cis3410/current/
- tinco 4y agoWhen I did my undergraduate degree, you could opt to do that for 1 extra point. Compiling for an ISA is not that much more difficult than compiling for a bytecode IR like that of Java or .Net which I imagine is the typical target of these courses.
- fancyfredbot 4y agoCheck out project Oberon by Niklaus Wirth. It involves building a compiler from scratch for an ISA designed from scratch and building an OS for it. You can even put the processor onto an FPGA and run the code you compiled on physical hardware you designed.
- AlphaSite 4y agoThe prior course involves implementing a code generating compiler with optimisation passes etc for a language.
- samsquire 4y agoI compiled a basic C program and then read the output of GCC -S to see the assembly. I used what I learned from GCC output Then I wrote a basic amd64 expression compiler with simple live ranges and linear register allocation and ANF (a normal form) for postorder AST traversal for code generation. It can only evaluate nested add and mul expressions. https://github.com/samsquire/compiler https://github.com/samsquire/compiler or on replit: https://replit.com/@Chronological/Compiler3#main.py https://replit.com/@Chronological/Compiler3#main.py Finding opcodes is a struggle but there is this website: https://www.felixcloutier.com/x86/ https://www.felixcloutier.com/x86/ and https://www.cs.uaf.edu/2016/fall/cs301/lecture/09_28_machinecode.html https://www.cs.uaf.edu/2016/fall/cs301/lecture/09_28_machine... (the table at the bottom is really useful) To learn code generation not targeting amd64, I wrote an imaginary assembly language and then wrote a switch based virtual machine and a compiler for it with a frontend that looks similar to javascript. https://github.com/samsquire/multiversion-concurrency-control https://github.com/samsquire/multiversion-concurrency-contro... in ProgramParser.java, LanguageInterpreter.java, and LanguageInterpreterRunner.java for where main() is.
- JonChesterfield 4y agoIntel's documentation covers encoding. I remember it being relatively heavy going to decipher, been a few years since I've worked on x64 though.
- noelwelsh 4y agoVarious books do this. Essentials of Compilation is a recent one: https://github.com/IUCompilerCourse/Essentials-of-Compilation https://github.com/IUCompilerCourse/Essentials-of-Compilatio... You can find a PDF and the associated course with a very minimal amount of following links. If you are thinking of making your own language, it's also good to learn something about programming language theory, if you don't already. Many languages make mistakes that have been solved 25+ years ago. PLAI is good for that: https://www.plai.org/ https://www.plai.org/
- agentultra 4y agoThe https://www.nand2tetris.org/ https://www.nand2tetris.org/ is pretty neat. The compiler in part 2 isn’t very advanced (it doesn’t cover much in the space of optimization or the state of the art) but taken as a whole you end up designing the chip, ISA, VM, language and compiler, so it’s pretty good for that aspect.