8 ms·
CS 6120: Advanced Compilers: The Self-Guided Online Course
- tripdout 3y agoI favorited this last year and still didn't get around to it. But from what I saw of one of the videos, it looks really good, super interesting stuff.
- jheriko 3y ago[flagged]
- simonebrunozzi 3y agoCompilers is one of these field that didn't evolve much for ~30 years. I used to teach a course at Perugia University back in 2004-2006, and the material I could use was easily 15-20-25 years old, no sweat. Things seem to have changed recently.
- grumpyprole 3y agoProbably because most programming languages haven't changed, imperative, loops, pervasive side-effects etc. Writing a compiler for a pure functional language would certainly need new material.
- dunefox 3y ago> Writing a compiler for a pure functional language would certainly need new material. I mean, Haskell is also over 30 years old at this point.
- grumpyprole 3y agoYes, but it's changed a lot in 30 years.
- pjmlp 3y agoOn top of sibbling comment, Simon Peyton Jones seminal paper is from 1987.
- grumpyprole 3y agoHe has spent the last 30 years since publishing more papers on compiling functional languages.
- pjmlp 3y agoMostly related to adding new features to Haskell and GHC, not on compiling functional languages in general. Now functional logic languages like Verse, that is definitely new material.
- grumpyprole 3y agoThat's selling him very short. I would say most of his papers are applicable to functional programming in general and not just Haskell.
- pjmlp 3y agoOnly for those that understand functional programming as whatever Haskell does.
- snnn 3y agoNo, I think it is because most problems are too hard to solve. Think a simplified case: build a compiler for evaluating arithmetic expressions like "a*3+b" and make the generated code as efficient as possible. It is an NP-hard problem!
- pfdietz 3y agoThere's been a substantial change in compiler testing since then, with high volume randomized testing coming into vogue.
- looofooo0 3y agoRecent papers with llm Fuzzing appearing e.g. fuzz4all
- wslh 3y agoWe are working on projects related with cybersecurity and compilers. A reference we look at is [1] and [2]. I think we can publish the results in the coming months. [1] https://github.com/rohanpadhye/jqf/wiki/Fuzzing-a-Compiler https://github.com/rohanpadhye/jqf/wiki/Fuzzing-a-Compiler [2] https://news.ycombinator.com/item?id=36373410 https://news.ycombinator.com/item?id=36373410
- sjrd 3y agoI disagree. The landscape has significantly evolved into incremental compilation techniques in the past decade. While theoretical developments are mostly confined into parsers, practical implementations do it across the entire pipeline, all the way to whole-program optimizations and code generation.
- mr_foo 3y agoI'm not well versed in compiler design or implementation. But it seems to me that the target of compilers (machine code, assembly, etc.) has changed dramatically over time. The number of instructions and options on each successive processor generation seems to offer new optimizations or combinations for compilers.
- deleted 3y ago[deleted]
- jcranmer 3y agoThat's just not true. Just going by the new stuff that you could have taught back in 2004-2006 that you likely didn't, there's SSA construction, SLP vectorization, automatic peephole superoptimization, and that's just things I could name off the top of my head (and were papers for my qualifying exam :-P). What hasn't changed is the compiler textbooks, which tend to have way too heavy a focus on how to build a parser generator and almost nothing on how you actually architect a compiler, let alone designing modern computer architecture. But this is a gripe which has been around for decades.
- GrumpySloth 3y agoNitpick: SSA is late 80s, early 90s tech.
- jcranmer 3y agoI'm dating SSA from the "Efficiently computing static single assignment form and the control dependence graph" paper, which came out in October 1991; my understanding is that SSA wasn't heavily used before this technique came out.
- deleted 3y ago[deleted]
- riedel 3y agoI looks like this is still basically most of the stuff covered by the normal compiler construction course I attended held by Gerhard Goos 20 years ago. It links some newer papers, so I might have a look. I liked the book by Steven Muchnick 'Advanced Compiler Design and Implementation' . I have to admit that after 18 years not having looked at compiler source code, I feel not up to speed particularly with a lot of profiling and path based optimisations. Also I guess some more advanced SIMD stuff must be out there looking at all the ML.
- pjmlp 3y agoGreat to see that also have "A Unified Theory of Garbage Collection", at least these students will get the right picture on RC versus tracing GC.
- Delta231 3y agoI wonder if there's a predecessor course to this
- ayhanfuat 3y agoI am following Stanford's Compilers course and I like it: https://www.edx.org/learn/computer-science/stanford-university-compilers https://www.edx.org/learn/computer-science/stanford-universi...
- juank1809 3y agoI struggled a lot with this :(, especially the code assignments. I think I'm going to give it another try soon.
- arh68 3y agoLike CS 4120 Introduction to Compilers ? [1] https://www.cs.cornell.edu/courses/cs4120/2022sp/ https://www.cs.cornell.edu/courses/cs4120/2022sp/
- dunefox 3y ago> CS 6120 is a PhD-level Cornell CS course by Adrian Sampson on programming language implementation. So, is it very advanced then? I don't think I'm at PhD level when it comes to CS.
- fear91 3y agoThere doesn't seem to be a good in-depth academic resource for advanced compiler optimization. I've searched a lot and all the courses I found were introductory, the actual interesting techniques require diving deep into the source code of popular OSS compilers. I found that quite surprising.
- deleted 3y ago[deleted]
- freilanzer 3y agoIt seems to me that most academic courses overfit massively on parsing, while only teaching the rest superficially.
- jokoon 3y agoIf you want to build a parser, I recommend Lexy, it's a parser combinator. It's a bit difficult to use because it makes extensive use of templates.
- DinaCoder99 3y agoThe C++ requirement seems to place this dead in the water. In 2024 there is no shortage of better runtimes.
- mgaunard 3y agoWhile I don't know anything about that project, your comment sounds a bit unwarrantingly anti-C++. Most compilers and operating systems are built in C++ nowadays, it's the industry standard for any systems-level programming.
- IshKebab 3y agoYeah but only because Rust (and arguably Zig) didn't exist when those compilers and operating systems were written. C++ is the industry standard for existing software but it would be dubious at best to start a new project in C++. Especially something that has essentially no dependencies like a compiler. GUIs and games, sure.
- mgaunard 3y agoha, you're a Rust zealot; that explains it. Believing that all new software should be written in Rust rather than C++, and that this situation is inevitable anyway, is one of the core tenets of this church.
- IshKebab 3y ago> you're a Rust zealot; that explains it. I bet you were one of those people that dismissed anyone that liked the iPhone as being an Apple fanboy. You know people really like Rust because it's good right? I'm curious what other reason you think people have for promoting it? Even the White House says you should use Rust instead of C++. Are they Rust zealots?
- rrgok 3y agoI'm glad this exists. Now I can follow a guided course into advanced topics at my own pace. I've always wanted to have a career as a Compiler Engineer, but sadly, where I live doesn't offer much in terms of education and job opportunities. Looking at the USA, the job market is overwhelmingly competitive, and I honestly don't know how to get into it. The only experience I have is a course I did during my Bachelor's, and I loved every bit of it.
- JonChesterfield 3y agoIt's one of the fields with a brutal learning curve that a lot of people don't make it over. My best guess is it's the difficulty of writing code that manipulates other code, usually with quite different semantics and behavioural goals. There's also a lot of folk wisdom and general noise in the field. That makes compiler teams especially keen to hire people who have already spent ages building compilers. However that obviously has a bootstrapping problem so the larger teams also hire graduates who look like they might make it over that curve. That's roughly how I got into it. If you're experienced in general but not with compilers, the obvious play is to join a company doing whatever you're used to which happens to also have a compiler team, which roughly maps onto "largish software company", and aim to move laterally.
- rrgok 3y agoThank you for your input. That's a great idea.
- fancyfredbot 3y agoCompilers seem a great target for AI. Whole program optimisation using reinforcement learning could be massive, there's plenty of data, and collecting more data is relatively cheap. The course touches on superoptimisers but these don't really use AI. I think a change is coming in this space
- layer8 3y agoHow would you prove the resulting program is functionally still the same?
- danybittel 3y agoYou don't have to go that far. A compiler contains a lot of heuristics, I could see ai being useful for that.
- sabas123 3y agoThose heuristics are about "when" to apply certain transformations in a situation when two options are already proven equivalent. That is different from transforming correct code into possibly incorrect code.
- mgaunard 3y agoAI could be used to choose which of the valid transformations to apply and in which order. Current compilers do not guarantee that re-running the optimization passes a second time is idempotent. Every pass just does some "useful" transformations, it's best-effort.
- fancyfredbot 3y agoAs others have commented, it would be more like deciding which optimisation passes to run and in which order. Things like loop unrolling and inlining are not always helpful. Things like polyhedral optimization can be very slow. Running every optimization pass every time it might help is certainly far too slow.
- bregma 3y ago
- mnafees 3y agoThis course single-handedly became the reason for me to clear interviews at all the compiler engineering teams of MANGA (Meta, Apple, NVIDIA, Google, Amazon) companies when I was searching for my first full-time job while completing my Bachelors. My University's (University of Waterloo) curriculum around low-level computing systems was already par excellent, and of'course I was also contributing to open source compiler projects. But this course really helped me answer some of the toughest questions during my interviews!
- kettleballroll 3y ago> par excellent It's "par excellence", and it feels weird in that sentence structure anyways. A better way to put it would've been "my university has the low level computing systems curriculum par excellence".
- zarathustreal 3y ago[flagged]
- jeffreygoesto 3y agoWhy the bitter ad hominem? What or who hurt you that you can't let people like what they want? Bridging the hardware-software gap is hard, regardless of you appreciating it or not.
- zarathustreal 3y agoGiven that your comment has nothing to do with what I said, I’m assuming you misunderstood or are replying to the wrong one? “Weird comment” isn’t attacking the person (ad hominem) for who they are it’s attacking the action they chose to take. Likewise I don’t see how you took “let them have this one” as not letting people like what they want (granted, the GP comment wasn’t about liking something, it was a failed attempt to flex prior knowledge of this course, interviews at prestigious companies, and knowledge of esoteric language so I’m not entirely sure what you’re even referring to.) My comment didn’t mention “the hardware-software gap” or give any opinion on whatever that is so again..no clue what you’re on about
- mgaunard 3y agoBefore writing compilers, I think it's best to understand computer architectures, and what needs to be generated by the compiler to yield the most efficient machine code. Unfortunately, in my experience, computer architecture and even systems programming are domains that schools/universities appear to systematically increasingly de-prioritize, presumably because it is seen as too technical. That knowledge however is instrumental in landing some of the best jobs in the industry.
- zarathustreal 3y agoIn my experience, no one tends to see it as “too technical.” The typical (and imo obvious, logical) reason not to emphasize low-level details is because they’re implementation-specific, immediately outdated, and not particularly generalizable. As an aside, I also tend not to get along with the “systems programmer” types because they tend to make knowing these kinds of extremely specific factoids their entire professional personality. You end up with people that can write assembly but have no idea what a functor is.
- stevesimmons 3y agoExcellent final sentence!
- IshKebab 3y agoThere are things that are generally true across almost all architectures and have been for a long time. Like cache lines, cache hierarchies, false sharing, alignment, branch prediction, pipelining, etc. But I agree it isn't really necessary to write a compiler. Compilers tend to be pretty much the worst case from a microarchitectural point of view anyway - full of trees of small objects etc.
- mgaunard 3y agoI don't think they are. So long as you were not trained before the 70s, any knowledge there is still very relevant today. ISAs are still very similar, and so is the relative performance of the various operations. Registers, caches, pipelines, superscalar execution... it's all still mostly the same, and still drives what the codegen should look like. And yet you have so many people that don't know the considerations of what makes a simple loop fast or not.
- kungfupawnda 3y agodoes this course come with the labs? if so how can we test that our compiler is working?
- dfgdfg34545456 3y agoIs there a course like this but for pure functional languages?
- throwaway17_17 3y agoTL;DR — Nope, but here is a list of what such a course would use for reading material. As referenced in another comment, Simon Peyton Jones has a 1987 book on compiling functional languages, The Implementation of Functional Programming Languages. Follow that up with some of the referenced papers contained therein and you will have a base level to start at. Following the FP academic boom of the 80’s I would look to the following (in no particular order): papers of Stephanie Weirich, Simon Peyton Jones, Simon Marlow, Odersky, Philip Wadler and other authors you would find being referenced in their papers; the webpages for the Koka language, Idris, the Granule project, Haskell, and Ocaml all have references/publication sections that will contain a wealth of material. It is kind of a shame there is not a more ‘one-stop-shop’ for functional language compiling, but the research is so broad it would be hard to condense. You aren’t going to go wrong starting from SPJ’s book as the basis. Jones has a talk on YouTube about ‘Fitting Haskell into Nine Terms’ which is based on a paper that discusses the compilation strategy for GHC regarding the push to keep the internal System Fc language small and clean that is a good watch, along with the Jones talk on Compiling without Continuations also on YouTube and gives more internal views of GHC. Sorry the list is a bit scattershot, if you need more specifics I will certainly try to find a more narrow selection.
- JonChesterfield 3y agoSSA form is functional programming. As in the paper, but also it's quite literally true. The really gnarly part of compiling languages is dealing with load/store. If you translate a functional language to SSA form, what you get is an IR which doesn't have load/store in it. I.e. that's easy mode. I'd suggest compile to SSA with implicit memory semantics, then somewhat later splice in the part of the language runtime that deals with memory management and now you have an imperative SSA form, continue as usual. That is, they're the same problem really.
- terrelln 3y agoI went through this course online a few summers ago, and learned a ton. I highly recommend it! It was very engaging to put up PRs for small issues in the Bril IR, and work with the professor to fix them.