14 ms·
Introducing MIR
- thomnottom 11y agoI assume this is not the X Window replacement from Canonical.
- steveklabnik 11y ago"mid-level IR"
- donatj 11y agoI thought before clicking they were going to announce that the X Window replacement was being rewritten in Rust, lol. They should change the name.
- steveklabnik 11y agoThis isn't really a user-facing feature, so name clashes aren't as important. Context should be more than enough here.
- kibwen 11y agoI don't think they're concerned overly much with the marketing presence of a compiler implementation detail. :P
- lolidaisuki 11y agoThen why are they marketing it?
- ZenoArrow 11y agoOne name is in all caps, the other is not, and it's easy to add another word to your website searches to get the one you want (e.g. MIR Rust, Mir Ubuntu).
- mtgx 11y agoNor the Russian space station.
- skocznymroczny 11y agoYeah, there seem to be a few names that are overused for completely different projects. Mir, Dash, Unity
- oconnor663 11y ago"Rust" :)
- kibwen 11y agoI'm incredibly psyched for MIR. It'll represent the first conceptual "leap" for the language since Rust 1.0, checking off (or at least unblocking) a ton of the improvements that we've had in mind for years now (the list in the OP is hardly comprehensive!). It's also interesting that Swift, another LLVM-based project, has also transitioned to using their own language-specific intermediate representation (SIL). I think it's fascinating that, despite the availability of a robust and mature pluggable language backend, two professional languages independently decided that they needed another layer of abstraction between AST and LLVM IR. Not sure if this should be interpreted as a bug report against LLVM IR, or if it possibly represents a future trend in compiler design. EDIT: I should also mention that MIR has been available on play.rust-lang.org for a while, if anyone would like to play with it in their browser: https://play.rust-lang.org/?gist=fee8ccf28bae2c89107d&version=nightly https://play.rust-lang.org/?gist=fee8ccf28bae2c89107d&versio...
- odbol 11y agoDoes Rust using LLVM mean you can write iOS apps in Rust?
- steveklabnik 11y agoYes, though you don't have access to the GUI toolkit stuff, so it's most suitable for stuff like games.
- steveklabnik 11y agoIt's SIL, not HIL: http://llvm.org/devmtg/2015-10/slides/GroffLattner-SILHighLevelIR.pdf http://llvm.org/devmtg/2015-10/slides/GroffLattner-SILHighLe...
- kibwen 11y agoFixed, and upvoted for the link. :)
- majewsky 11y agoNot necessarily a future trend. When I took a class in compiler construction last semester, the professor considered it industry standard to use multiple (AFAIR he said "up to five") IRs in the same compiler/linker toolchain.
- sievebrain 11y agoIt's interesting to compare and contrast this against bytecode from other platforms like CIL or JVM bytecode. They're both essentially a kind of compiler IR and they allow for incremental compilation, can be interpreted, dynamically manipulated, structed as basic blocks, etc. But MIR is represented using a textual form that still has scoping and which still uses Rust syntax. I can see the appeal. But I wonder how much it complicates working with MIR. You'd need a more complex parser, I guess.
- dikaiosune 11y agoIIUC, MIR has a textual representation, but the compiler works on a control flow graph in memory. I don't think MIR is written to disk and then re-parsed during compilation.
- avita1 11y agoMy guess is that in the future, source code -> MIR could be cached (on disk) so that whole-program optimization/linking could happen without compiling all of the source to IR.
- Sean1708 11y agoIf that were to happen I expect they would devise some sort of bytecode to use, probably with a 1=>1 mapping to the textual representation.
- steveklabnik 11y agoInside rustc, MIR is a data structure. The outputs you see here are a human-readable export of that, the compiler does not emit text and then re-parse. Projects in Rust, like miri, hook directly into the compiler and use the datastructures as well, they don't parse the output you see here: https://github.com/tsion/miri/blob/master/src/bin/miri.rs https://github.com/tsion/miri/blob/master/src/bin/miri.rs (The post has been amended to try to clear this up a bit)
- 11y ago
- merb 11y agoHopefully the IntelliJ Plugin of Rust will iterate faster and support more completions. Actually it would be awesome to edit Rust the same way than Scala/Java.
- chrisseaton 11y agoI don't think this new work would be useful for that. It's an internal compiler IR for programs in the process of being optimised and lowered, not a general format for representing Rust programs for user applications.
- jdmichal 11y agoSpecifically, based on the thread covering WebAssembly as an output target [0], rewinding the graph into higher control structures like loops may be non-trivial. [0] https://github.com/rust-lang/rust/issues/33205#issue-150980448 https://github.com/rust-lang/rust/issues/33205#issue-1509804...
- kevindqc 11y agoNot super familiar with Rust, but will this have any impact on debugging? I imagine LLVM is generating debug symbols from what it receives (which would be fairly representative of the original program). Now with this extra step, won't LLVM generate debugging symbols for the MIR, which doesn't really map well to the original code (ie: LLVM won't know all those gotos were originally a loop)?
- steveklabnik 11y agoMIR keeps track of where in the original source it was generated, so you still get good debugging output.
- kevindqc 11y agonice! thanks :)
- pcwalton 11y agoDebug information is propagated down from the HIR through the MIR and then down to LLVM IR, just as LLVM propagates debug info down through LLVM IR to SelectionDAG and then to MachineInstrs.
- cm3 11y agoWill this allows us to get TCO? I'm used to writing recursive functions and miss TCO all the time.
- pcwalton 11y agoIn theory it could allow for sibling call optimization by rewriting those functions into loops. It would not allow full unrestricted TCO--that would require both ABI work and thorny high-level design issues around destructors that may well not be solvable at all. However, in practice LLVM already has full support for sibling call optimization internally, so implementing such an optimization at the MIR level wouldn't really buy us anything over what we already have.
- cm3 11y agoThen maybe rustc ought to have diagnostics that warn if a recursive function won't be optimized fully, and then I can either avoid recursion or rewrite to fit into the optimizer, but having to appease a changing optimizer isn't what I'd call maintainable code, so a function attribute might make more sense.
- geofft 11y agoThere's a reserved and unimplemented keyword "become", which is like "return" except it requires that it can optimize the tail call (and is a compile-time error if it can't). See this comment: https://github.com/rust-lang/rfcs/issues/271#issuecomment-189386398 https://github.com/rust-lang/rfcs/issues/271#issuecomment-18... Nobody's tried implementing it recently, though.
- Ericson2314 11y agoHow about non-sibling TCO provided their are no drop obligations? I'll gladly write the boilerplate of manually calling `drop` to get the ball rolling here.
- dbaupp 11y ago
- kccqzy 11y agoThis really sounds like GHC's Core, a simplified, desugared, explicit intermediate language used during compilation.
- drostie 11y agoWhat I want to see in a language would be a bunch of syntactic sugar which corresponds to a MIR like Rust's by a well-specified reduction that is likely to be consistent from build to build. The reason that this could be really cool is that you could hypothetically hash these MIRs and store these hashes in a public database of open-source code. (You would have to include hashes of whatever functions they call/depend on, and you would have to solve foreign-function imports.) Then you would have this cool language where modules are simply saying "here's a mapping of names to hashes in the public database," with automatic recognition of "hey, these two functions you're importing from these two modules are actually the same function, I don't need to disambiguate them." If your dependency graph isn't a tree and one of the branches can't cope with a dependency-upgrade, that's fine too -- you upgrade the dependency on the one branch only. The point is, all of your dependency-semantics are now reduced to a really straightforward reasoning about unique names.
- kibwen 11y agoWhy would you prefer to hash an intermediate representation, rather than hashing the source?
- lsaferite 11y agoMaybe two different sources compile to the same IR?
- cwzwarich 11y agoIf the IR contains file and line number info, that seems pretty unlikely.
- drostie 11y agoRight, the point would be for the hash to strip out file info, line numbers, whitespace, comments, any stylistic choices which turn out to be equivalent "under-the-hood" in some straightforward way, and probably even the bound variable names (with de Bruijn indexes).
- haberman 11y agoI'm curious how this new architecture preserves safety invariants. The post says MIR contains operations not available in Rust because they are unsafe. Does this mean every MIR transformation needs to be safety-preserving? Or is safety-checking the last stage that runs directly before LLVM IR generation?
- pcwalton 11y agoMIR transformations do have to be safety-preserving, but this isn't any different from all of the LLVM IR-level optimizations.
- haberman 11y agoBut the MIR definition of "safety" must be more encompassing than LLVM IR, right? LLVM IR has no concept of a borrow. Where in the MIR pipeline does borrow checking happen?
- pcwalton 11y agoI believe that when it's implemented it will happen first before any optimization, in order to provide the best diagnostics possible and to avoid doing wasted optimization if there are errors. But Niko would really be able to answer this more definitively.
- deleted 11y ago[deleted]
- pc2g4d 11y agoThat concerned me a bit, too. Seems like potentially a source of safety problems?
- pcwalton 11y agoNot any worse than LLVM optimizations are, and there are way more of those.
- munificent 11y agoThis is some really fantastically well done exposition. I loved it.
- nv-vn 11y agoAwesome. This reminds me of the recent changes to the OCaml compiler (flambda). If it's anything similar, can't wait for the performance improvements from this (which can hopefully continue bringing Rust up to C speed).
- steveklabnik 11y agoIn general, if Rust is significantly slower than equivalent C, it's a bug. Please file them if you find them.
- nbaksalyar 11y agoThat's awesome work, thank you! I'm wondering if MIR will make it possible to replace the backend IR. For instance, to use WebAssembly instead of LLVM IR. Or is it still a better idea to base it off of LLVM?
- steveklabnik 11y agoYes, in theory, you could write a new backend that uses MIR. Miri, linked at the end of the post[1], is an example: it interprets MIR, rather than using LLVM to generate machine code. 1: https://github.com/tsion/miri https://github.com/tsion/miri
- eddyb 11y agoThere is discussion of a WebAssembly backend which lowers MIR directly, see https://github.com/rust-lang/rust/issues/33205 https://github.com/rust-lang/rust/issues/33205.
- skrebbel 11y agoI suspect that the MIR idea could (should?) become a pretty common concept for a large number of LLVM-based languages. the LLVM IR is very low-level, and in most modern languages there's a big gap between the programmer-friendly features they have and "the bare essentials that still capture the same ideas". I also suspect that the act of designing a MIR for a language is a fun one, should be pretty damn enlightening. It really forces you to focus on what's "core" about the language. In my own, limited, experience with language design and compilers, I've found that doing something like a MIR greatly simplifies other parts. Code written in MIR (in my case, it didn't even have a concrete syntax, just an abstract syntax tree) is full of duplication and is ridiculously explicit, but it's super easy to process in all kinds of ways. Just plainly guessing, but based on this blog post, if I'd write a Rust interpreter, I'd probably run it on MIR and not on plain Rust or LLVM IR.
- eddyb 11y agoYou are correct that a Rust interpreter would work great on MIR, which is why Scott Olson (@tsion) has been working on one: https://github.com/tsion/miri https://github.com/tsion/miri (check out the slides and the report).
- mushishi 11y agoWhat is the reason the MIR was not a part of the compiler right from the start?
- pcwalton 11y agoBecause (a) I resisted it due to (mistakenly) thinking it was not necessary; (b) the language was changing a lot and having a MIR to change would have made that harder. If we had had a MIR from Rust 0.1, it probably would have had to be completely overhauled by this point.
- perflexive 11y agoI'm surprised to see no mention in the article or on HN about how this might affect writing crypto code in Rust. Maybe it's a little tangential, but I'm dying for constant time operations. Avoiding branches, etc. in Rust doesn't mean LLVM won't add some as an optimization, which is frustrating to say the least. It would be awesome to be able to define a block - similar to "unsafe" - that tells the compiler to disable optimizations that could introduce non-constant time operations. When I started reading the article, I though maybe this new development would open the door to something like that, but it doesn't appear to be the case. There's some work to do constant time ops in rust, but it's very experimental and untrustworthy. :/
- steveklabnik 11y agoSince this is before the LLVM passes, it doesn't really change anything, and can't.
- perflexive 11y agoI have no experience with LLVM internals, but couldn't you split up the code and disable some of LLVM's optimizations on blocks defined by the programmer? Or do you have to pass the entire program at once to LLVM? Maybe the ability to disable optimizations only on certain functions is possible, idk, but I think it certainly would be nice.
- steveklabnik 11y agoI'm not knowledgeable about that, to be honest.
- lolidaisuki 11y agoWhat a horrible choice for a name. Canonical has had Mir for a while now. Choosing an unique name isn't that hard. Not everything needs to be 3 letters. https://launchpad.net/mir https://launchpad.net/mir This had me confused for a while: > Introducing MIR (rust-lang.org)
- spv 11y agoI am not sure if Julia already has something similiar to this ? Can anyone throw some light on this.
- spv 11y agoI am not sure if Julia already has something like this, because there was some work precompilation. Can someone throw some light on this ?
- ihnorton 11y agoYes, Julia uses a similar "lowered" form: http://docs.julialang.org/en/latest/devdocs/ast/#lowered-form http://docs.julialang.org/en/latest/devdocs/ast/#lowered-for... If you are interested in more details, there was a talk two years about the compiler design: https://www.youtube.com/watch?v=osdeT-tWjzk https://www.youtube.com/watch?v=osdeT-tWjzk (many type names and implementation details have changed since then, but conceptually it is still very applicable).
- monomaniar 11y agoevery rust topic, i vote.
- mirimir 11y agoMIR riffing on peace or world?
- sudeepj 11y agoWow! More than the actual concept (which is cool), I found the level of technical writing is of high calibre .. highly educating. Pleasure to read such articles.
- esarbe 11y agoI wonder; can someone explain to me how this compares to Scalas' TASTY? It seems to be on about the same layer, but I understand too little about compilers to be sure.