10 ms·
I'm incredibly psyched for MIR. It'll represent the first conceptual "leap" for the language since Rust 1.0, checking off (or at least unblocking) a ton of the
by kibwen 11y ago
I'm incredibly psyched for MIR. It'll represent the first conceptual "leap" for the language since Rust 1.0, checking off (or at least unblocking) a ton of the improvements that we've had in mind for years now (the list in the OP is hardly comprehensive!).
It's also interesting that Swift, another LLVM-based project, has also transitioned to using their own language-specific intermediate representation (SIL). I think it's fascinating that, despite the availability of a robust and mature pluggable language backend, two professional languages independently decided that they needed another layer of abstraction between AST and LLVM IR. Not sure if this should be interpreted as a bug report against LLVM IR, or if it possibly represents a future trend in compiler design.
EDIT: I should also mention that MIR has been available on play.rust-lang.org for a while, if anyone would like to play with it in their browser: https://play.rust-lang.org/?gist=fee8ccf28bae2c89107d&version=nightly https://play.rust-lang.org/?gist=fee8ccf28bae2c89107d&versio...
- odbol 11y agoDoes Rust using LLVM mean you can write iOS apps in Rust?
- steveklabnik 11y agoYes, though you don't have access to the GUI toolkit stuff, so it's most suitable for stuff like games.
- steveklabnik 11y agoIt's SIL, not HIL: http://llvm.org/devmtg/2015-10/slides/GroffLattner-SILHighLevelIR.pdf http://llvm.org/devmtg/2015-10/slides/GroffLattner-SILHighLe...
- kibwen 11y agoFixed, and upvoted for the link. :)
- majewsky 11y agoNot necessarily a future trend. When I took a class in compiler construction last semester, the professor considered it industry standard to use multiple (AFAIR he said "up to five") IRs in the same compiler/linker toolchain.
- kibwen 11y agoIf we're looking at the full toolchain, then LLVM itself does already have either two or three internal IRs, and god only knows how modern linkers work in practice.
- Someone 11y agoAlso, WebKit moved from LLVM to B3 (https://webkit.org/blog/5852/introducing-the-b3-jit-compiler/ https://webkit.org/blog/5852/introducing-the-b3-jit-compiler...).
- kibwen 11y agoI believe that's a bit different, as Webkit entirely replaced LLVM with a new backend, whereas here Rust is still ultimately emitting LLVM IR, just with a new well-defined step in-between.
- nostrademons 11y agoIt's been pretty common for compilers to require 3 different intermediate representations (parse tree, MIR, and LIR). My copy of Advanced Compiler Design and Optimization [1] (over 10 years old at this point) lists the 3-IR architecture on page 8. It references Sun's SPARC compilers, DEC's Alpha compilers, Intel's x86 compilers, and SGI's MIPS compilers as implementations that use it. What has changed are the boundaries. Traditionally, semantic analysis (including typechecking) operated on the parse tree, which resembled the human-written language extensively. Optimization operated on MIR, then it'd be lowered to an architecture-specific LIR via register allocation and instruction selection, and another round of optimization (instruction scheduling, etc.) would be applied. The purpose of all of this was to provide multiple language front-ends on top of a single compiler. In the 80s and 90s, compilers were often written by the hardware vendor, so they would tightly optimize the MIR and LIR for their own architecture, and use the parse-tree => MIR lowering to support multiple surface languages. LLVM chose a LIR that's closer to what many compilers had been using as MIR, and then moved the optimization passes into the LIR, and hid the architecture-specific backends within the LLVM project itself, out of the eyes of language designers. It could do this because of open-source: with a shared body of compiler code owned by everyone and gradual consolidation of the hardware market, it became easier to contribute your backend to LLVM than to maintain your own compiler stack and fight for adoption. (GCC actually had a fairly similar architecture first, but the GCC IR was very difficult to comprehend if you weren't a GCC maintainer, which made it impractical as a compilation target for outside projects. They generated C code instead and let GCC compile it.) That in turn made it much easier to write a compiler and experiment with language design, since you only had to figure out how to translate your language to LLVM's IR rather than work out the details of scheduling and register allocation. That, in turn, allowed greater complexity in language features: Swift and Rust have language features that go beyond what cutting-edge research was <10 years ago, and do so in a production language that you can use now. And so it's not surprising that they're now re-introducing a MIR to manage the additional complexity introduced by the new language features. [1] http://www.amazon.com/Advanced-Compiler-Design-Implementation-Muchnick/dp/1558603204 http://www.amazon.com/Advanced-Compiler-Design-Implementatio...
- infogulch 11y agoI tried to upvote you, but I fat-finger downvoted you instead. Sorry. :(
- shawn-butler 11y agoThere is a long tradition of type-directed compilation. Haskell compilation system refers to it as the simplifier or "middle end". The parsing and type-checking produces a IR known as 'Core'. The middle-end optimizes it as a series of source-to-source transformations on Core code. The back end of the compiler transforms Core code into an internal representation of a C dialect called C--, via yet another intermediate language STG (short for "Spineless Tagless G-machine"). Finally, the C-- code output can either be printed as C code for compilation with GCC, converted directly into native binary, or converted to LLVM virtual machine code for compilation with LLVM. Any output is linked against the Haskell runtime. The innovation would come in creating the optimizations in the middle-end that could not be possible in the LLVM backend. Proof is in the pudding. Introducing another layer without any meaningful results is how projects bloat (NIH syndrome). There is a LLVM backend that implements most if not all of the GHC "middle-end" optimizations[0] and should provide you with details/context you are seeking. [0] http://blog.llvm.org/2010/05/glasgow-haskell-compiler-and-llvm.html http://blog.llvm.org/2010/05/glasgow-haskell-compiler-and-ll...
- Tehnix 11y ago>The innovation would come in creating the optimizations in the middle-end that could not be possible in the LLVM backend >There is a LLVM backend that implements most if not all of the GHC "middle-end" optimizations I'm not sure I follow, are you saying that the optimisations are possible or not in LLVM? I don't think the LLVM optimisation passes can make guarantees such as is needed for (say) the rewrite rules that rely heavily on purity.
- shawn-butler 11y agoPlease read the link I provided and search the page for "Problems with the backend" ...There is one major issue remaining with the LLVM backend that I currently know of, its inability to implement a optimisation used by GHC called 'TABLES_NEXT_TO_CODE' (TNTC)...
- aseipp 11y ago
- hacker_9 11y agoIt's not a bug, and the answer is provided in the link: "Faster execution time. You may have noticed that in the new compiler pipeline, optimization appears twice. That’s no accident: previously, the compiler relied solely on LLVM to perform optimizations, but with MIR, we can do some Rust-specific optimizations before ever hitting LLVM – or, for that matter, before monomorphizing code. Rust’s rich type system should provide fertile ground for going beyond LLVM’s optimizations."
- kibwen 11y agoThere's still an implicit question there, concerning why one would choose to develop a new IR rather than enhancing LLVM (the latter of which the Rust devs (and especially the Swift devs) have had years of experience doing). Though I'm not trying to suggest that Rust made the wrong decision or is suffering from NIH, because MIR is great, and there's no reason to suspect that shoving absolutely everything down into LLVM is necessarily a tractable solution. I'm just curious which Rust-specific and Swift-specific optimizations these IRs enable above and beyond LLVM IR's capabilities, and what that says about each source language in turn (especially if we regard LLVM IR as a proxy for C, being LLVM's original target).
- rayiner 11y agoIt's easiest to do optimizations on an IR that has the appropriate level of granularity. If you're trying to optimize address calculations, it's easier to work with something like LLVM's getelementptr, which represents an address calculation as one IR node, than a C-level IR (where the address calculation would be implicit in an array or structure field access), or an ASM-level IR (where the address calculation might be broken up over several different instructions). LLVM is pretty low-level. It's a bit lower-level than C and has a very simple type system. It has no idea about unique pointers versus references, has no concept of borrowing, etc. I bet you could encode the necessary information for performing say borrow-checking into LLVM IR, but working with that would not necessarily be pleasant. A good example of this is ARC optimization in Swift, which gets rid of unnecessary reference counting operations. That optimization operates on SIL, which has special instructions for reference counting. You could represent those instructions as magic intrinsics in LLVM IR, but you have no way of imposing any structural restrictions on how those intrinsics are used. Moreover, any pass which operated on LLVM IR to optimize uses of those intrinsics, would have to make deep assumptions about the semantics of those operations. Those assumptions, being Swift-specific, would mean the pass could not be used by other languages, which removes a lot of the benefit of making it an LLVM pass in the first place.
- eloisius 11y agoI can't wait to see improved borrow checking! I'm not sure it'll fix this specific case, but I often find myself needing to introduce a scope just to hint the borrow checker into doing the right thing. For example: fn handle_message(&mut self, find_node_query: messages::FindNodeQuery) { let origin = node::UdpNode::deserialize(find_node_query.get_origin()); let target = Address::from_str(find_node_query.get_target()); { // Borrow self.routing_table first time let nodes: Vec<&Box<node::Node>> = self.routing_table.nearest_to(&target); let response = Message::Response( self.transaction_ids.generate(), Response::FindNode(&self.self_node, nodes)); origin.send(response.serialize()); } // Done borrowing self.routing_table here // Borrow self.routing_table again self.routing_table.insert(origin); } edit: upon digging into the details, it looks like it may: https://github.com/rust-lang/rfcs/blob/db66dd2039b10b759084079827d85aba1c26256c/active/0000-seme-regions.md https://github.com/rust-lang/rfcs/blob/db66dd2039b10b7590840...
- steveklabnik 11y agoYes, this is "non-lexical borrows"/"SEME regions", and is exactly what MIR will enable. Or at least, is one of the things :)
- Shoop 11y agoCould we see non-lexical borrows within the year?
- deleted 11y ago[deleted]
- steveklabnik 11y agoI'm not sure I can say for sure when they'll be ready, as MIR isn't 100% done, and they're built on top, so it's tough. There's also the difference between "lands in nightly" and "lands in stable", which could be some time for a feature so large.
- 11y ago
- sklogic 11y agoThere is nothing surprising in this fact. I always argued that even a very simple language needs not one, not two, but dozens of consequent IRs. Why? See the recently opensourced ChezScheme.
- takeda 11y agoThis is a common approach in compilers. It makes things more manageable. Each of the languages is simpler but has more constraints until you reach to the constraints that machine provides. Here is example of compiler construction class: http://web.cs.ucla.edu/~palsberg/course/cs132/project.html http://web.cs.ucla.edu/~palsberg/course/cs132/project.html The mid languages were named after character in Winnie the Pooh, you can also see how factorial example looked in each stage.
- DannyBee 11y ago" Not sure if this should be interpreted as a bug report against LLVM IR, or if it possibly represents a future trend in compiler design." Compilers have pretty much been the way (with the exception of GCC) since about 1975, if not earlier. :) LLVM is deliberately low level. SWIFT uses a higher level IR because they want to do static analysis and language specific optimizations on it. This makes sense, LLVM's IR is not going to be good at that. You could transmit metadata, but it's probably not a good architectural solution compared to progressive lowering of IR.
- mtgx 11y agoWill Mir create any of the problems Swift's bitcode creates? https://medium.com/@FredericJacobs/why-i-m-not-enabling-bitcode-f35cd8fbfcc5 https://medium.com/@FredericJacobs/why-i-m-not-enabling-bitc...
- pcwalton 11y agoMost of those problems are non-issues (e.g. backdoor injection--Apple always has the ability to do this) and the remaining issues have nothing to do with Rust.
- pdpi 11y agoThe LLVM bitcode problems are mostly caused by using the LLVM IR as a distribution format. As long as the rust MIR doesn't leak into the outside world, I don't see how it might turn into an issue
- Joky 11y agoI don't think it is a bug against LLVM: there are basically two way of doing this: either you extend LLVM with more intrinsic or concept that carry your high-level semantics to the LLVM level (this is what people from Azul are doing [0]) or build an other intermediate representation like Swift and Rust. For the swift case, you can see the talk at last year LLVM Dev Meeting [1]. There are advantages and drawback to each approaches. [0] : "LLVM for a managed language: what we've learned" [1] : "Swift's High-Level IR: A Case Study of Complementing LLVM IR with Language-Specific Optimization" Both have slides and video here: http://llvm.org/devmtg/2015-10/ http://llvm.org/devmtg/2015-10/