5 ms·
Mir: A lightweight JIT compiler project
- DominoTree 7y agoHey everyone, please stop naming things Mir for a while. Thanks!
- giancarlostoro 7y ago> https://en.wikipedia.org/wiki/Mir_(disambiguation)#Science_and_technology https://en.wikipedia.org/wiki/Mir_(disambiguation)#Science_a... I feel like this list is missing some more, but yeah, there's been a few 'Mir' projects.
- jordigh 7y agoI was thinking of this Mir: http://docs.mir.dlang.io/latest/index.html http://docs.mir.dlang.io/latest/index.html
- giancarlostoro 7y agoI was thinking of Rust's MIR being missing too: https://blog.rust-lang.org/2016/04/19/MIR.html https://blog.rust-lang.org/2016/04/19/MIR.html
- wyldfire 7y ago> That is, we are introducing a new intermediate representation (IR) of your program that we call MIR: MIR stands for mid-level IR, because the MIR comes between the existing HIR ("high-level IR", roughly an abstract syntax tree) and LLVM (the "low-level" IR). It's so terribly confusing because LLVM itself defines a .mir (Machine IR) that is a syntax between its IR and the backend (which could be thought of as mid-level IR). When I heard about Rust's MIR, I assumed it was some clever way to generate target-dependent IR. The good news is that we now have M(L)IR: the one IR to bring them all and in the darkness bind them.
- anp 7y agoIf anything I expect MLIR to make things more confusing as Rust’s MIR is probably going to be around for the long haul.
- crystaldev 7y agoI was thinking of this Mir: https://mir-server.io/ https://mir-server.io/
- walkingolof 7y agoI was thinking of the Soviet Spacestation Mir, probably will tell you my age... https://en.wikipedia.org/wiki/Mir https://en.wikipedia.org/wiki/Mir
- crystaldev 7y agoEveryone thinks of the Mir space station first, no?
- catalogia 7y agoProbably most people past their mid 20s, I don't know that many schools teach about it.
- crystaldev 7y agoThat sounds right, anybody with memory of 2001 when its deorbit was in the news.
- pleasewaiting 7y ago> DominoTree Are you the same DominoTree as the renowned racist and miscreant Nick Price? If not, that's an unfortunate coincidence for you to have chosen that username! edit: Judging by the votes, I suppose it MUST be! :^)
- NoImmatureAdHom 7y agoPlease don't do this here.
- deleted 7y ago[deleted]
- bakery2k 7y agoFrom GitHub [1]: "Plans to try MIR light-weight JIT first for CRuby or/and MRuby implementation" "MIR is strongly typed" Is there an explanation of how the project bridges the gap between dynamically-typed Ruby and statically-typed MIR? More generally, I'd love to see something like MRuby+MIR be successful. It would be great to see an alternative to the aging LuaJIT. [1] https://github.com/vnmakarov/mir https://github.com/vnmakarov/mir
- vnorilo 7y agoSeems to me that MIR operates on a (much) lower level, basically abstract away the physical machine and its finite register set. As such, it would replace LLVM (or GCC) middle and back end. The goal is much faster compilation without sacrificing more than ~20% of performance. Dynamic types and garbage collection would then be implemented on/for the abstract MIR machine.
- chrisseaton 7y ago> Is there an explanation of how the project bridges the gap between dynamically-typed Ruby and statically-typed MIR? My understanding is that MIR doesn’t tackle this problem, so the code does stay pretty dynamic when compiled, just like its sister project YARV MJIT. They both need an intermediate profiling mode to specialise and monomorphise but nobody is building that as far as I know.
- monocasa 7y agoIt's probably like how V8 works. Even though JS is dynamically typed, V8 will keep internal static type definitions around based on what it sees when running, and trap out internally to the slow path when the types don't match what it thinks should happen rather than throwing type errors.
- mratsim 7y agoI would be quite interested in an IR/JIT assembler specialized for vector instructions with ARM Neon and x86-64 SSE~AVX512 output. Ideally it handles register allocations and generated function caching as well. The current JIT assembler (ASMJIT, Xbyak) requires you to handle register allocations. LLVM is as mentioned quite a heavy dependency to have.
- MaxBarraclough 7y agoHow about using GCC's NEON intrinsics from C?
- Asm2D 7y agoasmjit has a register allocator, for sure not the highest quality one, but it's there in the asmjit's Compiler infrastructure.
- kbumsik 7y agoHey Mir display server is still alive :)
- DC-3 7y agoYet another drastic pivot!
- mark_l_watson 7y agoGood analysis on memory usage and how that negates use in mobile and IoT projects. Ruby was my go to language for a long dry spell when I had little Lisp development jobs (except for Clojure). Ruby is a great language, as Matz says, Ruby is designed for developer happiness. I stopped using Ruby when more Common Lisp work came my way and then I used Python for five years of deep learning work. I have favorite languages but I used what customers wanted. That said, I still keep up with Ruby news.
- scythe 7y agoHow is this meaningfully different in scope and intention vs Parrot? That project went on for a long time until every language (including its original target, Raku) decided they’d rather build their own more specialized JIT. What would prevent MIR from meeting the same fate?
- jashmatthews 7y agoMIR is much more flexible. It's like CraneLift but more basic. Anything you write in MIR can be compiled by MIR. Parrot only JITs Parrot bytecode, so the applications are much more limited.
- pizlonator 7y agoThis project is trying to do too many things. It seems like Ruby needs a profile guided optimizer, which means building an IR that is suitable for profile-guided optimization. That’s way different from classic IRs like this since it means having provisions for OSR exit. I recommend looking at these slides to learn how to do it. http://www.filpizlo.com/slides/pizlo-speculation-in-jsc-slides.pdf http://www.filpizlo.com/slides/pizlo-speculation-in-jsc-slid... http://www.filpizlo.com/slides/pizlo-splash2018-jsc-compiler-slides.pdf http://www.filpizlo.com/slides/pizlo-splash2018-jsc-compiler...
- chrisseaton 7y agoI agree - I think MIR is trying to produce better machine code more quickly, but that isn't the problem Ruby faces! The problem Ruby faces is having to inline through ten levels of metaprogramming in order to get any meaningfully-sized compilation unit that you can optimise, and only then it makes sense to worry about code generation. However, the reason that this is not so simple is that Ruby is a vastly more complicated language than JavaScript (I've worked on implementing both.) Ruby has an enormous standard library and most Ruby programs are just endless calls to the that library, so your compiler must be able to understand the library semantically, either by rewriting it in Ruby (not likely at scale in MRI) or by adding tens of thousands of individually optimised intrinsics (again not likely). Ruby is digging itself further into a local optima with these approaches optimisations, rather than looking further around for a better global optima.
- pizlonator 7y agoYour description of optimizing Ruby sounds like it is exactly like optimizing JS. JSC’s main optimizing IR (the DFG) is all about understanding the standard library semantically. We write the standard library in JS+hacks (called “builtins”) or C++ (pick on a per function basis). Some of those functions have opcodes in DFG, or are built out of primitives that have their own specialized opcodes in DFG. Maybe the reason why Ruby optimization has problems is nobody has done it the JSC way.
- 7y ago
- ksec 7y agoThere are quite a few decent active Ruby Implementation going on at the moment. CRuby with MJIT MIR ( This ) JRuby ( Ruby on JVM ) TruffleRuby ( Ruby on Graal ) Artichoke ( Ruby on Rust ) And I remember someone mentioned making Ruby with Tracing JIT. ( Not Topaz ) Unfortunately My Google fu is not good enough I can no longer find it.
- boulos 7y agoRubinius. Here are Evan’s slides from the 2009 LLVM developer conference: https://llvm.org/devmtg/2009-10/Phoenix_AcceleratingRuby.pdf https://llvm.org/devmtg/2009-10/Phoenix_AcceleratingRuby.pdf EngineYard took Rubinius in a few directions, but I think the main lasting impact was all the RSpec work they did along the way.
- ksec 7y agoEdited my original post. I meant Implementation that are still actively developed. Both Rubinius and Topaz are no longer being maintained.
- YorickPeterse 7y agoI believe Rubinius _is_ being maintained (again), it's just going in a direction very different from Ruby.
- lukegru 7y agoYes, there are quite a few Ruby implementations but MIR is not one nor is it trying to be. It's a standalone library useful for use in a tier 1 JIT for any dynamic language, along with other uses outside of JITs having to do with native code generation. The author does have plans to integrate it into CRuby to work alongside MJIT.
- nn3 7y agoInteresting. Seems to be the first modern non trivial compiler that avoids using SSA internally. I thought SSA had cornered the market, but perhaps now there is an opposite trend (ok one example doesn't make a trend) <quote> No SSA (single static assignment form) for: Faster optimizations for short optimizations pipeline and small functions (a target usage scenario) Currently SSA could be used only for two optimizations (CCP and GCSE). SSA usage would mean 4 additional passes over IR. If we implement more optimizations, SSA transition is possible when additional time for expensive in/out SSA passes will be less than additional time for non-SSA optimization implementation Simpler and more compact generator code because we can avoid to implement a lot of nontrivial code (for dominator and dominator frontier calculation, a good out of SSA code) </quote>
- tom_mellior 7y ago> Seems to be the first modern non trivial compiler that avoids using SSA internally. Of course it's debatable what "modern" and "non trivial" mean, and if they apply to this project. It's very small, but on the (one!) small benchmark the author cites it seems to do quite well, so it's certainly not completely naive. For whatever it's worth, CompCert doesn't use SSA either, and that's certainly a non-trivial compiler, though arguably the non-triviality does not stem from any advanced optimizations it does.
- MaxBarraclough 7y agoThe article stresses how their JIT is much more lightweight than GCC/LLVM, which is perfectly valid, but why not compare MIR against the other portable lightweight JIT engines out there? They're not the first to think of it. The article mentions Cranelift, but that's a 'middleweight JIT' with a proper SSA IR. I was surprised to see LibJIT has more LOC than Cranelift - I thought it was lighter. (Imperfect proxy for runtime 'weight', of course.) If you want a lightweight portable JIT engine, there's already GNU Lightning [0], and the atrociously-named Lightening fork [1] (used in the new JIT in the GNU Guile Scheme interpreter, which turned up on the HN front page recently). Here's a 1996 paper (preprint) on a research JIT named VCODE which executed around 8 instructions to generate each instruction in its output. [2] (Sadly it was never released, as far as I can tell, and is presumably long dead.) Anyway, with all that said, I wish this project well. No-one's managed to get good performance out of Ruby yet, so it's certainly ambitious. Google gave up on Unladen Swallow, and that was a JIT for Python, which, as I understand it, is more amenable to JIT than Ruby. Even failing that, having a quality rival to GNU Lightning would be worthwhile. [0] https://www.gnu.org/software/lightning/ https://www.gnu.org/software/lightning/ [1] https://www.wingolog.org/archives/2019/05/24/lightening-run-time-code-generation https://www.wingolog.org/archives/2019/05/24/lightening-run-... [2] http://www-leland.stanford.edu/class/cs343/resources/vcode-annotated.pdf http://www-leland.stanford.edu/class/cs343/resources/vcode-a...
- tom_mellior 7y ago> implement the GCC C extensions necessary for the CRuby JIT implementation That's the point where "C is nice and simple, it's easy to whip up a compiler" invariably turns into "why the #^(&)# didn't I use an existing frontend?". Real-world C code is messy. The entire sub-project of implementing a C compiler is a needless distraction that will turn into a huge time-suck with zero benefit to the author. See also: "Why Says C is Simple?" https://people.eecs.berkeley.edu/~necula/cil/cil016.html https://people.eecs.berkeley.edu/~necula/cil/cil016.html