6 ms·

A light-weight JIT compiler based on MIR (Medium Internal Representation)

62 points by bshanks 6y ago

Rochus 6y ago
Very interesting project; definitely worth considering. Combined with e.g. https://github.com/Ravenbrook/mps https://github.com/Ravenbrook/mps and https://github.com/facebookarchive/luaffifb https://github.com/facebookarchive/luaffifb one could build a nice, lean general purpose VM.
- Throwawayaerlei 6y ago
  One thing that hit me on reading the January 20th, 2020 blog description of MIR https://developers.redhat.com/blog/2020/01/20/mir-a-lightweight-jit-compiler-project/ https://developers.redhat.com/blog/2020/01/20/mir-a-lightwei... is that it doesn't generate position independent code. Wouldn't that need to be changed for a copying GC like Ravenbrook's?
  - KMag 6y ago
    I think it's pretty rare for copying GC to be extended to the JIT'd machine code. That is, it only makes a difference if you're moving around the machine code in memory, not if you're moving around data. (The vtables can still be moved around.. all of the function pointers in them are absolute.) Does Ravenbook also use copying for machine code?
    - readittwice 6y ago
      There is the caveat that generated machine code can embed addresses of objects that get relocated by the GC. In this case the code needs to be patched even though the code itself doesn't move.
      - KMag 6y ago
        I'm not familiar with Ravenbook. How does it handle GC roots outside the managed heap? I presume it updates the GC roots as it moves the objects they point to, or does it not support GC roots outside the managed heap? If the code refers to managed objects, you need to mark those locations in the machine code as containing GC roots, otherwise, you might GC the referenced objects, which is just as bad as moving them.
  - readittwice 6y ago
    I don't really know that particular GC, but not necessarily. Code objects are usually stored in a separate area and might be managed differently than regular objects. So a copying GC might not relocate code - maybe not even collect it. In case objects are relocated by the GC, there is usually "relocation information" which can be used to patch the code after it got relocated.
  - Rochus 6y ago
    MPS has different collectors, and I don't think that the generated code should be subject to a collector or include addresses to collectable objects. MPS is an very interesting library I recently started to familiarize myself with; I think it should be considered by programming languages currently using Boehm GC.
ksec 6y ago
Previously on HN [1], turns out it was me :) who submitted this. And sparked a long discussion between Engineers from GraalVM TruffleRuby and Javascripts Core, along with few other VM code authors. I am wondering what's new? Since the author hasn't said much about it. [1] https://news.ycombinator.com/item?id=22098399 https://news.ycombinator.com/item?id=22098399
Boxxed 6y ago
The readme has a link to the blog post describing MIR -- it's a great read and is probably where you actually want to go: https://developers.redhat.com/blog/2020/01/20/mir-a-lightweight-jit-compiler-project/ https://developers.redhat.com/blog/2020/01/20/mir-a-lightwei...
JoshTriplett 6y ago
Thought this might be related to https://rustc-dev-guide.rust-lang.org/mir/index.html https://rustc-dev-guide.rust-lang.org/mir/index.html at first, but it seems to be an entirely unrelated project using the same name.
- bshanks 6y ago
  Yes, it's unrelated. The author says "The central notion of JIT is a well-defined intermediate language called Medium Internal Representation, or MIR. You can find the name in Steven Muchnik’s famous book “Advanced Compiler Design and Implementation.” The Rust team also uses this term for a Rust intermediate language. Still, I decided to use the same name because I like it: MIR means “peace” and “world” in Russian." in https://developers.redhat.com/blog/2020/01/20/mir-a-lightweight-jit-compiler-project/ https://developers.redhat.com/blog/2020/01/20/mir-a-lightwei...
oscargrouch 6y ago
Have an eye on this project. It's the equivalent of Lua and SQLite of JIT engines.
- WalterGR 6y ago
  As the go-to JIT for embedded / scripting / extension languages? Or is there another scenario you envision?
  - oscargrouch 6y ago
    Wouldn't say its the go-to. There are a lot of cool JIT's around that can be used as the backend to something. But this is one of the most simple, small and embedded that i'm aware of.

62 points by bshanks 6y ago

Very interesting project; definitely worth considering. Combined with e.g. https://github.com/Ravenbrook/mps https://github.com/Ravenbrook/mps and https://github.com/facebookarchive/luaffifb https://github.com/facebookarchive/luaffifb one could build a nice, lean general purpose VM.

Throwawayaerlei 6y ago
One thing that hit me on reading the January 20th, 2020 blog description of MIR https://developers.redhat.com/blog/2020/01/20/mir-a-lightweight-jit-compiler-project/ https://developers.redhat.com/blog/2020/01/20/mir-a-lightwei... is that it doesn't generate position independent code. Wouldn't that need to be changed for a copying GC like Ravenbrook's?
- KMag 6y ago
  I think it's pretty rare for copying GC to be extended to the JIT'd machine code. That is, it only makes a difference if you're moving around the machine code in memory, not if you're moving around data. (The vtables can still be moved around.. all of the function pointers in them are absolute.) Does Ravenbook also use copying for machine code?
  - readittwice 6y ago
    There is the caveat that generated machine code can embed addresses of objects that get relocated by the GC. In this case the code needs to be patched even though the code itself doesn't move.
    - KMag 6y ago
      I'm not familiar with Ravenbook. How does it handle GC roots outside the managed heap? I presume it updates the GC roots as it moves the objects they point to, or does it not support GC roots outside the managed heap? If the code refers to managed objects, you need to mark those locations in the machine code as containing GC roots, otherwise, you might GC the referenced objects, which is just as bad as moving them.
- readittwice 6y ago
  I don't really know that particular GC, but not necessarily. Code objects are usually stored in a separate area and might be managed differently than regular objects. So a copying GC might not relocate code - maybe not even collect it. In case objects are relocated by the GC, there is usually "relocation information" which can be used to patch the code after it got relocated.
- Rochus 6y ago
  MPS has different collectors, and I don't think that the generated code should be subject to a collector or include addresses to collectable objects. MPS is an very interesting library I recently started to familiarize myself with; I think it should be considered by programming languages currently using Boehm GC.

ksec 6y ago

Previously on HN [1], turns out it was me :) who submitted this. And sparked a long discussion between Engineers from GraalVM TruffleRuby and Javascripts Core, along with few other VM code authors. I am wondering what's new? Since the author hasn't said much about it. [1] https://news.ycombinator.com/item?id=22098399 https://news.ycombinator.com/item?id=22098399

Boxxed 6y ago

The readme has a link to the blog post describing MIR -- it's a great read and is probably where you actually want to go: https://developers.redhat.com/blog/2020/01/20/mir-a-lightweight-jit-compiler-project/ https://developers.redhat.com/blog/2020/01/20/mir-a-lightwei...

JoshTriplett 6y ago

Thought this might be related to https://rustc-dev-guide.rust-lang.org/mir/index.html https://rustc-dev-guide.rust-lang.org/mir/index.html at first, but it seems to be an entirely unrelated project using the same name.

bshanks 6y ago
Yes, it's unrelated. The author says "The central notion of JIT is a well-defined intermediate language called Medium Internal Representation, or MIR. You can find the name in Steven Muchnik’s famous book “Advanced Compiler Design and Implementation.” The Rust team also uses this term for a Rust intermediate language. Still, I decided to use the same name because I like it: MIR means “peace” and “world” in Russian." in https://developers.redhat.com/blog/2020/01/20/mir-a-lightweight-jit-compiler-project/ https://developers.redhat.com/blog/2020/01/20/mir-a-lightwei...

oscargrouch 6y ago

Have an eye on this project. It's the equivalent of Lua and SQLite of JIT engines.

WalterGR 6y ago
As the go-to JIT for embedded / scripting / extension languages? Or is there another scenario you envision?
- oscargrouch 6y ago
  Wouldn't say its the go-to. There are a lot of cool JIT's around that can be used as the backend to something. But this is one of the most simple, small and embedded that i'm aware of.