4 ms·

While I think this is great, Ruby implementations are notorious for being tricky to implement 100%. Rewriting Ruby in Rust is great and all but even TruffleRuby

While I think this is great, Ruby implementations are notorious for being tricky to implement 100%. Rewriting Ruby in Rust is great and all but even TruffleRuby isn't at the point where the authors recommend running a Rails app on it. I wonder if there's a way to add some Rust into MRI. Perhaps someone could write a YARV VM or a version of the JIT in Rust. It'd complicate the build pipeline, but it'd be iterative and improve the main implementation of Ruby.

jashmatthews 7y ago
> version of the JIT in Rust I'm building a tracing JIT for CRuby in Rust at the moment. I started out trying to build a method JIT but mixing type analysis and iterative inlining to deal with real idiomatic Ruby code gets complex very fast. With tracing, both follow naturally.
- aardvark179 7y ago
  What is your plan for C extensions? They are heavily used in Ruby and tend to create a tricky barrier to optimisation. The fact the standard library is mostly written in C doesn’t help either.
  - jashmatthews 7y ago
    Same thing LuaJIT does. Implement the really performance critical functions in IR and emit direct calls to C implementations of the complex ones. In LuaJIT the FFI specifies enough type information to be able to call native functions from a trace. I’m targeting optimizing through ActiveSupport String.blank? now I’ve got some micro benchmarks running.
- ksec 7y ago
  Is that something different to what vmakarov is doing [1], the work of current MJIT from k0kubun [2] seems to be based on this earlier work. And I believe vmakarov is working on RTL from YARV instructions, and a light weight JIT [3]. If I remember correctly there was a Tracing JIT for CRuby in 2016, but discontinued due to memory consumption [1] https://bugs.ruby-lang.org/issues/12589 https://bugs.ruby-lang.org/issues/12589 [2] https://medium.com/@k0kubun/the-method-jit-compiler-for-ruby-2-6-388ee0989c13 https://medium.com/@k0kubun/the-method-jit-compiler-for-ruby... [3] https://www.youtube.com/watch?v=emhYoI_RiOA https://www.youtube.com/watch?v=emhYoI_RiOA
  - jashmatthews 7y ago
    Yup. Quite different. Similar to RuJIT but I’m trying quite a different approach to trace recording and using CraneLift, an optimized code generation backend, rather than using tinycc Light weight JIT only helps to remove the overhead of GCC etc. Not to be able to inline, constant fold etc. Something like the JRuby IR is required for that.
p4lindromica 7y ago
Hi, I'm the author of Artichoke. Ruby has a great black box testing suite called ruby/spec [0] that is shared among multiple Ruby implementations. Artichoke has a custom runner [1] to track progress on implementation completeness. If Artichoke passes ruby/spec and is not compatible with MRI, that's a bug in the specs. re: adding Rust to MRI, I'm working on extracting the mruby backend from the core VM infra + core/stdlib impls [3] which would let us use MRI as the backing VM via rutie. When the Artichoke core and stdlib is complete, this would mean that the MRI runtime could be implemented entirely in Rust [2]. [0] https://github.com/ruby/spec https://github.com/ruby/spec [1] https://github.com/artichoke/artichoke/tree/master/spec-runner https://github.com/artichoke/artichoke/tree/master/spec-runn... [2] https://github.com/artichoke/artichoke/issues/92 https://github.com/artichoke/artichoke/issues/92 [3] https://artichoke.github.io/artichoke/artichoke_core/ https://artichoke.github.io/artichoke/artichoke_core/
- eregon 7y ago
  Thanks for the praise on ruby/spec :) OTOH, I'm sure there are such bugs where the specs don't cover every corner-case behavior. Specs are added when finding bugs in alternative implementations and following MRI's NEWS for new features, but there is no guarantee specs are complete.
  - p4lindromica 7y ago
    Hi eregon, I've submitted a fix to ruby/spec already! \o/ (I'm @lopopolo on GitHub.) One spec I know is missing is how capture groups should not be expanded into globals on `String#=~`. I'll file an issue [0]. [0] https://github.com/ruby/spec/issues/676 https://github.com/ruby/spec/issues/676
- chrisseaton 7y ago
  So what is the current pass rate for language and core specs? I didn’t see a number in any of those links or did I miss it? Do I have to build and run it myself to find out? If you can run mspec at all then that’s pretty good to be honest.
jb3689 7y ago
> I wonder if there's a way to add some Rust into MRI Aaron Patterson spoke to this in a recent interview. In fact I think he specifically mentioned using Rust as a possibility for a JIT
steveklabnik 7y ago
I re-implemented a single method on Array a few years back, it is absolutely feasible in a technical sense. These kinds of things are always more complicated than “is it technically possible,” of course.

jashmatthews 7y ago

> version of the JIT in Rust I'm building a tracing JIT for CRuby in Rust at the moment. I started out trying to build a method JIT but mixing type analysis and iterative inlining to deal with real idiomatic Ruby code gets complex very fast. With tracing, both follow naturally.

aardvark179 7y ago
What is your plan for C extensions? They are heavily used in Ruby and tend to create a tricky barrier to optimisation. The fact the standard library is mostly written in C doesn’t help either.
- jashmatthews 7y ago
  Same thing LuaJIT does. Implement the really performance critical functions in IR and emit direct calls to C implementations of the complex ones. In LuaJIT the FFI specifies enough type information to be able to call native functions from a trace. I’m targeting optimizing through ActiveSupport String.blank? now I’ve got some micro benchmarks running.
ksec 7y ago
Is that something different to what vmakarov is doing [1], the work of current MJIT from k0kubun [2] seems to be based on this earlier work. And I believe vmakarov is working on RTL from YARV instructions, and a light weight JIT [3]. If I remember correctly there was a Tracing JIT for CRuby in 2016, but discontinued due to memory consumption [1] https://bugs.ruby-lang.org/issues/12589 https://bugs.ruby-lang.org/issues/12589 [2] https://medium.com/@k0kubun/the-method-jit-compiler-for-ruby-2-6-388ee0989c13 https://medium.com/@k0kubun/the-method-jit-compiler-for-ruby... [3] https://www.youtube.com/watch?v=emhYoI_RiOA https://www.youtube.com/watch?v=emhYoI_RiOA
- jashmatthews 7y ago
  Yup. Quite different. Similar to RuJIT but I’m trying quite a different approach to trace recording and using CraneLift, an optimized code generation backend, rather than using tinycc Light weight JIT only helps to remove the overhead of GCC etc. Not to be able to inline, constant fold etc. Something like the JRuby IR is required for that.

p4lindromica 7y ago

Hi, I'm the author of Artichoke. Ruby has a great black box testing suite called ruby/spec [0] that is shared among multiple Ruby implementations. Artichoke has a custom runner [1] to track progress on implementation completeness. If Artichoke passes ruby/spec and is not compatible with MRI, that's a bug in the specs. re: adding Rust to MRI, I'm working on extracting the mruby backend from the core VM infra + core/stdlib impls [3] which would let us use MRI as the backing VM via rutie. When the Artichoke core and stdlib is complete, this would mean that the MRI runtime could be implemented entirely in Rust [2]. [0] https://github.com/ruby/spec https://github.com/ruby/spec [1] https://github.com/artichoke/artichoke/tree/master/spec-runner https://github.com/artichoke/artichoke/tree/master/spec-runn... [2] https://github.com/artichoke/artichoke/issues/92 https://github.com/artichoke/artichoke/issues/92 [3] https://artichoke.github.io/artichoke/artichoke_core/ https://artichoke.github.io/artichoke/artichoke_core/

eregon 7y ago
Thanks for the praise on ruby/spec :) OTOH, I'm sure there are such bugs where the specs don't cover every corner-case behavior. Specs are added when finding bugs in alternative implementations and following MRI's NEWS for new features, but there is no guarantee specs are complete.
- p4lindromica 7y ago
  Hi eregon, I've submitted a fix to ruby/spec already! \o/ (I'm @lopopolo on GitHub.) One spec I know is missing is how capture groups should not be expanded into globals on `String#=~`. I'll file an issue [0]. [0] https://github.com/ruby/spec/issues/676 https://github.com/ruby/spec/issues/676
chrisseaton 7y ago
So what is the current pass rate for language and core specs? I didn’t see a number in any of those links or did I miss it? Do I have to build and run it myself to find out? If you can run mspec at all then that’s pretty good to be honest.

jb3689 7y ago

> I wonder if there's a way to add some Rust into MRI Aaron Patterson spoke to this in a recent interview. In fact I think he specifically mentioned using Rust as a possibility for a JIT

steveklabnik 7y ago

I re-implemented a single method on Array a few years back, it is absolutely feasible in a technical sense. These kinds of things are always more complicated than “is it technically possible,” of course.