9 ms·
Artichoke – A Ruby made with Rust
- ultrarunner 7y agoNo c extensions? So most popular gems won’t work? Does that make this essentially an intellectual pursuit without practical application?
- untog 7y agoNo reason it can't be added in the future.
- msbarnett 7y agoWithout a GIL it would be incredibly difficult, would it not? AFAIK C extensions to MRI are not thread safe. This could, of course, expose a different C API that allows for thread safe extensions, but I think the original comment is complaining about lack of compatibility with all of the important gems that depend on the existing MRI C API. This has always been an issue that held back JRuby.
- XMPPwocky 7y agoCould one add, effectively, a RWLock GIL - one where any number of Ruby interpreter threads can run at once (since they know how to not step on each other's toes), but any C code which tried to "grab the GIL" would run exclusively w.r.t. interpreter threads (and other C code)?
- riffraff 7y agoI believe the "guild" proposal makes it such that the global lock becomes guild-specific, which seems similar to what you propose.
- pjmlp 7y agoJRuby does not have a GIL.
- y4mi 7y agoMy personal experience with Ruby is extremely shallow, but a casual glance at the search results wrt the GIL in Ruby seems to say it actually does have one. I could be wrong, as I'm not an experienced Ruby developer, however. I.e. https://thoughtbot.com/blog/untangling-ruby-threads https://thoughtbot.com/blog/untangling-ruby-threads
- msbarnett 7y agoI’m aware. It also lacks compatibility with a lot of important C gems (not only or even predominantly because of the lack of a GIL, of course), which I’m pointing out has always impacted its adoption.
- ROFISH 7y agoThe proposed API, MRB_API, is used in mruby. With enough popular support for that C API, it could force mainline ruby to modernize it's C API to that one. Assuming MRB_API thread-safe-able. mruby is not thread safe at all, being a single-threaded script runner.
- p4lindromica 7y agoHi, I'm the author of Artichoke. My expectation is that MRB_API can be implemented in a thread-safe way. The API manipulates the interpreter state (in the case of mruby, an `mrb_state`). The interpreter state itself and the implementation of the API can be thread safe or they can not be. I don't think mruby even assumes the presence of threads on target systems, so it is completely reasonable that the implementation is not thread safe.
- p4lindromica 7y agoHi, I'm the author of Artichoke. I don't have a good answer for whether or not implementing the MRI API would be difficult. I _suspected_ it would be, which is why I listed it as a non-goal. There is a lot of other work to do before we get to the point of C API compatibility. However, it may be possible to use `rubysys` in rutie as a base for an MRI compatible API: https://github.com/artichoke/artichoke/issues/88 https://github.com/artichoke/artichoke/issues/88
- pohl 7y agoYou might want to clarify in the readme that the red X means non-goal. Until I read this comment I thought it was a goal that had not yet been implemented.
- p4lindromica 7y agoin the 14 hours since I wrote the comment you replied to, I moved an MRI C API from non-goal to goal [0]. :) The red X means the goal is not yet achieved. [0] https://github.com/artichoke/artichoke/pull/108 https://github.com/artichoke/artichoke/pull/108
- jashmatthews 7y agoSomething similar to Racket places or V8 isolates can be implemented and keep compatibility with C extensions expecting GIL. It just needs somebody to actually do the work of swapping out hundreds of global variable accesses in MRI using something like CIL.
- eregon 7y agoDo you mean every thread would have its own place/isolate for executing C extensions? I think one difficulty there is some global state in C extensions might expect to be truly process-global. Also, how would you isolate global variables in the C extension? If the C extension is a dynamic library, it's typically only loaded once per process. Maybe something like dlmopen() to load multiple copies of a native library would help, but it's not portable.
- jashmatthews 7y agoYeah, this is a real problem. The PG gem, for example, uses some global variables. IMHO the only way to do it is to add incremental parallelism which leaves the GIL in place. Racket has already shown a solid path here. Guilds would have a major performance problem: can't allocate objects without GIL. It's also a tricky mental model and requires invasive changes to existing Ruby code to handle frozen objects. Places don't share a heap so they don't need the GIL to allocate objects and have independent GC rather than global GC. It's a model which fits better with existing Ruby code. The GVL can be relaxed while executing Ruby and grabbed by native methods. WDYT?
- eregon 7y ago> Guilds would have a major performance problem: can't allocate objects without GIL. Why not? It'd be possible to have TLABs, isn't it? But yes, GC would still be for all Guilds at once. Racket places don't allow to share objects, only arrays of primitive types, which seems very restrictive. And Racket futures are even more restricted. I'm not sure about Guilds since it's not there yet, but it already sounds closer to Ruby multithreading and more flexible to me from an usage point of view.
- p4lindromica 7y agoHi I'm the author of Artichoke. Thanks for your interest in the project. You can track progress on this issue on GitHub [0]. Would love your help on this if you could lend it. :) [0] https://github.com/artichoke/artichoke/issues/99 https://github.com/artichoke/artichoke/issues/99
- Exuma 7y agoI wonder how much this would change the performance of ruby... that's all kind of cool except no c extensions, that basically makes all the gems I use unusable.
- dymk 7y agoThey're going to have to spend an immense amount of time implementing all the performance optimizations present in mri/yarv bytecode engine before they achieve performance parity. The amount of engineering effort to remove the GIL and simultaniously make C extensions work is probably not worth simply choosing a multi-process w/ shared memory solution for true parallelism.
- p4lindromica 7y agoHi, I'm the author of Artichoke. I wouldn't say all gems are unusable. I implemented a Rack-compatible application server in an early version of Artichoke that is capable of serving a Sinatra Rack app. https://github.com/artichoke/ferrocarril https://github.com/artichoke/ferrocarril
- pmontra 7y agoJRuby doesn't support C extensions too https://github.com/jruby/jruby/wiki/C-Extension-Alternatives https://github.com/jruby/jruby/wiki/C-Extension-Alternatives It's a parallel world in which developers either use pure Ruby gems or Java jars. For example, the database drivers are JDBC adapters straight from the Java world not the usual mysql2 or pg gems. Obviously the programs that use them are not portable unless there is a compatibility layer. In that example, ActiveRecord abstracts the drivers. I don't know anything about Rust but maybe it could end up in the same way.
- subroutine 7y agoA little oxidation and this would make the perfect logo: https://www.amazon.com/dp/B00IK9F9F4 https://www.amazon.com/dp/B00IK9F9F4
- samatman 7y agoThis is a great idea but why would it be called anything other than Jasper: https://en.wikipedia.org/wiki/Jasper https://en.wikipedia.org/wiki/Jasper It doesn't get closer to a Ruby made with Rust than that...
- iovrthoughtthis 7y agoThat would literally make my year, as a Jasper.
- rishav_sharan 7y agoOnce this language is more mature, comparing it with Crystal would be great. Artichoke will have the added benefit of having Ruby's entire ecosystem as its behest.
- _hardwaregeek 7y agoWhile I think this is great, Ruby implementations are notorious for being tricky to implement 100%. Rewriting Ruby in Rust is great and all but even TruffleRuby isn't at the point where the authors recommend running a Rails app on it. I wonder if there's a way to add some Rust into MRI. Perhaps someone could write a YARV VM or a version of the JIT in Rust. It'd complicate the build pipeline, but it'd be iterative and improve the main implementation of Ruby.
- jashmatthews 7y ago> version of the JIT in Rust I'm building a tracing JIT for CRuby in Rust at the moment. I started out trying to build a method JIT but mixing type analysis and iterative inlining to deal with real idiomatic Ruby code gets complex very fast. With tracing, both follow naturally.
- aardvark179 7y agoWhat is your plan for C extensions? They are heavily used in Ruby and tend to create a tricky barrier to optimisation. The fact the standard library is mostly written in C doesn’t help either.
- jashmatthews 7y agoSame thing LuaJIT does. Implement the really performance critical functions in IR and emit direct calls to C implementations of the complex ones. In LuaJIT the FFI specifies enough type information to be able to call native functions from a trace. I’m targeting optimizing through ActiveSupport String.blank? now I’ve got some micro benchmarks running.
- ksec 7y agoIs that something different to what vmakarov is doing [1], the work of current MJIT from k0kubun [2] seems to be based on this earlier work. And I believe vmakarov is working on RTL from YARV instructions, and a light weight JIT [3]. If I remember correctly there was a Tracing JIT for CRuby in 2016, but discontinued due to memory consumption [1] https://bugs.ruby-lang.org/issues/12589 https://bugs.ruby-lang.org/issues/12589 [2] https://medium.com/@k0kubun/the-method-jit-compiler-for-ruby-2-6-388ee0989c13 https://medium.com/@k0kubun/the-method-jit-compiler-for-ruby... [3] https://www.youtube.com/watch?v=emhYoI_RiOA https://www.youtube.com/watch?v=emhYoI_RiOA
- mkorfmann 7y agoIs it possible to install ruby gems in the browser playground? So, Does ruby gem install via web assembly runtime work?
- p4lindromica 7y agoNot yet, but you can track progress here: https://github.com/artichoke/artichoke/issues/105 https://github.com/artichoke/artichoke/issues/105
- pankajdoharey 7y agoI actually like the idea, being a Rust illiterate i have way of reading it. I like the tone of it "No GIL". I hope it does come to fruition. But i would love ruby programs to eventually be able produce machine code, like tons of lisp and scheme implementations do. Even Smalltalk did. I hope with "No GIL" that should be somewhere in the pipeline.
- judofyr 7y agoI love seeing more Ruby implementations! Looking forward to seeing how it will progress. Are there any details on how it’s implemented? How does it execute Ruby? Does it have its own bytecode? Does it compile to machine code on the fly? What will make it faster than MRI?
- p4lindromica 7y agoThank you for your kind words. Growing pains of blowing up on HN lol, I do not have this written down yet, but I have a ticket to start documenting this [0]. The idea is to have a core set of traits [1] that when implemented allow an implementation to load an interpreter agnostic Ruby core and Ruby Standard Library. There is currently only one interpreter that implements these traits, and it is backed by mruby. My ultimate goal is to move off of mruby to either an MRI-backed interpeter via rutie or a native Rust-implemented backend + VM + parser. [0] https://github.com/artichoke/artichoke/issues/102 https://github.com/artichoke/artichoke/issues/102 [1] https://artichoke.github.io/artichoke/artichoke_core/ https://artichoke.github.io/artichoke/artichoke_core/
- p4lindromica 7y agoFor an example of how Artichoke can be faster than MRI, I'm currently working on extending the oniguruma-based `Regexp` implementation to have a fast-path backed by the Rust regex crate in some circumstances. In testing this can speed up `String#scan` by a factor of 10 for some `Regexp`s