4 ms·

Hi, I'm the author of Artichoke. Ruby has a great black box testing suite called ruby/spec [0] that is shared among multiple Ruby implementations. Artichoke ha

Hi, I'm the author of Artichoke. Ruby has a great black box testing suite called ruby/spec [0] that is shared among multiple Ruby implementations. Artichoke has a custom runner [1] to track progress on implementation completeness. If Artichoke passes ruby/spec and is not compatible with MRI, that's a bug in the specs. re: adding Rust to MRI, I'm working on extracting the mruby backend from the core VM infra + core/stdlib impls [3] which would let us use MRI as the backing VM via rutie. When the Artichoke core and stdlib is complete, this would mean that the MRI runtime could be implemented entirely in Rust [2]. [0] https://github.com/ruby/spec https://github.com/ruby/spec [1] https://github.com/artichoke/artichoke/tree/master/spec-runner https://github.com/artichoke/artichoke/tree/master/spec-runn... [2] https://github.com/artichoke/artichoke/issues/92 https://github.com/artichoke/artichoke/issues/92 [3] https://artichoke.github.io/artichoke/artichoke_core/ https://artichoke.github.io/artichoke/artichoke_core/

eregon 7y ago
Thanks for the praise on ruby/spec :) OTOH, I'm sure there are such bugs where the specs don't cover every corner-case behavior. Specs are added when finding bugs in alternative implementations and following MRI's NEWS for new features, but there is no guarantee specs are complete.
- p4lindromica 7y ago
  Hi eregon, I've submitted a fix to ruby/spec already! \o/ (I'm @lopopolo on GitHub.) One spec I know is missing is how capture groups should not be expanded into globals on `String#=~`. I'll file an issue [0]. [0] https://github.com/ruby/spec/issues/676 https://github.com/ruby/spec/issues/676
chrisseaton 7y ago
So what is the current pass rate for language and core specs? I didn’t see a number in any of those links or did I miss it? Do I have to build and run it myself to find out? If you can run mspec at all then that’s pretty good to be honest.
- p4lindromica 7y ago
  I can run MSpec to completion for language, core, and library specs. Thank you for the encouragement :D. I've implemented a runner that can skip known things that cause the spec to hang (e.g. Mutex specs since Artichoke has a single-threaded implementation of Mutex that deadlocks during the specs). $ pushd spec-runner/vendor/ruby/language $ cargo run --bin spec-runner **/*.rb Passed 1078, skipped 35, not implemented 2, failed 476 specs. $ popd; pushd spec-runner/vendor/ruby/core $ cargo run --bin spec-runner **/*.rb Passed 5412, skipped 998, not implemented 201, failed 8629 specs. $ popd; pushd spec-runner/vendor/ruby/library $ cargo run --bin spec-runner **/*.rb Passed 0, skipped 22, not implemented 16, failed 3276 specs. The library specs pass number is a bug in my spec runner. All packages added to Artichoke pass ruby/spec. Those packages are : delegate, forwardable, json, monitor, ostruct, set, srscan, and uri.
- p4lindromica 7y ago
  I can generate a set of pass rates during build and output it to the gh-pages branch. That's a good idea. Thank you. https://github.com/artichoke/artichoke/issues/101 https://github.com/artichoke/artichoke/issues/101
_hardwaregeek 7y ago
Hey, I didn't mean to dismiss your work. MRI is frankly a lot of complicated, at times archaic C and I'd certainly take a nice pile of Rust instead :D. I'm mostly just concerned because as far as I know there's maybe 3-4 implementations of Ruby that are actively developed and a whole lot more abandoned ones ^[1]. Have you considered taking some of the pieces of Artichoke and finding a way to incorporate them into MRI? ^[1] https://github.com/codicoscepticos/ruby-implementations https://github.com/codicoscepticos/ruby-implementations

eregon 7y ago

Thanks for the praise on ruby/spec :) OTOH, I'm sure there are such bugs where the specs don't cover every corner-case behavior. Specs are added when finding bugs in alternative implementations and following MRI's NEWS for new features, but there is no guarantee specs are complete.

p4lindromica 7y ago
Hi eregon, I've submitted a fix to ruby/spec already! \o/ (I'm @lopopolo on GitHub.) One spec I know is missing is how capture groups should not be expanded into globals on `String#=~`. I'll file an issue [0]. [0] https://github.com/ruby/spec/issues/676 https://github.com/ruby/spec/issues/676

chrisseaton 7y ago

So what is the current pass rate for language and core specs? I didn’t see a number in any of those links or did I miss it? Do I have to build and run it myself to find out? If you can run mspec at all then that’s pretty good to be honest.

p4lindromica 7y ago
I can run MSpec to completion for language, core, and library specs. Thank you for the encouragement :D. I've implemented a runner that can skip known things that cause the spec to hang (e.g. Mutex specs since Artichoke has a single-threaded implementation of Mutex that deadlocks during the specs). $ pushd spec-runner/vendor/ruby/language $ cargo run --bin spec-runner **/*.rb Passed 1078, skipped 35, not implemented 2, failed 476 specs. $ popd; pushd spec-runner/vendor/ruby/core $ cargo run --bin spec-runner **/*.rb Passed 5412, skipped 998, not implemented 201, failed 8629 specs. $ popd; pushd spec-runner/vendor/ruby/library $ cargo run --bin spec-runner **/*.rb Passed 0, skipped 22, not implemented 16, failed 3276 specs. The library specs pass number is a bug in my spec runner. All packages added to Artichoke pass ruby/spec. Those packages are : delegate, forwardable, json, monitor, ostruct, set, srscan, and uri.
p4lindromica 7y ago
I can generate a set of pass rates during build and output it to the gh-pages branch. That's a good idea. Thank you. https://github.com/artichoke/artichoke/issues/101 https://github.com/artichoke/artichoke/issues/101

_hardwaregeek 7y ago

Hey, I didn't mean to dismiss your work. MRI is frankly a lot of complicated, at times archaic C and I'd certainly take a nice pile of Rust instead :D. I'm mostly just concerned because as far as I know there's maybe 3-4 implementations of Ruby that are actively developed and a whole lot more abandoned ones ^[1]. Have you considered taking some of the pieces of Artichoke and finding a way to incorporate them into MRI? ^[1] https://github.com/codicoscepticos/ruby-implementations https://github.com/codicoscepticos/ruby-implementations