9 ms·

MRI Developers Don't Use RubySpec and It's Hurting Ruby

291 points by jc00ke 12y ago

jc00ke 12y ago
I mistakenly cut off the full title of the post: "Matz's Ruby Developers Don't Use RubySpec and It's Hurting Ruby"
serve_yay 12y ago
Jeez, what a mess.
danielweber 12y ago
Why should they have been using RubySpec?
- joevandyk 12y ago
  Did you read the article? It contains 10 reasons why the MRI tests aren't sufficient.
  - danielweber 12y ago
    The Ruby devs also doesn't use any libraries I wrote, and that's okay. Why is RubySpec the special sauce that is so special that the devs not using it is horrible?
    - joevandyk 12y ago
      Because RubySpec is the best specification for Ruby that exists. By far.
    - masklinn 12y ago
      > Why is RubySpec the special sauce that is so special that the devs not using it is horrible? Assuming TFA isn't actively lying[0]: because it fixes the issues outlined which allow (amongst other things) cross-implementation and cross-version uses, because it tests behaviours not tested by the MRI suite anyway and because it's there as an extensive test suite for the language. Oh, and because according to TFA it found regressions in just about every release it's existed for. Regressions which could thus have been caught before the release was cut. [0] and I don't see any reason to assume otherwise, quite the opposite.
      - headius 12y ago
        I wouldn't say TFA is lying but it is presenting only part of the truth. MRI does have an actively developed test suite, which they run in CI and which we use to test compatibility in JRuby. MRI has contributed to RubySpec in the past and many contributors I know stopped (like I did) for nontechnical reasons. They are not ignoring it, but it is a hard project to collaborate on given the attitudes of the project lead that should be evident in this post.
    - stormbrew 12y ago
      Honestly, just the fact that rubyspec exposes a segfault in a new release is more than reason enough that they should be using it. It is de facto a more complete test suite than the ad hoc one they've been making. Given that, the onus is on the MRI developers to demonstrate why they are actively avoiding the use of a tool that could improve the stability of their language and environment.
      - headius 12y ago
        They have used and contributed to RubySpec in the past, but many (like me) were turned off by the maintainers' attitudes toward contributors and lack of respect. For example, see the zenspider link elsewhere in this thread. Nobody questions RubySpec as a project. But there are many nontechnical reasons why it was never adopted wholeheartedly by ruby-core.
        joevandyk 12y ago
        Now that the main maintainer quit, can those nontechnical reasons be resolved?
        jpgvm 12y ago
        Hopefully. I would be willing to take a crack at it.
        vidarh 12y ago
        If you (or anyone else) do, I'd be ready to help.
      - mnarayan01 12y ago
        > It is de facto a more complete test suite than the ad hoc one they've been making. This is a _huge_ stretch. It's hardly surprising that the suite of tests MRI is running against aren't failing against a _released_ version.
    - click170 12y ago
      I see what you're saying, but this isn't about using any one person's pet project. This discussion would be moot if MRI Devs were using a different competing third-party Ruby-Spec like tool, but they aren't. They wrote their own tools which contain documented bugs and flaws which they are not interested in fixing, all the while a much better tool exists that they refuse to use.
    - MBCook 12y ago
      That's basically how I read this. It seems that Ruby is specified de facto by what MRI does. Reading this blog post it sounds like the developer of RubySpec did this: 1. Decided Ruby needed more formal specification 2. Made their own formal specification 3. Kept telling the core developers to use it 4. Is giving up after years of the core developers not bending to the way he thinks that Ruby should work The whole thing just read as kind of arrogant to me. Why should the ruby developers have to follow his specification? The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process. Instead of saying "you should be using my test", you could commit mew tests and patches to upstream. If some of the existing tests or two large or don't test only one function, why not submit a patch to fix those issues. That doesn't mean the entire test week needs to be thrown out and the language respecified. This seemed more like someone trying to force themselves into a position and giving up after a few years.
      - masklinn 12y ago
        > 1. Decided Ruby needed more formal specification 2. Made their own formal specification RubySpec isn't a formal specification in any way, shape or form. RubySpec is an implementation test suite, and for MRI a regression suite. It uses language constructs and expects a behaviour A based on Ruby version B (which is itself assumed to match MRI version B). RubySpec isn't somebody thinking up his own definition of Ruby (let alone doing so formally), it's somebody encoding MRI behaviour into a test suite because he needed that to code his own Ruby implementation. > Why should the ruby developers have to follow his specification? There's nothing to follow, there's a test suite to run. A test suite which encodes existing MRI behaviour. Why should ruby developers run it? Because it catches bugs is a pretty good reason. > The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process. How is segfaulting on existing code not an indictment of the core process when just running an existing test suite would have caught it? > Instead of saying "you should be using my test", you could commit mew tests and patches to upstream. Have you missed the laundry sheet of issues in the existing MRI testing system?
      - vidarh 12y ago
        The problem with "MRI as spec" is that nobody really knows how the language is defined. E.g. large parts of the 1.8.x series behaved differently from what most people thought it did in terms of the bootstrapping of inheritance of core classes. I wrote a blog post about Ruby behaviour based on introspecting the 1.8.6 interpreter that some people insisted was wrong because it worked differently than intended. Yet the behaviour persisted for years until it silently changed at some point (I don't know which revision). In that case it was no big deal since the reason the real behaviour was pretty much unknown was that nobody depended on the behaviour, but from what I've seen this is fairly common with MRI. As a "spec" it is one of the most unstable environments I've worked with. Though I love Ruby the language, MRI is a problem as the default implementation and atrocious as a "spec". I frankly hope one of the other implementations gains enough traction to force the core team to the "negotiating table" over a proper specification.
    - krschultz 12y ago
      RubySpec is nothing more than Ruby code doing things that worked in the previous version. Whenever it breaks, that is a regression in MRI. Arguably it's no different than something breaking in your current Ruby program, and after spending a bunch of time debugging you realize it is a problem in MRI. If that happens, you would file a bug against MRI. RubySpec just makes it more apparent.
joevandyk 12y ago
Why don't MRI developers use RubySpec?
- nateberkopec 12y ago
  As linked in another comment, basically: https://bugs.ruby-lang.org/issues/7549#note-2 https://bugs.ruby-lang.org/issues/7549#note-2 1. Matz is, and always will be BDFL. 2. The core Ruby team wants to talk in terms of C and MRI being the Ruby spec, not a third party/written RubySpec
  - Tobu 12y ago
    They could always have their own fork/branch for upcoming releases of Ruby. It's not like the spec has to freeze the language.
  - vidarh 12y ago
    That's an absolutely terrifying attitude from a language developer....
    - nateberkopec 12y ago
      Is it? Python is exactly the same. It has no language spec (defined by its implementatation), and it has a BDFL who hasn't pushed for one.
      - nedbat 12y ago
        It would be good for Python to have more rigor in its definition. But the situation here doesn't have an analog in the Python world: there is no "PythonSpec" third-party test suite that is a) more complete than the official suite, b) exposing segfaults in previously working code, and c) going unused by the core developers.
      - vidarh 12y ago
        That other languages are equally badly specified is not a good argument for why it's not terrifying. Then again, I don't care about Python. I do care about Ruby.
sams99 12y ago
The thing I find depressing and frustrating about this kind of discussion is the fatalism and non-constructiveness. I really wish it was: "I set up a server that runs ruby spec on ruby-head daily and automatically reports spec failures to ruby-bugs" So many companies are making big bucks off Ruby, yet so little are willing to fork out a bit of money and time to make Ruby better.
- zem 12y ago
  it's unreasonable to expect the dev who has already put a ton of work into developing and maintaining rubyspec and making it work nicely with all the ruby implementations and versions he could to have to put in the additional work to do this as well. I don't blame him for getting discouraged at having to chase a moving target on top of all that - I agree with him that having the MRI devs contribute to rubyspec was a reasonable thing to expect.
  - sams99 12y ago
    Throwing away a project that is clearly working is depressing. can't Heroku/GitHub/37 signals or someone sponsor a week of dev time to automate a system that runs the suite on latest? It's just such a better outcome, nobody needs to change workflows its just that information gets reported upstream earlier in a much more useful time.
    - mahmoudimus 12y ago
      The Ruby devs attitudes in the discussions linked FTA makes me question the longevity of Ruby as a serious Enterprise language.
      - manyxcxi 12y ago
        When did it become a serious Enterprise™ language? Not being flippant or denigrating Ruby, but when I do work for established enterprises, it's never in Ruby. Java, .NET, PHP, and occasionally Python. Just because startups use it to get off the ground quickly, or to build the front end of their website doesn't make it enterprise.
Terr_ 12y ago
Regardless of other warts with the language, I really love how the Java Language Specification lays down the law.
- mbell 12y ago
  I find Java's approach to specs to be one of the worst parts of the ecosystem. I've come across numerous obvious bugs with various Java specced libraries that have been marked `won't fix` because the bug is actually either a bug in the spec or just got left out in order to rush the spec out the door (rush being a loose term here given how often Java actually updates it's various specs).
  - sinistersnare 12y ago
    At least the bugs are documented. Ruby bugs just are, theres nothing confirming them. Java bugs need to stay there because if they were fixed then it would be a backward incompatible change.
    - mbell 12y ago
      That attitude is exactly why I have no interest in working with Java.
      - mindcrime 12y ago
        Wait, you're saying you'd prefer undocumented bugs and a constant stream of breaking changes to the runtime that force you to constantly port and report existing, working code? I don't get it... the Java way seems pretty right to me. That said, I will allow that they might have been a little bit too rigid about not allowing breaking changes. At the very least this policy has resulted in Java evolving more slowly than it might have otherwise. But I don't necessarily find that to be unacceptable, all things told.
        mbell 12y ago
        I prefer that bugs get fixed when found. In my experience the Java way is 'oh, a bug, well, that's a bug defined in the spec so we can't fix it now, we'll make a note for the next version'...5 years pass. The 'I don't want to touch working code' argument is another strange Javaism. If you expect your code to not rot your going to have to constantly maintain it anyway.
dyeje 12y ago
Some of the discussions he links to in the article are disturbing. Many posts refusing to implement process because they don't want any process at all. Doesn't seem like a healthy approach.
- headius 12y ago
  Be wary of this cultivated list of links. There are others out there that would make it very clear why the proposed design process was a misfit given the projects and people involved.
hartator 12y ago
Has Matz stated the reasons they are not using RubySpec? Knowing him, I can't really believe he has chosen to ignore RubySpec without good reasons.
- drivingmenuts 12y ago
  https://bugs.ruby-lang.org/issues/7549#note-2 https://bugs.ruby-lang.org/issues/7549#note-2 Apparently, he and some of the key devs don't like design by committee or much in the way of bureaucracy.
  - click170 12y ago
    Matz: community members can deprive of the power of the dictator, when he apparently loose his ability to make rational decision. After deprivation, the community will make up new rules, hopefully flexible ones. I don't want that situation, but the day will come sooner of later I think that's a very disappointing point of view, and I say that as a someone who used Ruby in the past. In light of this I can say it will not be my go-to language in the future. I feel like after a certain point of community uptake, a project surpasses any single developer and becomes a part of that community in and of itself. I understand wanting to maintain control of your pet project, but for community projects I feel like there should be community control. That said, you should have the freedom and the right to do whatever you want with your pet project. Maybe the solution is for the community to fork so they can make the changes they see fit. Given the way the MRI devs and Matz in particular have responded, perhaps it's time for the community to fork Ruby if they don't like the way Matz insists on remaining the dictator. It's his project and he can do what he wants with it, but we don't have to use it.
    - drivingmenuts 12y ago
      I am of the opinion that, while he took advice and help from the larger Ruby community, he never considered it a "community" project in the sense of communities surrounding other languages.
  - Sivart13 12y ago
nateberkopec 12y ago
Rubyspec is a great project for all the reasons Brian outlines. However, it's also a failure - in part, due to Brian and his attitude towards contributors. See this twitter conversation: https://twitter.com/the_zenspider/status/547527644535726080 https://twitter.com/the_zenspider/status/547527644535726080 He's been grinding on Rubyspec for years, bless him, but I think there's a reason why he was unable to rally the community behind his effort, both in terms of gathering more contributors and in making it "official" in terms of the language spec. JRuby, currently the only non-MRI ruby implementation that you can seriously consider for production use, runs the MRI test suite against JRuby. RubySpec is far from The Only Solution, though Brian would like you to think it is.
- headius 12y ago
  I came to the thread to make sure someone was spreading truth rather than FUD, and this was the first comment I saw. Bless you.
- jfaucett 12y ago
  "RubySpec is far from The Only Solution" you're right, but I remember learning quit a bit of ruby by working through RubySpec back in the day. and I remember liking how well structured, logical and organized the suite was, very exemplary. so I'm sorry to see it go. I have no perspective on Brian or the mri devs so I can't say anything about the situation around it, I just liked the code and spec and found it very well done. It would certainly be nice to have such a ruby specification and test suite that was endorsed by ruby community at large (unlike what I saw in those example MRI tests).
- btown 12y ago
  > JRuby, currently the only non-MRI ruby implementation that you can seriously consider for production use Are there concrete shortcomings when using Rubinius in production, or is it just not battle-tested yet?
  - cheald 12y ago
    Every time I've tried to use it, it's either been unstable (segfaults are fun) or orders of magnitude slower than MRI. It's a promising project, but as far as "no GIL, true concurrency, mature GC" implementations go, JRuby delivers the goods. Edit: For kicks, I'm trying to boot my primary app with RBX right now. RTLK threw some "id for nil" exceptions due to some initializer stuff, so I disabled that, and now, trying to run my test suite, after 2 minutes of sitting there doing who-knows-what, it finally attempts to boot my app and dies with a "Missing constant" error despite the fact that I put in debug code to ensure that the file containing said module is a) loaded, b) executing properly, and c) that said constant actually exists in the module tree. This code boots and runs just fine under MRI 1.9, 2.0, 2.1, and JRuby 1.7. I might be able to get it running with some more work and explicit load path management, but I'm off to a party for now, so I'll poke at it a bit when I get back.
jagawhowho 12y ago
Ruby is about to die permanently. A language that copied emacs-lisp but with syntax? Lol that gives it features to impress the ignoramus majority but real Haskell or Lisp programmers know better.
- deleted 12y ago
  [deleted]
jes5199 12y ago
I almost didn't catch the epilogue - he's shutting down the RubySpec project entirely, because it hasn't accomplished what he hoped.
- headius 12y ago
  I believe he's shutting it down as a political move. They're still going to use it to maintain Ruby compatibility and they'll still need to run it against MRI. It just isn't a standalone project now.
lgleason 12y ago
Both Charles Nutter and Matz are smart guys. They are also both great people. I say this having spent time with both of them. This whole, my implementation is better than theirs coming from Brian is crazy. Matz and Charles have helped to set the tone for the community. While I appreciate Brian's passion, it sounds like he needs to check his ego. No matter how smart we are, we can always learn things from other people. Without the collaboration of others neither JRuby or Ruby would be what they are today which is why they are successful.
- vidarh 12y ago
  He might need to check his ego, but the state of Ruby specification is deplorable, and despite being a big fan of Ruby, it's also a big road block to development. The small number of mature Ruby runtimes is a bad sign.
  - whistlerbrk 12y ago
    See, iirc, Matz years ago specifically said "fork Ruby". He wants lots of different versions which I feel to some extent means diverging from a formal spec, and then occasionally maybe coming back to it.
    - vidarh 12y ago
      And that's even worse. I'm working on a Ruby compiler. While it is far from being at a stage where this is a real problem, currently there's no sane way of knowing whether or not I'm "close enough" to MRI other than extensive testing of every single piece of Ruby code I want to work. "Diverging from a formal spec" is one thing. As it is, many versions of MRI has diverged from how the core team thinks it works. If MRI is "the spec", then Ruby changes from revision to revision as the test coverage is nowhere good enough to prevent regressions or changes in behaviour. Basically: Nobody knows Ruby. Frankly, I very much hope that another Ruby implementations overtake MRI sufficiently in quality to out compete it enough to shift the initiative of definining the language to a more responsible team.
- deleted 12y ago
  [deleted]
lnanek2 12y ago
Matz is the creator of Ruby. If this guy really wanted to play ball, why didn't he just submit PRs for improving the real Ruby's test suite? Instead he just made his own thing, well duh it didn't end up a part of the real Ruby. OK, a hosting provider wanted their own Ruby implementation to fix concurrency issues, but that doesn't suddenly make the creator of the language have to do things their way. If you fork or reimplement a project and do whatever you want, well you took control so you gained something, but the creator has no obligation to change to your fork/reimplementation.
- krschultz 12y ago
  That glosses over all of the arguments the OP made about making a test suite that can work with all implementations of Ruby.
tessierashpool 12y ago
I don't want to get in the middle of drama, but if you take anything away from this, read the actual code. Just read the actual code of the MRI tests.
- deleted 12y ago
  [deleted]
msie 12y ago
What of the ISO standard that mRuby is based on? http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=59579 http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_... Aw crap, I have to buy this doc to see it?
- deleted 12y ago
  [deleted]
- cremno 12y ago
  The final draft is available: https://www.ipa.go.jp/osc/english/ruby/ https://www.ipa.go.jp/osc/english/ruby/ But the ISO document is mostly irrelevant, if we talk about Ruby implementations like CRuby, JRuby, or Rubinius.
bhrgunatha 12y ago
> Later that year, at RubyConf 2008, I gave a talk titled, What Does My Ruby Do about RubySpec. Matz and several other MRI developers attended. Immediately after my talk, contributors to Rubinius sat down with Matz and other MRI developers to discuss their effort to create an ISO specification for Ruby. We asked whether RubySpec could be part of the specification but were told that it was not appropriate to include it. This seems telling. Does anyone have any concrete information about why Matz and the Ruby team are opposed to using RubySpec then? Has there been any progress on creating an ISO spec? EDIT: It seems Ruby has a published ISO spec since April 2012 - http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=59579 http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...
- deleted 12y ago
  [deleted]
vorg 12y ago
It's disappointing to hear about such problems with Ruby. I hit the same problem with the Groovy Language spec when I first came across Groovy. Its creator, James Strachan, initiated an implementation, test kit, and spec all within 6 months of each other (impl beta-1 in Dec 2003, and spec JSR-241 in May 2004). The project managers who took over from him, Graeme Rocher and Guillaume Laforge, changed direction by stopping work on the spec and refocusing the Groovy reference implementation to be the scripting language behind Grails. (Of course, Groovy 'n' Grails was intended to chisel away at some of the market share of Ruby on Rails but that's another story.) Strachan often wrote that the spec was to enable anyone to make their own implementation of Groovy if they want to, and right up to his very last posting ever http://groovy.329449.n5.nabble.com/Paris-write-up-tt395560.html#a395571 http://groovy.329449.n5.nabble.com/Paris-write-up-tt395560.h... on the Groovy mailing list on 5 Dec 2005, he maintained that what they were building was the reference implementation. If Rocher and Laforge had come clean about how they turned the RI into the language itself, the backlash might have blown over quickly, but instead they led developers along for many years afterwards, not changing the spec to dormant until April 2012. Projects other than Grails who've tried to build atop Groovy have had to risk the ref impl changing in breaking ways between versions. The most spectacular incident was when Groovy++, an experimental static compiler built by Alex Tkachman that hooked via annotations into Groovy's AST, had to drop back down from Groovy 1.8 to 1.7 in 2011, and my own side project was also affected by the change. It turned out Rocher and Laforge had secretly employed a mate to extend Groovy with the exact same static type-checking and compilation functionality as Groovy++ and were obviously trying to shake us off. Unlike Ruby, Groovy only has one other implementation, GrooScript, built by Jorge Franco, which generates JavaScript from Groovy syntax. When the developers of the most used implementation of a language want to protect their control, it certainly does hurt the ecosystem, turning it into an "echo system".
- marktangotango 12y ago
  Are you sure you're not assigning to malice that which could be due to ignorance or indifference?
  - vorg 12y ago
senthilnayagam 12y ago
@senthilnayagam: . @yukihiro_matz can you respond http://t.co/qeeVAluOhJ http://t.co/qeeVAluOhJ @yukihiro_matz: @senthilnayagam I am not in charge of testing. But as far as I understand it has been communication problems. Blaming no use. @senthilnayagam: . @yukihiro_matz when merb could merge with rails, rubyspec shpuld merge with MRI & become official reference for all Ruby implementations
issaria 12y ago
THIS Brian again, nobody uses rubinius and it doesn't hurt anyone. This guy has a long history of promoting rubinius by attacking MRI, first time I saw him was on Baruco 2013, if you watch the video, his attitude is so annoying. Evan Phoenix's Ruby implementation was an interesting project, but now totally ruined by this childish guy. Brian, my advise is to stop bitching about MRI and seriously improve the documentation of rubinius, the are many interesting topics on rubini.us, but most of them are WIP since forever.
crb002 12y ago
MRI unit tests are the spec. https://github.com/ruby/ruby/tree/trunk/test/ruby https://github.com/ruby/ruby/tree/trunk/test/ruby
grover_hartmann 12y ago
Fuck, this idiot is now being pendatic in #ruby as well. 2015-01-02 14:45:30 brixen raise your hand if you have implemented Ruby and used MRI's tests Glad he didn't get things to go his way. He cries too much. If you don't like some projects go write your fucking own.

291 points by jc00ke 12y ago

jc00ke 12y ago

I mistakenly cut off the full title of the post: "Matz's Ruby Developers Don't Use RubySpec and It's Hurting Ruby"

serve_yay 12y ago

Jeez, what a mess.

danielweber 12y ago

Why should they have been using RubySpec?

joevandyk 12y ago
Did you read the article? It contains 10 reasons why the MRI tests aren't sufficient.
- danielweber 12y ago
  The Ruby devs also doesn't use any libraries I wrote, and that's okay. Why is RubySpec the special sauce that is so special that the devs not using it is horrible?
  - joevandyk 12y ago
    Because RubySpec is the best specification for Ruby that exists. By far.
  - masklinn 12y ago
    > Why is RubySpec the special sauce that is so special that the devs not using it is horrible? Assuming TFA isn't actively lying[0]: because it fixes the issues outlined which allow (amongst other things) cross-implementation and cross-version uses, because it tests behaviours not tested by the MRI suite anyway and because it's there as an extensive test suite for the language. Oh, and because according to TFA it found regressions in just about every release it's existed for. Regressions which could thus have been caught before the release was cut. [0] and I don't see any reason to assume otherwise, quite the opposite.
    - headius 12y ago
      I wouldn't say TFA is lying but it is presenting only part of the truth. MRI does have an actively developed test suite, which they run in CI and which we use to test compatibility in JRuby. MRI has contributed to RubySpec in the past and many contributors I know stopped (like I did) for nontechnical reasons. They are not ignoring it, but it is a hard project to collaborate on given the attitudes of the project lead that should be evident in this post.
  - stormbrew 12y ago
    Honestly, just the fact that rubyspec exposes a segfault in a new release is more than reason enough that they should be using it. It is de facto a more complete test suite than the ad hoc one they've been making. Given that, the onus is on the MRI developers to demonstrate why they are actively avoiding the use of a tool that could improve the stability of their language and environment.
    - headius 12y ago
      They have used and contributed to RubySpec in the past, but many (like me) were turned off by the maintainers' attitudes toward contributors and lack of respect. For example, see the zenspider link elsewhere in this thread. Nobody questions RubySpec as a project. But there are many nontechnical reasons why it was never adopted wholeheartedly by ruby-core.
      - joevandyk 12y ago
        Now that the main maintainer quit, can those nontechnical reasons be resolved?
        jpgvm 12y ago
        Hopefully. I would be willing to take a crack at it.
        vidarh 12y ago
        If you (or anyone else) do, I'd be ready to help.
    - mnarayan01 12y ago
      > It is de facto a more complete test suite than the ad hoc one they've been making. This is a _huge_ stretch. It's hardly surprising that the suite of tests MRI is running against aren't failing against a _released_ version.
  - click170 12y ago
    I see what you're saying, but this isn't about using any one person's pet project. This discussion would be moot if MRI Devs were using a different competing third-party Ruby-Spec like tool, but they aren't. They wrote their own tools which contain documented bugs and flaws which they are not interested in fixing, all the while a much better tool exists that they refuse to use.
  - MBCook 12y ago
    That's basically how I read this. It seems that Ruby is specified de facto by what MRI does. Reading this blog post it sounds like the developer of RubySpec did this: 1. Decided Ruby needed more formal specification 2. Made their own formal specification 3. Kept telling the core developers to use it 4. Is giving up after years of the core developers not bending to the way he thinks that Ruby should work The whole thing just read as kind of arrogant to me. Why should the ruby developers have to follow his specification? The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process. Instead of saying "you should be using my test", you could commit mew tests and patches to upstream. If some of the existing tests or two large or don't test only one function, why not submit a patch to fix those issues. That doesn't mean the entire test week needs to be thrown out and the language respecified. This seemed more like someone trying to force themselves into a position and giving up after a few years.
    - masklinn 12y ago
      > 1. Decided Ruby needed more formal specification 2. Made their own formal specification RubySpec isn't a formal specification in any way, shape or form. RubySpec is an implementation test suite, and for MRI a regression suite. It uses language constructs and expects a behaviour A based on Ruby version B (which is itself assumed to match MRI version B). RubySpec isn't somebody thinking up his own definition of Ruby (let alone doing so formally), it's somebody encoding MRI behaviour into a test suite because he needed that to code his own Ruby implementation. > Why should the ruby developers have to follow his specification? There's nothing to follow, there's a test suite to run. A test suite which encodes existing MRI behaviour. Why should ruby developers run it? Because it catches bugs is a pretty good reason. > The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process. How is segfaulting on existing code not an indictment of the core process when just running an existing test suite would have caught it? > Instead of saying "you should be using my test", you could commit mew tests and patches to upstream. Have you missed the laundry sheet of issues in the existing MRI testing system?
    - vidarh 12y ago
      The problem with "MRI as spec" is that nobody really knows how the language is defined. E.g. large parts of the 1.8.x series behaved differently from what most people thought it did in terms of the bootstrapping of inheritance of core classes. I wrote a blog post about Ruby behaviour based on introspecting the 1.8.6 interpreter that some people insisted was wrong because it worked differently than intended. Yet the behaviour persisted for years until it silently changed at some point (I don't know which revision). In that case it was no big deal since the reason the real behaviour was pretty much unknown was that nobody depended on the behaviour, but from what I've seen this is fairly common with MRI. As a "spec" it is one of the most unstable environments I've worked with. Though I love Ruby the language, MRI is a problem as the default implementation and atrocious as a "spec". I frankly hope one of the other implementations gains enough traction to force the core team to the "negotiating table" over a proper specification.
  - krschultz 12y ago
    RubySpec is nothing more than Ruby code doing things that worked in the previous version. Whenever it breaks, that is a regression in MRI. Arguably it's no different than something breaking in your current Ruby program, and after spending a bunch of time debugging you realize it is a problem in MRI. If that happens, you would file a bug against MRI. RubySpec just makes it more apparent.

joevandyk 12y ago

Why don't MRI developers use RubySpec?

nateberkopec 12y ago
As linked in another comment, basically: https://bugs.ruby-lang.org/issues/7549#note-2 https://bugs.ruby-lang.org/issues/7549#note-2 1. Matz is, and always will be BDFL. 2. The core Ruby team wants to talk in terms of C and MRI being the Ruby spec, not a third party/written RubySpec
- Tobu 12y ago
  They could always have their own fork/branch for upcoming releases of Ruby. It's not like the spec has to freeze the language.
- vidarh 12y ago
  That's an absolutely terrifying attitude from a language developer....
  - nateberkopec 12y ago
    Is it? Python is exactly the same. It has no language spec (defined by its implementatation), and it has a BDFL who hasn't pushed for one.
    - nedbat 12y ago
      It would be good for Python to have more rigor in its definition. But the situation here doesn't have an analog in the Python world: there is no "PythonSpec" third-party test suite that is a) more complete than the official suite, b) exposing segfaults in previously working code, and c) going unused by the core developers.
    - vidarh 12y ago
      That other languages are equally badly specified is not a good argument for why it's not terrifying. Then again, I don't care about Python. I do care about Ruby.

sams99 12y ago

The thing I find depressing and frustrating about this kind of discussion is the fatalism and non-constructiveness. I really wish it was: "I set up a server that runs ruby spec on ruby-head daily and automatically reports spec failures to ruby-bugs" So many companies are making big bucks off Ruby, yet so little are willing to fork out a bit of money and time to make Ruby better.

zem 12y ago
it's unreasonable to expect the dev who has already put a ton of work into developing and maintaining rubyspec and making it work nicely with all the ruby implementations and versions he could to have to put in the additional work to do this as well. I don't blame him for getting discouraged at having to chase a moving target on top of all that - I agree with him that having the MRI devs contribute to rubyspec was a reasonable thing to expect.
- sams99 12y ago
  Throwing away a project that is clearly working is depressing. can't Heroku/GitHub/37 signals or someone sponsor a week of dev time to automate a system that runs the suite on latest? It's just such a better outcome, nobody needs to change workflows its just that information gets reported upstream earlier in a much more useful time.
  - mahmoudimus 12y ago
    The Ruby devs attitudes in the discussions linked FTA makes me question the longevity of Ruby as a serious Enterprise language.
    - manyxcxi 12y ago
      When did it become a serious Enterprise™ language? Not being flippant or denigrating Ruby, but when I do work for established enterprises, it's never in Ruby. Java, .NET, PHP, and occasionally Python. Just because startups use it to get off the ground quickly, or to build the front end of their website doesn't make it enterprise.

Terr_ 12y ago

Regardless of other warts with the language, I really love how the Java Language Specification lays down the law.

mbell 12y ago
I find Java's approach to specs to be one of the worst parts of the ecosystem. I've come across numerous obvious bugs with various Java specced libraries that have been marked `won't fix` because the bug is actually either a bug in the spec or just got left out in order to rush the spec out the door (rush being a loose term here given how often Java actually updates it's various specs).
- sinistersnare 12y ago
  At least the bugs are documented. Ruby bugs just are, theres nothing confirming them. Java bugs need to stay there because if they were fixed then it would be a backward incompatible change.
  - mbell 12y ago
    That attitude is exactly why I have no interest in working with Java.
    - mindcrime 12y ago
      Wait, you're saying you'd prefer undocumented bugs and a constant stream of breaking changes to the runtime that force you to constantly port and report existing, working code? I don't get it... the Java way seems pretty right to me. That said, I will allow that they might have been a little bit too rigid about not allowing breaking changes. At the very least this policy has resulted in Java evolving more slowly than it might have otherwise. But I don't necessarily find that to be unacceptable, all things told.
      - mbell 12y ago
        I prefer that bugs get fixed when found. In my experience the Java way is 'oh, a bug, well, that's a bug defined in the spec so we can't fix it now, we'll make a note for the next version'...5 years pass. The 'I don't want to touch working code' argument is another strange Javaism. If you expect your code to not rot your going to have to constantly maintain it anyway.

dyeje 12y ago

Some of the discussions he links to in the article are disturbing. Many posts refusing to implement process because they don't want any process at all. Doesn't seem like a healthy approach.

headius 12y ago
Be wary of this cultivated list of links. There are others out there that would make it very clear why the proposed design process was a misfit given the projects and people involved.

hartator 12y ago

Has Matz stated the reasons they are not using RubySpec? Knowing him, I can't really believe he has chosen to ignore RubySpec without good reasons.

drivingmenuts 12y ago
https://bugs.ruby-lang.org/issues/7549#note-2 https://bugs.ruby-lang.org/issues/7549#note-2 Apparently, he and some of the key devs don't like design by committee or much in the way of bureaucracy.
- click170 12y ago
  Matz: community members can deprive of the power of the dictator, when he apparently loose his ability to make rational decision. After deprivation, the community will make up new rules, hopefully flexible ones. I don't want that situation, but the day will come sooner of later I think that's a very disappointing point of view, and I say that as a someone who used Ruby in the past. In light of this I can say it will not be my go-to language in the future. I feel like after a certain point of community uptake, a project surpasses any single developer and becomes a part of that community in and of itself. I understand wanting to maintain control of your pet project, but for community projects I feel like there should be community control. That said, you should have the freedom and the right to do whatever you want with your pet project. Maybe the solution is for the community to fork so they can make the changes they see fit. Given the way the MRI devs and Matz in particular have responded, perhaps it's time for the community to fork Ruby if they don't like the way Matz insists on remaining the dictator. It's his project and he can do what he wants with it, but we don't have to use it.
  - drivingmenuts 12y ago
    I am of the opinion that, while he took advice and help from the larger Ruby community, he never considered it a "community" project in the sense of communities surrounding other languages.
- Sivart13 12y ago

nateberkopec 12y ago

Rubyspec is a great project for all the reasons Brian outlines. However, it's also a failure - in part, due to Brian and his attitude towards contributors. See this twitter conversation: https://twitter.com/the_zenspider/status/547527644535726080 https://twitter.com/the_zenspider/status/547527644535726080 He's been grinding on Rubyspec for years, bless him, but I think there's a reason why he was unable to rally the community behind his effort, both in terms of gathering more contributors and in making it "official" in terms of the language spec. JRuby, currently the only non-MRI ruby implementation that you can seriously consider for production use, runs the MRI test suite against JRuby. RubySpec is far from The Only Solution, though Brian would like you to think it is.

headius 12y ago
I came to the thread to make sure someone was spreading truth rather than FUD, and this was the first comment I saw. Bless you.
jfaucett 12y ago
"RubySpec is far from The Only Solution" you're right, but I remember learning quit a bit of ruby by working through RubySpec back in the day. and I remember liking how well structured, logical and organized the suite was, very exemplary. so I'm sorry to see it go. I have no perspective on Brian or the mri devs so I can't say anything about the situation around it, I just liked the code and spec and found it very well done. It would certainly be nice to have such a ruby specification and test suite that was endorsed by ruby community at large (unlike what I saw in those example MRI tests).
btown 12y ago
> JRuby, currently the only non-MRI ruby implementation that you can seriously consider for production use Are there concrete shortcomings when using Rubinius in production, or is it just not battle-tested yet?
- cheald 12y ago
  Every time I've tried to use it, it's either been unstable (segfaults are fun) or orders of magnitude slower than MRI. It's a promising project, but as far as "no GIL, true concurrency, mature GC" implementations go, JRuby delivers the goods. Edit: For kicks, I'm trying to boot my primary app with RBX right now. RTLK threw some "id for nil" exceptions due to some initializer stuff, so I disabled that, and now, trying to run my test suite, after 2 minutes of sitting there doing who-knows-what, it finally attempts to boot my app and dies with a "Missing constant" error despite the fact that I put in debug code to ensure that the file containing said module is a) loaded, b) executing properly, and c) that said constant actually exists in the module tree. This code boots and runs just fine under MRI 1.9, 2.0, 2.1, and JRuby 1.7. I might be able to get it running with some more work and explicit load path management, but I'm off to a party for now, so I'll poke at it a bit when I get back.

jagawhowho 12y ago

Ruby is about to die permanently. A language that copied emacs-lisp but with syntax? Lol that gives it features to impress the ignoramus majority but real Haskell or Lisp programmers know better.

deleted 12y ago
[deleted]

jes5199 12y ago

I almost didn't catch the epilogue - he's shutting down the RubySpec project entirely, because it hasn't accomplished what he hoped.

headius 12y ago
I believe he's shutting it down as a political move. They're still going to use it to maintain Ruby compatibility and they'll still need to run it against MRI. It just isn't a standalone project now.

lgleason 12y ago

Both Charles Nutter and Matz are smart guys. They are also both great people. I say this having spent time with both of them. This whole, my implementation is better than theirs coming from Brian is crazy. Matz and Charles have helped to set the tone for the community. While I appreciate Brian's passion, it sounds like he needs to check his ego. No matter how smart we are, we can always learn things from other people. Without the collaboration of others neither JRuby or Ruby would be what they are today which is why they are successful.

vidarh 12y ago
He might need to check his ego, but the state of Ruby specification is deplorable, and despite being a big fan of Ruby, it's also a big road block to development. The small number of mature Ruby runtimes is a bad sign.
- whistlerbrk 12y ago
  See, iirc, Matz years ago specifically said "fork Ruby". He wants lots of different versions which I feel to some extent means diverging from a formal spec, and then occasionally maybe coming back to it.
  - vidarh 12y ago
    And that's even worse. I'm working on a Ruby compiler. While it is far from being at a stage where this is a real problem, currently there's no sane way of knowing whether or not I'm "close enough" to MRI other than extensive testing of every single piece of Ruby code I want to work. "Diverging from a formal spec" is one thing. As it is, many versions of MRI has diverged from how the core team thinks it works. If MRI is "the spec", then Ruby changes from revision to revision as the test coverage is nowhere good enough to prevent regressions or changes in behaviour. Basically: Nobody knows Ruby. Frankly, I very much hope that another Ruby implementations overtake MRI sufficiently in quality to out compete it enough to shift the initiative of definining the language to a more responsible team.
deleted 12y ago
[deleted]

lnanek2 12y ago

Matz is the creator of Ruby. If this guy really wanted to play ball, why didn't he just submit PRs for improving the real Ruby's test suite? Instead he just made his own thing, well duh it didn't end up a part of the real Ruby. OK, a hosting provider wanted their own Ruby implementation to fix concurrency issues, but that doesn't suddenly make the creator of the language have to do things their way. If you fork or reimplement a project and do whatever you want, well you took control so you gained something, but the creator has no obligation to change to your fork/reimplementation.

krschultz 12y ago
That glosses over all of the arguments the OP made about making a test suite that can work with all implementations of Ruby.

tessierashpool 12y ago

I don't want to get in the middle of drama, but if you take anything away from this, read the actual code. Just read the actual code of the MRI tests.

deleted 12y ago
[deleted]

msie 12y ago

What of the ISO standard that mRuby is based on? http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=59579 http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_... Aw crap, I have to buy this doc to see it?

deleted 12y ago
[deleted]
cremno 12y ago
The final draft is available: https://www.ipa.go.jp/osc/english/ruby/ https://www.ipa.go.jp/osc/english/ruby/ But the ISO document is mostly irrelevant, if we talk about Ruby implementations like CRuby, JRuby, or Rubinius.

bhrgunatha 12y ago

> Later that year, at RubyConf 2008, I gave a talk titled, What Does My Ruby Do about RubySpec. Matz and several other MRI developers attended. Immediately after my talk, contributors to Rubinius sat down with Matz and other MRI developers to discuss their effort to create an ISO specification for Ruby. We asked whether RubySpec could be part of the specification but were told that it was not appropriate to include it. This seems telling. Does anyone have any concrete information about why Matz and the Ruby team are opposed to using RubySpec then? Has there been any progress on creating an ISO spec? EDIT: It seems Ruby has a published ISO spec since April 2012 - http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=59579 http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...

deleted 12y ago
[deleted]

vorg 12y ago

It's disappointing to hear about such problems with Ruby. I hit the same problem with the Groovy Language spec when I first came across Groovy. Its creator, James Strachan, initiated an implementation, test kit, and spec all within 6 months of each other (impl beta-1 in Dec 2003, and spec JSR-241 in May 2004). The project managers who took over from him, Graeme Rocher and Guillaume Laforge, changed direction by stopping work on the spec and refocusing the Groovy reference implementation to be the scripting language behind Grails. (Of course, Groovy 'n' Grails was intended to chisel away at some of the market share of Ruby on Rails but that's another story.) Strachan often wrote that the spec was to enable anyone to make their own implementation of Groovy if they want to, and right up to his very last posting ever http://groovy.329449.n5.nabble.com/Paris-write-up-tt395560.html#a395571 http://groovy.329449.n5.nabble.com/Paris-write-up-tt395560.h... on the Groovy mailing list on 5 Dec 2005, he maintained that what they were building was the reference implementation. If Rocher and Laforge had come clean about how they turned the RI into the language itself, the backlash might have blown over quickly, but instead they led developers along for many years afterwards, not changing the spec to dormant until April 2012. Projects other than Grails who've tried to build atop Groovy have had to risk the ref impl changing in breaking ways between versions. The most spectacular incident was when Groovy++, an experimental static compiler built by Alex Tkachman that hooked via annotations into Groovy's AST, had to drop back down from Groovy 1.8 to 1.7 in 2011, and my own side project was also affected by the change. It turned out Rocher and Laforge had secretly employed a mate to extend Groovy with the exact same static type-checking and compilation functionality as Groovy++ and were obviously trying to shake us off. Unlike Ruby, Groovy only has one other implementation, GrooScript, built by Jorge Franco, which generates JavaScript from Groovy syntax. When the developers of the most used implementation of a language want to protect their control, it certainly does hurt the ecosystem, turning it into an "echo system".

marktangotango 12y ago
Are you sure you're not assigning to malice that which could be due to ignorance or indifference?
- vorg 12y ago

senthilnayagam 12y ago

@senthilnayagam: . @yukihiro_matz can you respond http://t.co/qeeVAluOhJ http://t.co/qeeVAluOhJ @yukihiro_matz: @senthilnayagam I am not in charge of testing. But as far as I understand it has been communication problems. Blaming no use. @senthilnayagam: . @yukihiro_matz when merb could merge with rails, rubyspec shpuld merge with MRI & become official reference for all Ruby implementations

issaria 12y ago

THIS Brian again, nobody uses rubinius and it doesn't hurt anyone. This guy has a long history of promoting rubinius by attacking MRI, first time I saw him was on Baruco 2013, if you watch the video, his attitude is so annoying. Evan Phoenix's Ruby implementation was an interesting project, but now totally ruined by this childish guy. Brian, my advise is to stop bitching about MRI and seriously improve the documentation of rubinius, the are many interesting topics on rubini.us, but most of them are WIP since forever.

crb002 12y ago

MRI unit tests are the spec. https://github.com/ruby/ruby/tree/trunk/test/ruby https://github.com/ruby/ruby/tree/trunk/test/ruby

grover_hartmann 12y ago

Fuck, this idiot is now being pendatic in #ruby as well. 2015-01-02 14:45:30 brixen raise your hand if you have implemented Ruby and used MRI's tests Glad he didn't get things to go his way. He cries too much. If you don't like some projects go write your fucking own.