5 ms·

Fun Numbers - Ruby and Perl (Massive difference in core test suite size)

24 points by draegtun 18y ago

perplexes 18y ago
Code size does not imply functionality. Does Perl have more core functionality? Does Ruby have more expressiveness in its tests, or how they test? Are the Perl core tests testing the core libs, the language itself? Both? What are the Ruby tests doing? This benchmark, like most, is meaningless.
jeremymcanally 18y ago
Perhaps they should add RubySpec then come back with those numbers. IIRC, there's somewhere in the neighborhood of 50,000 assertions in there and growing everyday. Unlike Perl, Ruby has elected to make these specs an open community process usable across all implementations. Oops.
- wolverian 18y ago
  Does the regular Ruby use RubySpec when you run "make test" or such? Perl 6 does separate the specification from the implementations.
  - AndrewO 18y ago
    Yes. http://svn.ruby-lang.org/repos/ruby/trunk/Makefile.in http://svn.ruby-lang.org/repos/ruby/trunk/Makefile.in Find "test-rubyspec". It even `git clone`s rubyspec if you ask it to (update-rubyspec).
    - wolverian 18y ago
      Thanks, good to know.
      - AndrewO 18y ago
        Now, granted early 1.8's tests were sparse. RubySpec is a by-product of the alternative implementations and a couple of test attempts (http://blog.headius.com/2008/05/rubyspec-bringing-ruby-test-suites.html http://blog.headius.com/2008/05/rubyspec-bringing-ruby-test-...). A lot of developers continue to use MRI 1.8 despite 1.9.1 being out due to API changes. So you could still make the case that the main interpreter being used was at one point not well tested. However, forward progress in the language is tested, so there's nothing really to be remedied.
mechanical_fish 18y ago
I doubt that Ruby is so feature poor to justify Perl having more than 50 tests for every 1 of Ruby's. Perhaps Ruby isn't "feature poor". Perhaps Perl is "edge-case rich". The language is rather complex. Browsing at random through the dreaded Camel Book, here are some things I find, in no particular order: a goto statement with three forms (page 105), typeglobs (pg 115), explicit syntax for references (pg 116), a text-formatting mechanism in core (pg 121), expressions that evaluate completely differently in "scalar context" vs "list context" (pg 45), taint mode (pg 357)... A more fundamental question is: Where do these numbers come from? And an even more fundamental question is: Since when did counting tests prove anything? Isn't that just like counting lines of code, or counting the number of bugs that have had to be fixed? (Give me a day or two and I'll double the raw number of Ruby's tests! I'll just add more assertions!)
- jeremymcanally 18y ago
  The difference in how values are treated alone is enough to quintuple the test cases. :)
rfunduk 18y ago
Talk about flame-bait. I'm not sure who they're arguing with here. Developers that write code in Ruby have a testing culture... I've never heard anyone imply that Matz and the core development team care even slightly about it. In short, your average Ruby project and Ruby the project are entirely unrelated, they're written in different languages, even!
- raganwald 18y ago
  I took the line Hello, core Ruby developers! You've got an awesome language there. Could you start testing, please? Your community seems to do OK. Why not you? to suggest that the author of the post is fully aware that the Ruby Core team and the Community of Ruby Users are two separate things with different attitudes towards testing.
  - codeodor 18y ago
    I agree with you, but if you get detailed, you might also question the use of "culture" as in "who has the testing culture again?" to be more indicative of Ruby developers as a community (which is a word the author also uses in praise of it) I think the words were poorly chosen. I understand the main point, which IMO supports your assertion. But knowing the habits of people surfing the web, I can certainly understand how it doesn't appear that way to everyone.
mc 18y ago
This is the second post I've read claiming that Perl is better tested than Ruby. Why is this a topic of discussion? Is someone from the Ruby community running around saying otherwise?
RobertL 18y ago
The object is to make apps that customers want and to make them fast. If you can do that with minimal testing then I "advantage Ruby". Another way to put it is "Uhh... guys... you don't get paid for testing"

24 points by draegtun 18y ago

Code size does not imply functionality. Does Perl have more core functionality? Does Ruby have more expressiveness in its tests, or how they test? Are the Perl core tests testing the core libs, the language itself? Both? What are the Ruby tests doing? This benchmark, like most, is meaningless.

jeremymcanally 18y ago

Perhaps they should add RubySpec then come back with those numbers. IIRC, there's somewhere in the neighborhood of 50,000 assertions in there and growing everyday. Unlike Perl, Ruby has elected to make these specs an open community process usable across all implementations. Oops.

wolverian 18y ago
Does the regular Ruby use RubySpec when you run "make test" or such? Perl 6 does separate the specification from the implementations.
- AndrewO 18y ago
  Yes. http://svn.ruby-lang.org/repos/ruby/trunk/Makefile.in http://svn.ruby-lang.org/repos/ruby/trunk/Makefile.in Find "test-rubyspec". It even `git clone`s rubyspec if you ask it to (update-rubyspec).
  - wolverian 18y ago
    Thanks, good to know.
    - AndrewO 18y ago
      Now, granted early 1.8's tests were sparse. RubySpec is a by-product of the alternative implementations and a couple of test attempts (http://blog.headius.com/2008/05/rubyspec-bringing-ruby-test-suites.html http://blog.headius.com/2008/05/rubyspec-bringing-ruby-test-...). A lot of developers continue to use MRI 1.8 despite 1.9.1 being out due to API changes. So you could still make the case that the main interpreter being used was at one point not well tested. However, forward progress in the language is tested, so there's nothing really to be remedied.

mechanical_fish 18y ago

I doubt that Ruby is so feature poor to justify Perl having more than 50 tests for every 1 of Ruby's. Perhaps Ruby isn't "feature poor". Perhaps Perl is "edge-case rich". The language is rather complex. Browsing at random through the dreaded Camel Book, here are some things I find, in no particular order: a goto statement with three forms (page 105), typeglobs (pg 115), explicit syntax for references (pg 116), a text-formatting mechanism in core (pg 121), expressions that evaluate completely differently in "scalar context" vs "list context" (pg 45), taint mode (pg 357)... A more fundamental question is: Where do these numbers come from? And an even more fundamental question is: Since when did counting tests prove anything? Isn't that just like counting lines of code, or counting the number of bugs that have had to be fixed? (Give me a day or two and I'll double the raw number of Ruby's tests! I'll just add more assertions!)

jeremymcanally 18y ago
The difference in how values are treated alone is enough to quintuple the test cases. :)

rfunduk 18y ago

Talk about flame-bait. I'm not sure who they're arguing with here. Developers that write code in Ruby have a testing culture... I've never heard anyone imply that Matz and the core development team care even slightly about it. In short, your average Ruby project and Ruby the project are entirely unrelated, they're written in different languages, even!

raganwald 18y ago
I took the line Hello, core Ruby developers! You've got an awesome language there. Could you start testing, please? Your community seems to do OK. Why not you? to suggest that the author of the post is fully aware that the Ruby Core team and the Community of Ruby Users are two separate things with different attitudes towards testing.
- codeodor 18y ago
  I agree with you, but if you get detailed, you might also question the use of "culture" as in "who has the testing culture again?" to be more indicative of Ruby developers as a community (which is a word the author also uses in praise of it) I think the words were poorly chosen. I understand the main point, which IMO supports your assertion. But knowing the habits of people surfing the web, I can certainly understand how it doesn't appear that way to everyone.

mc 18y ago

This is the second post I've read claiming that Perl is better tested than Ruby. Why is this a topic of discussion? Is someone from the Ruby community running around saying otherwise?

RobertL 18y ago

The object is to make apps that customers want and to make them fast. If you can do that with minimal testing then I "advantage Ruby". Another way to put it is "Uhh... guys... you don't get paid for testing"