6 ms·

201 failing tests is really shocking to me. Does anyone know of similar pages for languages such as ruby or python?

mitchty 15y ago
http://rubyspec.org/ http://rubyspec.org/ is the language "spec" that most ruby implementations of note, jruby/rubinius/mri, use to test that they are compliant. I couldn't find a page with automatic tests running against each implementations head/master branch. But I do see crypt() like specs in there: https://github.com/rubyspec/rubyspec.git https://github.com/rubyspec/rubyspec.git
skymt 15y ago
Python's test page: http://python.org/dev/buildbot/all/waterfall http://python.org/dev/buildbot/all/waterfall And here's one with just Python 2.7 on high-priority platforms: http://www.python.org/dev/buildbot/all/waterfall?category=2.7.stable http://www.python.org/dev/buildbot/all/waterfall?category=2....
judofyr 15y ago
Here's the CI for MRI (the major Ruby implementation): http://59.106.172.211/~chkbuild/ruby-trunk/recent.html http://59.106.172.211/~chkbuild/ruby-trunk/recent.html
jrockway 15y ago
Perl will not install if any of its tests fail after it's built. I'm surprised that PHP's test suite isn't run on the user's box after building it, allowing end users to "double check" the release process.
- shabble 15y ago
  I'd imagine the majority of people are installing it via their distro's binary packages, so they're not going to run the tests anyway. Of course, the packager for the distro should be running tests, and raising some serious questions when stuff fails.
- maratd 15y ago
  In fact, it is. Just type make test. There are thousands of unit tests and it takes a long time. I generally run it and expect a 98% passing rate. If it's below that, I might have an issue. One or two failing tests will never get noticed.
ars 15y ago
It seems like they add tests for bugs before actually fixing the bugs. I can see that being useful, but they need to do a better job of marking which bugs are ignorable and which are not. This is a an open source project - anyone want to volunteer?
- arthurschreiber 15y ago
  Just because it's an open source project does not mean that your contributions will be happily accepted.
  - elliottcarlson 15y ago
    Nor does that mean you can't try.

http://rubyspec.org/ http://rubyspec.org/ is the language "spec" that most ruby implementations of note, jruby/rubinius/mri, use to test that they are compliant. I couldn't find a page with automatic tests running against each implementations head/master branch. But I do see crypt() like specs in there: https://github.com/rubyspec/rubyspec.git https://github.com/rubyspec/rubyspec.git

skymt 15y ago

Python's test page: http://python.org/dev/buildbot/all/waterfall http://python.org/dev/buildbot/all/waterfall And here's one with just Python 2.7 on high-priority platforms: http://www.python.org/dev/buildbot/all/waterfall?category=2.7.stable http://www.python.org/dev/buildbot/all/waterfall?category=2....

judofyr 15y ago

Here's the CI for MRI (the major Ruby implementation): http://59.106.172.211/~chkbuild/ruby-trunk/recent.html http://59.106.172.211/~chkbuild/ruby-trunk/recent.html

jrockway 15y ago

Perl will not install if any of its tests fail after it's built. I'm surprised that PHP's test suite isn't run on the user's box after building it, allowing end users to "double check" the release process.

shabble 15y ago
I'd imagine the majority of people are installing it via their distro's binary packages, so they're not going to run the tests anyway. Of course, the packager for the distro should be running tests, and raising some serious questions when stuff fails.
maratd 15y ago
In fact, it is. Just type make test. There are thousands of unit tests and it takes a long time. I generally run it and expect a 98% passing rate. If it's below that, I might have an issue. One or two failing tests will never get noticed.

ars 15y ago

It seems like they add tests for bugs before actually fixing the bugs. I can see that being useful, but they need to do a better job of marking which bugs are ignorable and which are not. This is a an open source project - anyone want to volunteer?

arthurschreiber 15y ago
Just because it's an open source project does not mean that your contributions will be happily accepted.
- elliottcarlson 15y ago
  Nor does that mean you can't try.