6 ms·

The Ruby Stdlib is a Ghetto

71 points by chanks 16y ago

AndrewO 16y ago
I got the updated version of the Pickaxe recently and skimmed through all of the 1.9 updates. I was appalled to see REXML is still the blessed XML parser. I liked it when I started using Ruby and I still see it in the wild, but to me that's usually the sign of old code or a developer not tuned in to the latest developments. On the other hand, there has been some progress. It's good to see MiniTest replace Test::Unit (while also remaining mostly backwards compatible).
- Argorak 16y ago
  REXML is kept because many projects that need a pure ruby XML parser rely on it and it works for them. Usually, if you use XML beyond something small, you should use Nokogiri. Actually, 1.9.2 ships with replacements where possible: for example, CSV is now FasterCSV with a compatibility interface, Psych can be used instead of Syck for YAML parsing.
  - AndrewO 16y ago
    Good points (I'd forgotten about FasterCSV and Psych). I guess it's a mixed blessing if your library becomes part of stlib. On the one hand, it's gotta feel great to have so many people using your code. But on the other, your release cycle is pegged to Ruby's.
  - jseifer 16y ago
    Likewise for high performance http libraries.
    - Argorak 16y ago
      Yes, but most high-performance libraries are MRI specific. I'm still searching for a good Net::HTTP replacement that runs on both MRI and JRuby.
bobbywilson0 16y ago
I am not sure the author's solution, removing the said libs, is the right direction. I agree that the stdlib is crufty and slow, but they do 'work', and people certainly use them. I think the better option might be to deprecate and replace.
- bryanlarsen 16y ago
  Unfortunate choice of wording on the authors part, because it makes it seem like your solution is less drastic than his, whereas your solution is actually more drastic. To quote the author: "removal means move to a rubygem".
- rmoriz 16y ago
  To me the Stdlib API looks unfixable. So imho there's only one way: * put the existing stdlib into a legacy gem * create a new stdlib with a new, good, (incompatible) API from scratch * deal with the incompatibilities that will happen
  - bobbywilson0 16y ago
    I am not opposed to your solution, but sadly, I think it would be an extremely rare case. The ruby core team is very conservative with change, and I don't see some sweeping stdlib switch happening. James Edward Gray II was successful with ousting CSV for his improved implementation, and I think that's probably the way it will happen, if anything does happen. One library at a time.
  - vorador 16y ago
    This article should be of interest to you (it's about rewriting from scratch) : http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html
- tenderlove 16y ago
  Hi, I agree, deprecate and replace is the way to go. That's what I'm doing with Psych and with Fiddle. I really like this option because I don't want to break people's code. There is a downside though. I've tried shoehorning the REXML api on top of Nokogiri. The problem is that doing that work is thankless and not very fun. The API for REXML is just too large, and the deviations between the way it works and how libxml2 works are too many. I think if we add new libraries, and encourage people to move to the new ones, then start removing the old ones, that would be best.
rmoriz 16y ago
Database Error :(
- chanks 16y ago
  Copy: Much of Ruby’s standard library (the set of classes shipped with the Ruby VM itself) is old and crufty. For laughs, go look at the code for some of the classes that you’ve never used. Chances are it’s from 2000-2003 and doesn’t even look like idiomatic Ruby. I’m wondering what classes should be removed from the standard library or deprecated so that higher quality replacements can take their place. The canonical example is Ruby’s net/http library. Its performance and API are just terrible. (Side note: how do you know if an API is terrible? If you have to consult the docs even after having used the API for the past 5 years.) But because it’s in the standard library, most people use it as the base for higher-level API abstractions (e.g. httparty, rest-client). So looking at Ruby’s core RDoc, my suggested list for removal (where removal means move to a rubygem): Net::* DRb REXML RSS Rinda WEBrick XML Any others I missed? Will Ruby 1.9.3 or 2.0 get a good spring cleaning or will we have to live with these classes forever?
  - n8tron 16y ago
    Great comment - I completely agree about REXML. One question for you - what's an example of a http library in another language which you think provides a good API? Personally, I haven't found net/http to be any worse/better when compared to many other standard http libs.
    - catch23 16y ago
      Wasn't rexml's original implementation completely replaced in Ruby 1.9, Rubinius, & JRuby? It seems like the only people who might complain about it are those who are still stuck in 1.8 land.
  - jasonwatkinspdx 16y ago
    Drb is a bit unusual and rarely used, but I wonder what his objection is to the API?
lulin 16y ago
I don't think this is that much of a problem. If I want to use a modern library, I just type "gem install whatever" and I have the new library. Documentation for the stdlib is not really better or easier to find than that for third party libs, so I tend to use whatever seems nicer after a quick google search and I guess other people do it like that, too.
rmoriz 16y ago
It seems to me the massive "investments" made in mainly web-related frameworks (Rails, Merb, Sinatra + ORMs) didn't help the foundation of the Ruby ecosystem: Take rubygems, take RAA, take the MRI development process. If you got "brainwashed" in the last 5 years about code quality (DRY etc.) + test methods (TDD/BDD) and then look at today's Stdlib, you'll scream and want to put bleach into your eyes… To me it seems the "übercool" guys already moved away to their NoSQL/node.js/whatever next party, so it's not easy to motivate people to change such "messy" and large projects anymore. I'm also not sure about the number of Ruby interpreters around and their useful impact: If you e.g. change something for Ruby 1.9.x / 2.0.0, it will cost a lot of manpower to port those changes to JRuby, Rubinius, maglev, IronRuby and BlueRuby (if SAP's Ruby is still alive?) and most of them are just used by very few people. Maybe resources are better spent in creating a new Stdlib, improve the rubygems infrastructure (already started with gemcutter) and do something with RAA?
- mileszs 16y ago
  The Ruby language specification (http://www.rubyspec.org/ http://www.rubyspec.org/), to which all Ruby VMs are written, helps as far as the "number of Ruby interpreters around and their impact" goes. (Assuming I understood your concern correctly.) A new Ruby VM is considered legit if it passes Ruby Spec. That should help in both transparency and compatibility across implementations. Would you agree?
  - rmoriz 16y ago
    but you still have to implement a "language change" into all different VMs. This does cost time and money but delivers redundancy that has to be maintained. Which is great, if you don't have more important problems. A horrible Stdlib is imho a more important problem today.
  - joe_the_user 16y ago
    Following your link: "This website is presently being redesigned. We expect to be back soon." But I knew already that "rubyspec" wasn't a spec in the ordinary sense but a test-suit built with MSpec. The thing is that real spec actually fully specifies a language - at least up to syntax. There are real specs for Python and Java for instance. The only real spec for Ruby is still crufty Yacc code for MRI. A test suit doesn't do that and the whole Rubyspec thing is a testimony to people drinking too much of the Test Driven Development Cool Aid. TDD may be fine for some things but some formal design is needed for things like the creation of interpreters. Part of the challenge is that MRI has such a fluid syntax that specifying it conventionally would be quite hard. But I suspect that's what will have to happen if Ruby's going to go forward.
deleted 16y ago
[deleted]
brendano 16y ago
Hate to sound preachy, but compare to Python developers, who very carefully maintain and update their stdlib. I feel like it is risky to commit to using Ruby for the long-term.
- tptacek 16y ago
  Is that a joke? How many different interfaces are there to popen(3)? How many different ways to get the current time? If Ruby's standard library is a ghetto, Python's is a jungle. If you like Python, I don't know why you'd even think about "committing to Ruby for the long term"; you're a Python dev. They're practically the same language. Stop trying to pick fights.
  - bdr 16y ago
    Yes, there are many interfaces to popen, but that doesn't refute the grandparent, who claims that the Python stdlib is maintained and updated. See the top of http://docs.python.org/library/subprocess.html http://docs.python.org/library/subprocess.html -- the bad solution was deprecated, and a good solution was created. This is movement towards goodness, which the OP laments the lack of in Ruby.
    - kingkilr 16y ago
      It's the ciiircle of liiffeee...
  - viraptor 16y ago
    > How many different interfaces are there to popen(3)? popen, popen2 - both deprecated. The new "proper" one is available since 2.4 (quite a long time now) - subprocess. > How many different ways to get the current time? time.time() <- timestamp, datetime.now() <- date Is this that bad? Sure, popen and popen2 are silly, but they're left for compatibility, while subprocess was introduced to update stdlib (as OP claimed). It's hardly a jungle...
  - keyist 16y ago
    GP is trolling. No need to counter-troll while feeding. The flamebait on committing to Ruby does not invalidate his point on stdlib maintenance though. We can sit here and ping each other with examples of cruft all day. I think it can be shown objectively that Python's standard library is better maintained. I'll attempt to below. I'm not saying this has a great significance in picking one language over the other in and of itself. Just that if you want to talk specifically about stdlibs, the differences are pretty clear-cut imo. == In terms of quantity of updates == - Mar 13 2007: Ruby 1.8.6 launched - May 18 2008: Ruby 1.8.7 launched - Distance: ~14 months - Sep 19 2006 Python 2.5 launched - Oct 18 2008 Python 2.6 launched - Distance: ~23 months So discount the amount of 2.5->2.6 changes accordingly to account for the extra time. Then compare stdlib changes: - Ruby: http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS - Python: http://docs.python.org/whatsnew/2.6.html#new-and-improved-modules http://docs.python.org/whatsnew/2.6.html#new-and-improved-mo... or http://www.python.org/download/releases/2.6.1/NEWS.txt http://www.python.org/download/releases/2.6.1/NEWS.txt == In terms of quality of documentation == Compare http://www.rubydoc.info/docs/ruby-stdlib http://www.rubydoc.info/docs/ruby-stdlib to http://docs.python.org/library/ http://docs.python.org/library/ You'd need to compare across actual libraries (unittest vs Test/Unit etc). I didn't provide specific examples in links to avoid inevitable accusations of cherry-picking. == In terms of process == Compare http://www.python.org/dev/peps/pep-3108/ http://www.python.org/dev/peps/pep-3108/ to... closest equivalent I could find was http://redmine.ruby-lang.org/projects/roadmap/ruby-19 http://redmine.ruby-lang.org/projects/roadmap/ruby-19 . Anyone with a better link?
chrislloyd 16y ago
I wrote something similar a while ago[0]. Ruby should be the language and everything you "require" should be a Gem. It won't happen with Ruby (and isn't worth the effort). New languages, however, take heed. [0] http://thelincolnshirepoacher.com/pages/standard-libraries
- mcantor 16y ago
  Instead of forcing users to "install file" and "install path," which is bound to cause frustration, you could probably just have an "install stdlib" meta-package. Or at least an "install list stdlib" command that showed people all of the packages to install if they wanted stdlib functionality, which would be completely clear and allow for cherry-picking packages. Or install all packages in stdlib by default and let people remove them if they want.
  - chrislloyd 16y ago
    Yeah absolutely. I think what is important is that people can fork and change the stdlib. I also like the idea of platform independence coming from installing different Gems, i.e. "file-java" and "file-win32". However, that could work out to be a cluster-cuss in practice.
thomaz 16y ago
Rails went (and is still going) through the same process. They realized they had a lot of code not being used (or being replaced by other gems) and just moved everything away from the core. IMHO, with a lib sharing system such as rubygems, having a standard set of "external" libs in kind of unnecessary.
krainboltgreene 16y ago
One of the Ruby implementations is going to have to change the game, because the current standard is hacked around rather than used naturally. That isn't productive, both for hacking and professional work.
bokchoi 16y ago
Ghetto? How about just "old and crufty"?
- telemachos 16y ago
  It's not 'ghetto' in the slangy adjectival use (e.g., "That coat is so ghetto."), but 'ghetto' in the normal noun use: "The Ruby Stdlib is a Ghetto" (my emphasis). And if you still think that's hyperbole, it's an allusion to Zed Shaw's infamous "Rails Is A Ghetto" post.
  - mperham 16y ago
    Good catch. It was indeed a nod to Zed's post.
  - bhickey 16y ago
    I thought it was a reference to the tact and grace embodied by titles like "CouchDB: Perform like a pornstar".
Spakman 16y ago
I heard this about 3rd hand, so consume with a large helping of salt: the Ruby core developers are considering using the Rubinius stdlib for Ruby 2.0.

71 points by chanks 16y ago

I got the updated version of the Pickaxe recently and skimmed through all of the 1.9 updates. I was appalled to see REXML is still the blessed XML parser. I liked it when I started using Ruby and I still see it in the wild, but to me that's usually the sign of old code or a developer not tuned in to the latest developments. On the other hand, there has been some progress. It's good to see MiniTest replace Test::Unit (while also remaining mostly backwards compatible).

Argorak 16y ago
REXML is kept because many projects that need a pure ruby XML parser rely on it and it works for them. Usually, if you use XML beyond something small, you should use Nokogiri. Actually, 1.9.2 ships with replacements where possible: for example, CSV is now FasterCSV with a compatibility interface, Psych can be used instead of Syck for YAML parsing.
- AndrewO 16y ago
  Good points (I'd forgotten about FasterCSV and Psych). I guess it's a mixed blessing if your library becomes part of stlib. On the one hand, it's gotta feel great to have so many people using your code. But on the other, your release cycle is pegged to Ruby's.
- jseifer 16y ago
  Likewise for high performance http libraries.
  - Argorak 16y ago
    Yes, but most high-performance libraries are MRI specific. I'm still searching for a good Net::HTTP replacement that runs on both MRI and JRuby.

bobbywilson0 16y ago

I am not sure the author's solution, removing the said libs, is the right direction. I agree that the stdlib is crufty and slow, but they do 'work', and people certainly use them. I think the better option might be to deprecate and replace.

bryanlarsen 16y ago
Unfortunate choice of wording on the authors part, because it makes it seem like your solution is less drastic than his, whereas your solution is actually more drastic. To quote the author: "removal means move to a rubygem".
rmoriz 16y ago
To me the Stdlib API looks unfixable. So imho there's only one way: * put the existing stdlib into a legacy gem * create a new stdlib with a new, good, (incompatible) API from scratch * deal with the incompatibilities that will happen
- bobbywilson0 16y ago
  I am not opposed to your solution, but sadly, I think it would be an extremely rare case. The ruby core team is very conservative with change, and I don't see some sweeping stdlib switch happening. James Edward Gray II was successful with ousting CSV for his improved implementation, and I think that's probably the way it will happen, if anything does happen. One library at a time.
- vorador 16y ago
  This article should be of interest to you (it's about rewriting from scratch) : http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html
tenderlove 16y ago
Hi, I agree, deprecate and replace is the way to go. That's what I'm doing with Psych and with Fiddle. I really like this option because I don't want to break people's code. There is a downside though. I've tried shoehorning the REXML api on top of Nokogiri. The problem is that doing that work is thankless and not very fun. The API for REXML is just too large, and the deviations between the way it works and how libxml2 works are too many. I think if we add new libraries, and encourage people to move to the new ones, then start removing the old ones, that would be best.

rmoriz 16y ago

Database Error :(

chanks 16y ago
Copy: Much of Ruby’s standard library (the set of classes shipped with the Ruby VM itself) is old and crufty. For laughs, go look at the code for some of the classes that you’ve never used. Chances are it’s from 2000-2003 and doesn’t even look like idiomatic Ruby. I’m wondering what classes should be removed from the standard library or deprecated so that higher quality replacements can take their place. The canonical example is Ruby’s net/http library. Its performance and API are just terrible. (Side note: how do you know if an API is terrible? If you have to consult the docs even after having used the API for the past 5 years.) But because it’s in the standard library, most people use it as the base for higher-level API abstractions (e.g. httparty, rest-client). So looking at Ruby’s core RDoc, my suggested list for removal (where removal means move to a rubygem): Net::* DRb REXML RSS Rinda WEBrick XML Any others I missed? Will Ruby 1.9.3 or 2.0 get a good spring cleaning or will we have to live with these classes forever?
- n8tron 16y ago
  Great comment - I completely agree about REXML. One question for you - what's an example of a http library in another language which you think provides a good API? Personally, I haven't found net/http to be any worse/better when compared to many other standard http libs.
  - catch23 16y ago
    Wasn't rexml's original implementation completely replaced in Ruby 1.9, Rubinius, & JRuby? It seems like the only people who might complain about it are those who are still stuck in 1.8 land.
- jasonwatkinspdx 16y ago
  Drb is a bit unusual and rarely used, but I wonder what his objection is to the API?

lulin 16y ago

I don't think this is that much of a problem. If I want to use a modern library, I just type "gem install whatever" and I have the new library. Documentation for the stdlib is not really better or easier to find than that for third party libs, so I tend to use whatever seems nicer after a quick google search and I guess other people do it like that, too.

rmoriz 16y ago

It seems to me the massive "investments" made in mainly web-related frameworks (Rails, Merb, Sinatra + ORMs) didn't help the foundation of the Ruby ecosystem: Take rubygems, take RAA, take the MRI development process. If you got "brainwashed" in the last 5 years about code quality (DRY etc.) + test methods (TDD/BDD) and then look at today's Stdlib, you'll scream and want to put bleach into your eyes… To me it seems the "übercool" guys already moved away to their NoSQL/node.js/whatever next party, so it's not easy to motivate people to change such "messy" and large projects anymore. I'm also not sure about the number of Ruby interpreters around and their useful impact: If you e.g. change something for Ruby 1.9.x / 2.0.0, it will cost a lot of manpower to port those changes to JRuby, Rubinius, maglev, IronRuby and BlueRuby (if SAP's Ruby is still alive?) and most of them are just used by very few people. Maybe resources are better spent in creating a new Stdlib, improve the rubygems infrastructure (already started with gemcutter) and do something with RAA?

mileszs 16y ago
The Ruby language specification (http://www.rubyspec.org/ http://www.rubyspec.org/), to which all Ruby VMs are written, helps as far as the "number of Ruby interpreters around and their impact" goes. (Assuming I understood your concern correctly.) A new Ruby VM is considered legit if it passes Ruby Spec. That should help in both transparency and compatibility across implementations. Would you agree?
- rmoriz 16y ago
  but you still have to implement a "language change" into all different VMs. This does cost time and money but delivers redundancy that has to be maintained. Which is great, if you don't have more important problems. A horrible Stdlib is imho a more important problem today.
- joe_the_user 16y ago
  Following your link: "This website is presently being redesigned. We expect to be back soon." But I knew already that "rubyspec" wasn't a spec in the ordinary sense but a test-suit built with MSpec. The thing is that real spec actually fully specifies a language - at least up to syntax. There are real specs for Python and Java for instance. The only real spec for Ruby is still crufty Yacc code for MRI. A test suit doesn't do that and the whole Rubyspec thing is a testimony to people drinking too much of the Test Driven Development Cool Aid. TDD may be fine for some things but some formal design is needed for things like the creation of interpreters. Part of the challenge is that MRI has such a fluid syntax that specifying it conventionally would be quite hard. But I suspect that's what will have to happen if Ruby's going to go forward.

deleted 16y ago

[deleted]

brendano 16y ago

Hate to sound preachy, but compare to Python developers, who very carefully maintain and update their stdlib. I feel like it is risky to commit to using Ruby for the long-term.

tptacek 16y ago
Is that a joke? How many different interfaces are there to popen(3)? How many different ways to get the current time? If Ruby's standard library is a ghetto, Python's is a jungle. If you like Python, I don't know why you'd even think about "committing to Ruby for the long term"; you're a Python dev. They're practically the same language. Stop trying to pick fights.
- bdr 16y ago
  Yes, there are many interfaces to popen, but that doesn't refute the grandparent, who claims that the Python stdlib is maintained and updated. See the top of http://docs.python.org/library/subprocess.html http://docs.python.org/library/subprocess.html -- the bad solution was deprecated, and a good solution was created. This is movement towards goodness, which the OP laments the lack of in Ruby.
  - kingkilr 16y ago
    It's the ciiircle of liiffeee...
- viraptor 16y ago
  > How many different interfaces are there to popen(3)? popen, popen2 - both deprecated. The new "proper" one is available since 2.4 (quite a long time now) - subprocess. > How many different ways to get the current time? time.time() <- timestamp, datetime.now() <- date Is this that bad? Sure, popen and popen2 are silly, but they're left for compatibility, while subprocess was introduced to update stdlib (as OP claimed). It's hardly a jungle...
- keyist 16y ago
  GP is trolling. No need to counter-troll while feeding. The flamebait on committing to Ruby does not invalidate his point on stdlib maintenance though. We can sit here and ping each other with examples of cruft all day. I think it can be shown objectively that Python's standard library is better maintained. I'll attempt to below. I'm not saying this has a great significance in picking one language over the other in and of itself. Just that if you want to talk specifically about stdlibs, the differences are pretty clear-cut imo. == In terms of quantity of updates == - Mar 13 2007: Ruby 1.8.6 launched - May 18 2008: Ruby 1.8.7 launched - Distance: ~14 months - Sep 19 2006 Python 2.5 launched - Oct 18 2008 Python 2.6 launched - Distance: ~23 months So discount the amount of 2.5->2.6 changes accordingly to account for the extra time. Then compare stdlib changes: - Ruby: http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS - Python: http://docs.python.org/whatsnew/2.6.html#new-and-improved-modules http://docs.python.org/whatsnew/2.6.html#new-and-improved-mo... or http://www.python.org/download/releases/2.6.1/NEWS.txt http://www.python.org/download/releases/2.6.1/NEWS.txt == In terms of quality of documentation == Compare http://www.rubydoc.info/docs/ruby-stdlib http://www.rubydoc.info/docs/ruby-stdlib to http://docs.python.org/library/ http://docs.python.org/library/ You'd need to compare across actual libraries (unittest vs Test/Unit etc). I didn't provide specific examples in links to avoid inevitable accusations of cherry-picking. == In terms of process == Compare http://www.python.org/dev/peps/pep-3108/ http://www.python.org/dev/peps/pep-3108/ to... closest equivalent I could find was http://redmine.ruby-lang.org/projects/roadmap/ruby-19 http://redmine.ruby-lang.org/projects/roadmap/ruby-19 . Anyone with a better link?

chrislloyd 16y ago

I wrote something similar a while ago[0]. Ruby should be the language and everything you "require" should be a Gem. It won't happen with Ruby (and isn't worth the effort). New languages, however, take heed. [0] http://thelincolnshirepoacher.com/pages/standard-libraries

mcantor 16y ago
Instead of forcing users to "install file" and "install path," which is bound to cause frustration, you could probably just have an "install stdlib" meta-package. Or at least an "install list stdlib" command that showed people all of the packages to install if they wanted stdlib functionality, which would be completely clear and allow for cherry-picking packages. Or install all packages in stdlib by default and let people remove them if they want.
- chrislloyd 16y ago
  Yeah absolutely. I think what is important is that people can fork and change the stdlib. I also like the idea of platform independence coming from installing different Gems, i.e. "file-java" and "file-win32". However, that could work out to be a cluster-cuss in practice.

thomaz 16y ago

Rails went (and is still going) through the same process. They realized they had a lot of code not being used (or being replaced by other gems) and just moved everything away from the core. IMHO, with a lib sharing system such as rubygems, having a standard set of "external" libs in kind of unnecessary.

krainboltgreene 16y ago

One of the Ruby implementations is going to have to change the game, because the current standard is hacked around rather than used naturally. That isn't productive, both for hacking and professional work.

bokchoi 16y ago

Ghetto? How about just "old and crufty"?

telemachos 16y ago
It's not 'ghetto' in the slangy adjectival use (e.g., "That coat is so ghetto."), but 'ghetto' in the normal noun use: "The Ruby Stdlib is a Ghetto" (my emphasis). And if you still think that's hyperbole, it's an allusion to Zed Shaw's infamous "Rails Is A Ghetto" post.
- mperham 16y ago
  Good catch. It was indeed a nod to Zed's post.
- bhickey 16y ago
  I thought it was a reference to the tact and grace embodied by titles like "CouchDB: Perform like a pornstar".

Spakman 16y ago

I heard this about 3rd hand, so consume with a large helping of salt: the Ruby core developers are considering using the Rubinius stdlib for Ruby 2.0.