5 ms·

Semi-related: does anyone know why installing gems is so ridiculously slow? What is the thing doing? Downloading tarballs, yes, but then? It's a dynamic languag

by mbrock 12y ago

Semi-related: does anyone know why installing gems is so ridiculously slow? What is the thing doing? Downloading tarballs, yes, but then? It's a dynamic language; there is no compilation or verification! Why can I install Ruby packages using apt almost immediately, when gem/bundle install takes half a coffee break? I'm growing more impatient with the years. I have measured out my life with slow software. We talk about saving developer time with dynamic languages, but, as Flight of the Conchords sang, the sneakers don't seem to get much cheaper; what are your overheads?

ishtu 12y ago
gem: --no-document --verbose in your ~/.gemrc may speed up some things and show slowest steps (where '--no-document' means the same as deprecated '--no-rdoc --no-ri').
richthegeek 12y ago
Not that I install gems very often (Node.JS is my primary platform) but I find that the majority of the time is installing the different docs. Check the difference between "gem install sass" and "gem install sass --no-rdoc --no-ri" and be amazed.
- shawabawa3 12y ago
  Which as far as I'm concerned is a bug. I don't even know how to view gem documentation, and I've never wanted or needed to. --no-document should be the default They could make it download the docs on first view
ing33k 12y ago
imo resolving dependencies is one of the factor
tobeportable 12y ago
U can also run with option -j4 to get 4 workers doing the task; u can also add it to your global config : bundle config --global jobs 4
steveklabnik 12y ago
This may sound a bit glib, but the reason it's so slow is because basically every Rubyist goes "Does anyone know why installing gems is so slow? What is this thing doing?" And then _takes a coffee break instead of figuring it out_. There are very, very, very few people who actually do any work on core infrastructure projects. I don't blame them. The codebase of rubygems is... not exactly welcoming. I myself did some work a while back, got frustrated, and quit. But if you want the real reason, there it is. Oh, and when you _do_ overcome all these barriers and then actually make an improvement, people will say "lol rubyists learning data structures" instead of saying "thank you for saving a bit of the most precious resource I have every single day, time."
- mbrock 12y ago
  Yeah, good point. Still, it seems likely that a number of people have done at least some work on profiling and optimizing without success. Hasn't Gem slowness been an issue since, like, 2006? The reason I haven't even tried is mostly that it seems likely to be difficult and treacherous. But I might be wrong. Note that I wasn't only complaining. I just figured that there is some real reason for the slowness and that this reason might be known by the community (and I didn't find much googling for "why is ruby gems slow"). Call it preliminary research...
  - steveklabnik 12y ago
    > Still, it seems likely that a number of people have done at least some work on profiling and optimizing without success. You'd think that, wouldn't you? The reality is... not that. :/ And yeah, this isn't really about you specifically, sorry if it came off that way. This is just a general problem in the Ruby OSS world. I can't speak to too many other OSS worlds, as the Ruby one is the one I've been most involved in. If you're looking for more research, Andre Arko is one of the other people who's actually doing work in this area, and he's given a number of talks about why Bundler is slow: https://vimeo.com/67807956 https://vimeo.com/67807956 and related. He's been making a lot of strides, but it's not easy.
    - mbrock 12y ago
shime 12y ago
also, your ISP provider often has some shitty DNS so switching to Google's might help. give the big brother all your datas!
dragonwriter 12y ago
> Semi-related: does anyone know why installing gems is so ridiculously slow? What is the thing doing? Downloading tarballs, yes, but then? It's a dynamic language; there is no compilation or verification! Source gems that include C extensions may well have compilation; there is also, IIRC, automatic doc generation in the usual, default gem installation. > Why can I install Ruby packages using apt almost immediately, when gem/bundle install takes half a coffee break? Because the apt packages are prebuilt for your OS and architecture, they don't need to resolve platform constraints, possibly build after download, and do doc generation after download.

by mbrock 12y ago

ishtu 12y ago

gem: --no-document --verbose in your ~/.gemrc may speed up some things and show slowest steps (where '--no-document' means the same as deprecated '--no-rdoc --no-ri').

richthegeek 12y ago

Not that I install gems very often (Node.JS is my primary platform) but I find that the majority of the time is installing the different docs. Check the difference between "gem install sass" and "gem install sass --no-rdoc --no-ri" and be amazed.

shawabawa3 12y ago
Which as far as I'm concerned is a bug. I don't even know how to view gem documentation, and I've never wanted or needed to. --no-document should be the default They could make it download the docs on first view

ing33k 12y ago

imo resolving dependencies is one of the factor

tobeportable 12y ago

U can also run with option -j4 to get 4 workers doing the task; u can also add it to your global config : bundle config --global jobs 4

steveklabnik 12y ago

This may sound a bit glib, but the reason it's so slow is because basically every Rubyist goes "Does anyone know why installing gems is so slow? What is this thing doing?" And then _takes a coffee break instead of figuring it out_. There are very, very, very few people who actually do any work on core infrastructure projects. I don't blame them. The codebase of rubygems is... not exactly welcoming. I myself did some work a while back, got frustrated, and quit. But if you want the real reason, there it is. Oh, and when you _do_ overcome all these barriers and then actually make an improvement, people will say "lol rubyists learning data structures" instead of saying "thank you for saving a bit of the most precious resource I have every single day, time."

mbrock 12y ago
Yeah, good point. Still, it seems likely that a number of people have done at least some work on profiling and optimizing without success. Hasn't Gem slowness been an issue since, like, 2006? The reason I haven't even tried is mostly that it seems likely to be difficult and treacherous. But I might be wrong. Note that I wasn't only complaining. I just figured that there is some real reason for the slowness and that this reason might be known by the community (and I didn't find much googling for "why is ruby gems slow"). Call it preliminary research...
- steveklabnik 12y ago
  > Still, it seems likely that a number of people have done at least some work on profiling and optimizing without success. You'd think that, wouldn't you? The reality is... not that. :/ And yeah, this isn't really about you specifically, sorry if it came off that way. This is just a general problem in the Ruby OSS world. I can't speak to too many other OSS worlds, as the Ruby one is the one I've been most involved in. If you're looking for more research, Andre Arko is one of the other people who's actually doing work in this area, and he's given a number of talks about why Bundler is slow: https://vimeo.com/67807956 https://vimeo.com/67807956 and related. He's been making a lot of strides, but it's not easy.
  - mbrock 12y ago

shime 12y ago

also, your ISP provider often has some shitty DNS so switching to Google's might help. give the big brother all your datas!

dragonwriter 12y ago

> Semi-related: does anyone know why installing gems is so ridiculously slow? What is the thing doing? Downloading tarballs, yes, but then? It's a dynamic language; there is no compilation or verification! Source gems that include C extensions may well have compilation; there is also, IIRC, automatic doc generation in the usual, default gem installation. > Why can I install Ruby packages using apt almost immediately, when gem/bundle install takes half a coffee break? Because the apt packages are prebuilt for your OS and architecture, they don't need to resolve platform constraints, possibly build after download, and do doc generation after download.