3 ms·

I think the big thing here that threw me for a loop is just how many gems are pulled in by projects. 133 seems like a ridiculous amount of dependencies for a si

by danudey 13y ago

I think the big thing here that threw me for a loop is just how many gems are pulled in by projects. 133 seems like a ridiculous amount of dependencies for a single project. Our entire company runs on Python in the backend and we don't even have 50 (and that's including ones I use for command-line utilities or corncobs, for example, which aren't in our public-facing software). Even if the Python modules we have had the same number of vulnerabilities as gems, our exposure for a given system would be 1/10th, just from not using so many external libraries. Couple that with the fact that many of those gems are outside of end-user control, of generally unknown quality, might only be pulled in for one small piece of functionality, and might themselves depend on other gems which you weren't necessarily aware of and haven't necessarily vetted.

steveklabnik 13y ago
A blank Rails app installs ~40 gems last I checked.
chc 13y ago
I think part of it is that people tend to separate things out into lots of gems. For example, if you're using Ember in your Rails app, that's two gems right there just for that library. If you want to allow delegated authentication with the common providers (Facebook, Twitter, OpenID), that's like 10 gems. Your unit testing and error reporting comes from gems, your app server is a gem, every database you want to support is at least one gem, etc. It gets explosive, but it's really just an organization thing. One thing to keep in mind, though, is that not all gems are actually used at a given time. Your gems for testing, development and production all go in the Gemfile, but you're probably not going to install all of them on one system.
bstar77 13y ago
Not all gems are created equal. Large amount of gems don't necessarily mean that they are arbitrarily included. In fact, it's often better to pull in a dependency for things that specialize in a particular task because they are more likely to have hardened security and better performance. The flip side is actually knowing that this "meme generator" gem you are using actually pulls in 50 dependencies, many of which are terribly implemented. I've seen this problem just as rampant in the Node.js and jQuery worlds. As developers, we should be as aware as possible of the state of our critical apps dependencies. I've recently gone through an app that had over 50 gem dependencies and reduced it to under 20. One thing that's probably not considered is that testing frameworks (rspec) have tons of dependencies that are not scoped to production. I'm not sure if these reports considered that.
GlennS 13y ago
It would be interesting to know why this is. Are each of these dependencies really small on average? Or is there some big cascade of dependencies? Or is there a particular framework that's to blame? I've had a similar issue when using the Spring Framework in Java, where including it in your project at all brings a whole host of other (often not very good) libraries in. This is a problem in Java, which doesn't have any built in capacity for loading multiple versions of the same library. In Python, I suspect that I've been in a similar situation when using things like Pyproj, Numpy or Scipy, because those all depend on components written in C or Fortran. Perhaps next time I use any of those libraries I shall take some time to read more of the code and try to trace out what things they may be brining in.
- ddebernardy 13y ago
  > Are each of these dependencies really small on average? Or is there some big cascade of dependencies? Or is there a particular framework that's to blame? All of that, really. For better or worse, scores of gems have dependencies. Rails and a few popular others add large dependencies that cascade to other dependencies at that — some large, some not. It's extremely rare to run into quality gems with very few if any dependencies. (Sequel comes to mind, and that's just about the only 1st-class gem I can think of in pure Ruby.)
- chc 13y ago
  A lot of them really are small. There is a tendency for Ruby libraries to be separated into "core" and "feature" gems that you can pull in separately, For example, if you want to use DataMapper, you might be pulling in dm-core, dm-rails, dm-validations, dm-migrations, dm-timestamps, dm-postgres-adapter, dm-mysql-adapter, dm-is-searchable…

by danudey 13y ago

steveklabnik 13y ago

A blank Rails app installs ~40 gems last I checked.

chc 13y ago

I think part of it is that people tend to separate things out into lots of gems. For example, if you're using Ember in your Rails app, that's two gems right there just for that library. If you want to allow delegated authentication with the common providers (Facebook, Twitter, OpenID), that's like 10 gems. Your unit testing and error reporting comes from gems, your app server is a gem, every database you want to support is at least one gem, etc. It gets explosive, but it's really just an organization thing. One thing to keep in mind, though, is that not all gems are actually used at a given time. Your gems for testing, development and production all go in the Gemfile, but you're probably not going to install all of them on one system.

bstar77 13y ago

Not all gems are created equal. Large amount of gems don't necessarily mean that they are arbitrarily included. In fact, it's often better to pull in a dependency for things that specialize in a particular task because they are more likely to have hardened security and better performance. The flip side is actually knowing that this "meme generator" gem you are using actually pulls in 50 dependencies, many of which are terribly implemented. I've seen this problem just as rampant in the Node.js and jQuery worlds. As developers, we should be as aware as possible of the state of our critical apps dependencies. I've recently gone through an app that had over 50 gem dependencies and reduced it to under 20. One thing that's probably not considered is that testing frameworks (rspec) have tons of dependencies that are not scoped to production. I'm not sure if these reports considered that.

GlennS 13y ago

It would be interesting to know why this is. Are each of these dependencies really small on average? Or is there some big cascade of dependencies? Or is there a particular framework that's to blame? I've had a similar issue when using the Spring Framework in Java, where including it in your project at all brings a whole host of other (often not very good) libraries in. This is a problem in Java, which doesn't have any built in capacity for loading multiple versions of the same library. In Python, I suspect that I've been in a similar situation when using things like Pyproj, Numpy or Scipy, because those all depend on components written in C or Fortran. Perhaps next time I use any of those libraries I shall take some time to read more of the code and try to trace out what things they may be brining in.

ddebernardy 13y ago
> Are each of these dependencies really small on average? Or is there some big cascade of dependencies? Or is there a particular framework that's to blame? All of that, really. For better or worse, scores of gems have dependencies. Rails and a few popular others add large dependencies that cascade to other dependencies at that — some large, some not. It's extremely rare to run into quality gems with very few if any dependencies. (Sequel comes to mind, and that's just about the only 1st-class gem I can think of in pure Ruby.)
chc 13y ago
A lot of them really are small. There is a tendency for Ruby libraries to be separated into "core" and "feature" gems that you can pull in separately, For example, if you want to use DataMapper, you might be pulling in dm-core, dm-rails, dm-validations, dm-migrations, dm-timestamps, dm-postgres-adapter, dm-mysql-adapter, dm-is-searchable…