4 ms·

I've been reading in the comments regarding 1) the practical effect of breaking builds and 2) the security issues of how package names can be reused on npm once

I've been reading in the comments regarding 1) the practical effect of breaking builds and 2) the security issues of how package names can be reused on npm once they are unpublished (versioning aside for a moment). I wonder what other, similar, packaging distribution platforms are vulnerable to this sort of thing? I am not speaking from knowledge of any of the procedures of any of those I'm about to mention, but I have and do depend on some them. Thinking about this issue and some of those other tools that pull long strings of dependent packages does give me pause. Especially the replacement of some dependency with less than friendly code... breakage can be managed, but silent invaders... Does Perl & CPAN, Rust & crates.io, or Ruby & RubyGems.org suffer these same issues and it just just hasn't been a problem yet? Do they have means of avoiding this? Again, I haven't studied the question... but I think I may :-)

steveklabnik 11y ago
With crates.io, users cannot delete their packages. They can 'yank' a version, which means that no new packages can depend on it, but existing packages can. So in this specific case, there are two packages: kik, the subject of the legal threat, and left-pad, another module by the same author. If we were compelled through legal means to remove a package, as administorators, we could, which is what happened with kik. However, if said author got mad at us, they couldn't _delete_ their other packages, only yank them, and so people's builds would not break, like they did here. Does that make sense? (IIRC Rubygems acts in the same way)
- sbuttgereit 11y ago
  It does. Given that I've started a project in Rust which will have such dependencies, I feel even better that clearly there has been some good thinking put into this.
nirvdrum 11y ago
I've had deploys break because gems were removed from RubyGems.org. Using CI as a gate isn't sufficient because a gem can be removed in the window between a completed CI run and the deploy. Making it harder, generally if you have a gem installed locally you won't notice the problem until you need to install it on a fresh system. It used to be the case that a yanked gem was simply removed from the gem index, but the file was still published, so you'd still be able to install it. Now the gems are deleted, eliminating that option. So, other systems are definitely susceptible and it does burn people. I just don't think we've seen an issue on this scale yet. The largest I've hit have been when the twitter gem was yanked in advance of Twitter's shutdown, a fog-XYZ gem being yanked as new versions were released, and at least one yanked version of net-ssh. The only really effective way to avoid this is to mirror all your dependencies yourself (and not honor upstream yanks) or vendor all your gems. I know of exceedingly few people doing this. I've yet to encounter a legitimate cause for a yanked gem. But I worry more that a bad actor gains credentials to a high profile account and just wreaks havoc on the ecosystem. On that note, I have found and reported users that've committed their RubyGems.org credentials to GitHub in a "dotfiles" repository.
- krakensden 11y ago
  Dear Everyone Doing Devops: do not build gems on your production servers. Build a tarball, test the tarball through dev/staging, deploy the tarball to production. For any value of tarball- literal tarball, deb/rpm (actually CPIO under the hood, but who cares), prebaked AMI (not tar at all), docker image (tarball of JSON + nested tarballs. Seriously, it's tar all the way down). Everything will be better. Roll forwards. Roll backs. Speed. Attack surface. Reliance on 3rd party services that will rate limit you hard.
  - softawre 11y ago
    http://martinfowler.com/bliki/ImmutableServer.html http://martinfowler.com/bliki/ImmutableServer.html
  - nirvdrum 11y ago
    Your point is well-taken. Broken deploys are just the extreme case. You'd run into the same problem if two devs on the team just happened to install libraries at different times. One would get it, one would not. And you don't get a remotely helpful error message in that case. So, you're back to mirroring or vendoring everything you need anyway. Arguably that's better, but why even bother with a centralized package management system at all then?

steveklabnik 11y ago

With crates.io, users cannot delete their packages. They can 'yank' a version, which means that no new packages can depend on it, but existing packages can. So in this specific case, there are two packages: kik, the subject of the legal threat, and left-pad, another module by the same author. If we were compelled through legal means to remove a package, as administorators, we could, which is what happened with kik. However, if said author got mad at us, they couldn't _delete_ their other packages, only yank them, and so people's builds would not break, like they did here. Does that make sense? (IIRC Rubygems acts in the same way)

sbuttgereit 11y ago
It does. Given that I've started a project in Rust which will have such dependencies, I feel even better that clearly there has been some good thinking put into this.

nirvdrum 11y ago

I've had deploys break because gems were removed from RubyGems.org. Using CI as a gate isn't sufficient because a gem can be removed in the window between a completed CI run and the deploy. Making it harder, generally if you have a gem installed locally you won't notice the problem until you need to install it on a fresh system. It used to be the case that a yanked gem was simply removed from the gem index, but the file was still published, so you'd still be able to install it. Now the gems are deleted, eliminating that option. So, other systems are definitely susceptible and it does burn people. I just don't think we've seen an issue on this scale yet. The largest I've hit have been when the twitter gem was yanked in advance of Twitter's shutdown, a fog-XYZ gem being yanked as new versions were released, and at least one yanked version of net-ssh. The only really effective way to avoid this is to mirror all your dependencies yourself (and not honor upstream yanks) or vendor all your gems. I know of exceedingly few people doing this. I've yet to encounter a legitimate cause for a yanked gem. But I worry more that a bad actor gains credentials to a high profile account and just wreaks havoc on the ecosystem. On that note, I have found and reported users that've committed their RubyGems.org credentials to GitHub in a "dotfiles" repository.

krakensden 11y ago
Dear Everyone Doing Devops: do not build gems on your production servers. Build a tarball, test the tarball through dev/staging, deploy the tarball to production. For any value of tarball- literal tarball, deb/rpm (actually CPIO under the hood, but who cares), prebaked AMI (not tar at all), docker image (tarball of JSON + nested tarballs. Seriously, it's tar all the way down). Everything will be better. Roll forwards. Roll backs. Speed. Attack surface. Reliance on 3rd party services that will rate limit you hard.
- softawre 11y ago
  http://martinfowler.com/bliki/ImmutableServer.html http://martinfowler.com/bliki/ImmutableServer.html
- nirvdrum 11y ago
  Your point is well-taken. Broken deploys are just the extreme case. You'd run into the same problem if two devs on the team just happened to install libraries at different times. One would get it, one would not. And you don't get a remotely helpful error message in that case. So, you're back to mirroring or vendoring everything you need anyway. Arguably that's better, but why even bother with a centralized package management system at all then?