10 ms·

This doesn't surprise me. Horrify.. yes. I've noticed more dev teams succumbing to the temptation of easiness that many modern package managers provide (NPM, C

This doesn't surprise me. Horrify.. yes. I've noticed more dev teams succumbing to the temptation of easiness that many modern package managers provide (NPM, Cargo, Ivy, etc.) - especially as someone who has to work with offline systems on a regular basis. Because of that ease there are fewer tools and tutorials out there to support offline package management. There are more for using caches, though these are often along the lines of either 'the package manager will do this for you and it just works (but in case it doesn't, delete node_modules or cargo clean and re-try)', or stand up a dependency server on your own machine with these proxy settings (which has it's own security issues and is frequently disallowed by IT cybersecurity policies). As an example, many blog articles I found a while back suggest using yumdownloader from the yum-utils package. This is unfortunately not reliable, as there are some packages that get skipped. I have found I need to script reading a list of dependencies from a file; then for each dependency: create a directory for it, use repotrack to download its RPM and it's transitive dependency RPMs in the dependency's directory; then the script aggregates all the RPMs into one directory, removes the OS installed RPMs, uses createrepo to turn that directory into a RPM repository, and then makes an USF ISO image out of the directory for transfer onto offline system and installation.

Diggsey 6y ago
I disagree: the problem is not that package managers make things easy, it's just that several of them are poorly designed. The fact that pip/npm/gem etc. look for packages in a fallback location if not found in the private repository is a terrible design flaw. One which not all package managers have. For example, when you add a cargo dependency from a private registry, you have to specify the registry that the dependency comes from, so cargo will never go looking in some other place for that crate. I'm sure many other package managers also have designs that are not vulnerable in this way. Similarly, many package managers do not support pinning of transitive dependencies (with hashes), or pinning does not happen by default, so that many people are still using floating dependencies.
- specialp 6y ago
  In the case of RubyGems for some time now it has been throwing a warning if you do not use the `source` block to scope for gems coming from multiple gemservers.
- Spooky23 6y ago
  It's a little bit of both. Maybe "problem" is the wrong word. It's a risk that you need to understand and account for. If you're running a bank, it's an existential impact that you must avoid. If you're running a message board, it's not. Look at what happened when the "left-pad" function disappeared from npm a few years ago. IIRC, it broke react. The downside of package managers like this is that many people have no idea what they are using.
  - ryandrake 6y ago
    Coming from the embedded world, where a lot of projects are safety-critical, it always kind of shocks me to see how cavalier others in the software world are about bringing in third party dependencies. Need a bit of code that would take you a day to write? Naaah, just find a third party library that does it (and does god knows what else). And bam! Like that it's part of the build. No code review of the dependency. No security audit. No investigation of what other dependencies that dependency brings in. No investigation into license compatibility. Just slide it into the list of 200(!) other dependencies. Maybe I'm a dinosaur, but I was taught a long time ago to use a dependency only if there was no other feasible alternative, and to thoroughly know every dependency you bring in as if it were your own code, and treat it as such. Because at the end of the day, you're shipping that code to the customer and stamping your name or your company's name on it. It's your reputation on the line, not the person behind the StudMaster69 GitHub account.
    - adamc 6y ago
      Except a lot of projects aren't shipping anything to anyone, they are providing a service. There, you have to assess the effect if the service goes down or is compromised. There is a wide range of significances. If Roll20 (D&D tabletop site) goes down, it may affect their revenue, but no one is going to get hurt. Etc.
    - bluefirebrand 6y ago
      > Need a bit of code that would take you a day to write? Even if it is a small library that has only a single maintainer the chances of you replicating it in a day seem slim to me unless it is truly trivial, or the library was also written in a day. More likely you get a day in and realize that the problem has a whole bunch of gotchas that you didn't anticipate and the library maintainer already found and dealt with. Again, this is only if the problem isn't truly trivial
- austincheney 6y ago
  Whether the package managers are poorly designed is completely ancillary. It really is primarily about developer laziness, incompetence, easiness. Proof: https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ Sudden unplanned loss of availability is a catastrophic security problem, the A in the security CIA[1]. Worse is that the dependency that caused that problem was something that should never have been a dependency in the first place. Proper dependency management requires a degree of trust and integrity validation which are completely counter to automation. Most developers are eager to accept any resulting consequences because they don't own the consequences and because they are fearful of writing original code. [1] https://en.wikipedia.org/wiki/Information_security#Key_concepts https://en.wikipedia.org/wiki/Information_security#Key_conce...
  - ritchiea 6y ago
    You can call it laziness but lots of developers correctly assume they'd be out of their jobs or at best out of favor at their company if they raised a fuss about dependency management rather than use (flawed) industry standard tools and get to work on features. No one gets fired for using npm, you might get fired for insisting you build your own dependency management system because npm is insecure rather than working on your team's domain problem.
    - austincheney 6y ago
      > No one gets fired for using npm Most developers are eager to accept any resulting consequences because they don't own the consequences and because they are fearful of writing original code.
      - ritchiea 6y ago
        Some developers are fearful of writing original code. Others realize it's not going to be appreciated by their colleagues to write their own package manager to solve a problem most of the industry disregards. Imagine arguing for getting the "write our own package manager to replace npm/yarn/pip" ticket into a sprint.
- ywei3410 6y ago
  Does Cargo resolve transitive dependencies with a hash? So for example, if I have a dependency on tokio (which depends on tokio_core), I don't /think/ the meta-data on tokio forces the exact version of tokio_core on a first download/update? In which case, would you not get the same issue, if you do the same attack, but with a transitive dependency which you haven't specified?
- path411 6y ago
  I haven't used private packages, but it astonishes me you don't just add private packages with some kind of flag so it knows to not try to pull a public package. Anyone who uses this must have already understood and just overlooked this vulnerability when they realize their private package must have a unique name that doesn't match a public package
  - thw0rted 6y ago
    I was confused by this aspect of the article. I have scoped, private packages in `@myscope`. I set up my `npmrc` with `@myscope:registry=url/to/my/private/repo`. I just checked, and if I try to install `@myscope/commmon-library`, when it's not found on our repo, it will fail, because `npm` has associated `@myscope` with one and only one (private) registry. The only hiccup is that if I'm a new developer, and I haven't made this entry in my user-level `npmrc`, and I'm not using an existing project with a `.npmrc` in the project root, then it will try to hit `@myscope` on the public repo. But if I don't have the registry for `@myscope` configured at all, my actual dependencies won't work, so I should notice that right away. If nothing else, I suppose the takeaway is to grab the group scope for whatever private scope you choose on the public repo before somebody else squats on it. Still, at least for NPM, this seems like a solved problem, you just have to implement the existing solution correctly.

Diggsey 6y ago

I disagree: the problem is not that package managers make things easy, it's just that several of them are poorly designed. The fact that pip/npm/gem etc. look for packages in a fallback location if not found in the private repository is a terrible design flaw. One which not all package managers have. For example, when you add a cargo dependency from a private registry, you have to specify the registry that the dependency comes from, so cargo will never go looking in some other place for that crate. I'm sure many other package managers also have designs that are not vulnerable in this way. Similarly, many package managers do not support pinning of transitive dependencies (with hashes), or pinning does not happen by default, so that many people are still using floating dependencies.

specialp 6y ago
In the case of RubyGems for some time now it has been throwing a warning if you do not use the `source` block to scope for gems coming from multiple gemservers.
Spooky23 6y ago
It's a little bit of both. Maybe "problem" is the wrong word. It's a risk that you need to understand and account for. If you're running a bank, it's an existential impact that you must avoid. If you're running a message board, it's not. Look at what happened when the "left-pad" function disappeared from npm a few years ago. IIRC, it broke react. The downside of package managers like this is that many people have no idea what they are using.
- ryandrake 6y ago
  Coming from the embedded world, where a lot of projects are safety-critical, it always kind of shocks me to see how cavalier others in the software world are about bringing in third party dependencies. Need a bit of code that would take you a day to write? Naaah, just find a third party library that does it (and does god knows what else). And bam! Like that it's part of the build. No code review of the dependency. No security audit. No investigation of what other dependencies that dependency brings in. No investigation into license compatibility. Just slide it into the list of 200(!) other dependencies. Maybe I'm a dinosaur, but I was taught a long time ago to use a dependency only if there was no other feasible alternative, and to thoroughly know every dependency you bring in as if it were your own code, and treat it as such. Because at the end of the day, you're shipping that code to the customer and stamping your name or your company's name on it. It's your reputation on the line, not the person behind the StudMaster69 GitHub account.
  - adamc 6y ago
    Except a lot of projects aren't shipping anything to anyone, they are providing a service. There, you have to assess the effect if the service goes down or is compromised. There is a wide range of significances. If Roll20 (D&D tabletop site) goes down, it may affect their revenue, but no one is going to get hurt. Etc.
  - bluefirebrand 6y ago
    > Need a bit of code that would take you a day to write? Even if it is a small library that has only a single maintainer the chances of you replicating it in a day seem slim to me unless it is truly trivial, or the library was also written in a day. More likely you get a day in and realize that the problem has a whole bunch of gotchas that you didn't anticipate and the library maintainer already found and dealt with. Again, this is only if the problem isn't truly trivial
austincheney 6y ago
Whether the package managers are poorly designed is completely ancillary. It really is primarily about developer laziness, incompetence, easiness. Proof: https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ Sudden unplanned loss of availability is a catastrophic security problem, the A in the security CIA[1]. Worse is that the dependency that caused that problem was something that should never have been a dependency in the first place. Proper dependency management requires a degree of trust and integrity validation which are completely counter to automation. Most developers are eager to accept any resulting consequences because they don't own the consequences and because they are fearful of writing original code. [1] https://en.wikipedia.org/wiki/Information_security#Key_concepts https://en.wikipedia.org/wiki/Information_security#Key_conce...
- ritchiea 6y ago
  You can call it laziness but lots of developers correctly assume they'd be out of their jobs or at best out of favor at their company if they raised a fuss about dependency management rather than use (flawed) industry standard tools and get to work on features. No one gets fired for using npm, you might get fired for insisting you build your own dependency management system because npm is insecure rather than working on your team's domain problem.
  - austincheney 6y ago
    > No one gets fired for using npm Most developers are eager to accept any resulting consequences because they don't own the consequences and because they are fearful of writing original code.
    - ritchiea 6y ago
      Some developers are fearful of writing original code. Others realize it's not going to be appreciated by their colleagues to write their own package manager to solve a problem most of the industry disregards. Imagine arguing for getting the "write our own package manager to replace npm/yarn/pip" ticket into a sprint.
ywei3410 6y ago
Does Cargo resolve transitive dependencies with a hash? So for example, if I have a dependency on tokio (which depends on tokio_core), I don't /think/ the meta-data on tokio forces the exact version of tokio_core on a first download/update? In which case, would you not get the same issue, if you do the same attack, but with a transitive dependency which you haven't specified?
path411 6y ago
I haven't used private packages, but it astonishes me you don't just add private packages with some kind of flag so it knows to not try to pull a public package. Anyone who uses this must have already understood and just overlooked this vulnerability when they realize their private package must have a unique name that doesn't match a public package
- thw0rted 6y ago
  I was confused by this aspect of the article. I have scoped, private packages in `@myscope`. I set up my `npmrc` with `@myscope:registry=url/to/my/private/repo`. I just checked, and if I try to install `@myscope/commmon-library`, when it's not found on our repo, it will fail, because `npm` has associated `@myscope` with one and only one (private) registry. The only hiccup is that if I'm a new developer, and I haven't made this entry in my user-level `npmrc`, and I'm not using an existing project with a `.npmrc` in the project root, then it will try to hit `@myscope` on the public repo. But if I don't have the registry for `@myscope` configured at all, my actual dependencies won't work, so I should notice that right away. If nothing else, I suppose the takeaway is to grab the group scope for whatever private scope you choose on the public repo before somebody else squats on it. Still, at least for NPM, this seems like a solved problem, you just have to implement the existing solution correctly.