7 ms·

I disagree: the problem is not that package managers make things easy, it's just that several of them are poorly designed. The fact that pip/npm/gem etc. look

I disagree: the problem is not that package managers make things easy, it's just that several of them are poorly designed. The fact that pip/npm/gem etc. look for packages in a fallback location if not found in the private repository is a terrible design flaw. One which not all package managers have. For example, when you add a cargo dependency from a private registry, you have to specify the registry that the dependency comes from, so cargo will never go looking in some other place for that crate. I'm sure many other package managers also have designs that are not vulnerable in this way. Similarly, many package managers do not support pinning of transitive dependencies (with hashes), or pinning does not happen by default, so that many people are still using floating dependencies.

specialp 6y ago
In the case of RubyGems for some time now it has been throwing a warning if you do not use the `source` block to scope for gems coming from multiple gemservers.
Spooky23 6y ago
It's a little bit of both. Maybe "problem" is the wrong word. It's a risk that you need to understand and account for. If you're running a bank, it's an existential impact that you must avoid. If you're running a message board, it's not. Look at what happened when the "left-pad" function disappeared from npm a few years ago. IIRC, it broke react. The downside of package managers like this is that many people have no idea what they are using.
- ryandrake 6y ago
  Coming from the embedded world, where a lot of projects are safety-critical, it always kind of shocks me to see how cavalier others in the software world are about bringing in third party dependencies. Need a bit of code that would take you a day to write? Naaah, just find a third party library that does it (and does god knows what else). And bam! Like that it's part of the build. No code review of the dependency. No security audit. No investigation of what other dependencies that dependency brings in. No investigation into license compatibility. Just slide it into the list of 200(!) other dependencies. Maybe I'm a dinosaur, but I was taught a long time ago to use a dependency only if there was no other feasible alternative, and to thoroughly know every dependency you bring in as if it were your own code, and treat it as such. Because at the end of the day, you're shipping that code to the customer and stamping your name or your company's name on it. It's your reputation on the line, not the person behind the StudMaster69 GitHub account.
  - adamc 6y ago
    Except a lot of projects aren't shipping anything to anyone, they are providing a service. There, you have to assess the effect if the service goes down or is compromised. There is a wide range of significances. If Roll20 (D&D tabletop site) goes down, it may affect their revenue, but no one is going to get hurt. Etc.
  - bluefirebrand 6y ago
    > Need a bit of code that would take you a day to write? Even if it is a small library that has only a single maintainer the chances of you replicating it in a day seem slim to me unless it is truly trivial, or the library was also written in a day. More likely you get a day in and realize that the problem has a whole bunch of gotchas that you didn't anticipate and the library maintainer already found and dealt with. Again, this is only if the problem isn't truly trivial
    - Robotbeat 6y ago
      Usually you don’t need the entire functionality of a library. So writing something for your use case may only take a day.
  - edenhyacinth 6y ago
    The left-pad situation wasn't simply that lots of projects were relying on left-pad directly, it was that they were relying on projects that were relying on projects that were relying on left-pad. Some dependencies are too large to rewrite yourself - most statistical suites would fall under this definition - and while accepting their direct code might be acceptable, it's not usually feasible to fork their code and rewrite the parts that aren't. Lots of smaller parts you could write yourself quickly add up.
- _ph_ 6y ago
  Indeed. They make it far to tempting to just pull in a dependency, even if it is not really needed. The worst case of this are one-function packages in npm. And of course whenever you pull in a dependency, that might in a cascade pull in more dependencies. Somteimes the same package is pulled in several times, even in different versions. What looks elegant as a concept "we just have a graph of dependencies and automatically pull that in" quickly becomes an unmaintainable nightmare and consequently into a huge attack vector.
austincheney 6y ago
Whether the package managers are poorly designed is completely ancillary. It really is primarily about developer laziness, incompetence, easiness. Proof: https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ Sudden unplanned loss of availability is a catastrophic security problem, the A in the security CIA[1]. Worse is that the dependency that caused that problem was something that should never have been a dependency in the first place. Proper dependency management requires a degree of trust and integrity validation which are completely counter to automation. Most developers are eager to accept any resulting consequences because they don't own the consequences and because they are fearful of writing original code. [1] https://en.wikipedia.org/wiki/Information_security#Key_concepts https://en.wikipedia.org/wiki/Information_security#Key_conce...
- ritchiea 6y ago
  You can call it laziness but lots of developers correctly assume they'd be out of their jobs or at best out of favor at their company if they raised a fuss about dependency management rather than use (flawed) industry standard tools and get to work on features. No one gets fired for using npm, you might get fired for insisting you build your own dependency management system because npm is insecure rather than working on your team's domain problem.
  - austincheney 6y ago
    > No one gets fired for using npm Most developers are eager to accept any resulting consequences because they don't own the consequences and because they are fearful of writing original code.
    - ritchiea 6y ago
      Some developers are fearful of writing original code. Others realize it's not going to be appreciated by their colleagues to write their own package manager to solve a problem most of the industry disregards. Imagine arguing for getting the "write our own package manager to replace npm/yarn/pip" ticket into a sprint.
ywei3410 6y ago
Does Cargo resolve transitive dependencies with a hash? So for example, if I have a dependency on tokio (which depends on tokio_core), I don't /think/ the meta-data on tokio forces the exact version of tokio_core on a first download/update? In which case, would you not get the same issue, if you do the same attack, but with a transitive dependency which you haven't specified?
path411 6y ago
I haven't used private packages, but it astonishes me you don't just add private packages with some kind of flag so it knows to not try to pull a public package. Anyone who uses this must have already understood and just overlooked this vulnerability when they realize their private package must have a unique name that doesn't match a public package
- thw0rted 6y ago
  I was confused by this aspect of the article. I have scoped, private packages in `@myscope`. I set up my `npmrc` with `@myscope:registry=url/to/my/private/repo`. I just checked, and if I try to install `@myscope/commmon-library`, when it's not found on our repo, it will fail, because `npm` has associated `@myscope` with one and only one (private) registry. The only hiccup is that if I'm a new developer, and I haven't made this entry in my user-level `npmrc`, and I'm not using an existing project with a `.npmrc` in the project root, then it will try to hit `@myscope` on the public repo. But if I don't have the registry for `@myscope` configured at all, my actual dependencies won't work, so I should notice that right away. If nothing else, I suppose the takeaway is to grab the group scope for whatever private scope you choose on the public repo before somebody else squats on it. Still, at least for NPM, this seems like a solved problem, you just have to implement the existing solution correctly.

specialp 6y ago

In the case of RubyGems for some time now it has been throwing a warning if you do not use the `source` block to scope for gems coming from multiple gemservers.

Spooky23 6y ago

It's a little bit of both. Maybe "problem" is the wrong word. It's a risk that you need to understand and account for. If you're running a bank, it's an existential impact that you must avoid. If you're running a message board, it's not. Look at what happened when the "left-pad" function disappeared from npm a few years ago. IIRC, it broke react. The downside of package managers like this is that many people have no idea what they are using.

ryandrake 6y ago
Coming from the embedded world, where a lot of projects are safety-critical, it always kind of shocks me to see how cavalier others in the software world are about bringing in third party dependencies. Need a bit of code that would take you a day to write? Naaah, just find a third party library that does it (and does god knows what else). And bam! Like that it's part of the build. No code review of the dependency. No security audit. No investigation of what other dependencies that dependency brings in. No investigation into license compatibility. Just slide it into the list of 200(!) other dependencies. Maybe I'm a dinosaur, but I was taught a long time ago to use a dependency only if there was no other feasible alternative, and to thoroughly know every dependency you bring in as if it were your own code, and treat it as such. Because at the end of the day, you're shipping that code to the customer and stamping your name or your company's name on it. It's your reputation on the line, not the person behind the StudMaster69 GitHub account.
- adamc 6y ago
  Except a lot of projects aren't shipping anything to anyone, they are providing a service. There, you have to assess the effect if the service goes down or is compromised. There is a wide range of significances. If Roll20 (D&D tabletop site) goes down, it may affect their revenue, but no one is going to get hurt. Etc.
- bluefirebrand 6y ago
  > Need a bit of code that would take you a day to write? Even if it is a small library that has only a single maintainer the chances of you replicating it in a day seem slim to me unless it is truly trivial, or the library was also written in a day. More likely you get a day in and realize that the problem has a whole bunch of gotchas that you didn't anticipate and the library maintainer already found and dealt with. Again, this is only if the problem isn't truly trivial
  - Robotbeat 6y ago
    Usually you don’t need the entire functionality of a library. So writing something for your use case may only take a day.
- edenhyacinth 6y ago
  The left-pad situation wasn't simply that lots of projects were relying on left-pad directly, it was that they were relying on projects that were relying on projects that were relying on left-pad. Some dependencies are too large to rewrite yourself - most statistical suites would fall under this definition - and while accepting their direct code might be acceptable, it's not usually feasible to fork their code and rewrite the parts that aren't. Lots of smaller parts you could write yourself quickly add up.
_ph_ 6y ago
Indeed. They make it far to tempting to just pull in a dependency, even if it is not really needed. The worst case of this are one-function packages in npm. And of course whenever you pull in a dependency, that might in a cascade pull in more dependencies. Somteimes the same package is pulled in several times, even in different versions. What looks elegant as a concept "we just have a graph of dependencies and automatically pull that in" quickly becomes an unmaintainable nightmare and consequently into a huge attack vector.

austincheney 6y ago

Whether the package managers are poorly designed is completely ancillary. It really is primarily about developer laziness, incompetence, easiness. Proof: https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ Sudden unplanned loss of availability is a catastrophic security problem, the A in the security CIA[1]. Worse is that the dependency that caused that problem was something that should never have been a dependency in the first place. Proper dependency management requires a degree of trust and integrity validation which are completely counter to automation. Most developers are eager to accept any resulting consequences because they don't own the consequences and because they are fearful of writing original code. [1] https://en.wikipedia.org/wiki/Information_security#Key_concepts https://en.wikipedia.org/wiki/Information_security#Key_conce...

ritchiea 6y ago
You can call it laziness but lots of developers correctly assume they'd be out of their jobs or at best out of favor at their company if they raised a fuss about dependency management rather than use (flawed) industry standard tools and get to work on features. No one gets fired for using npm, you might get fired for insisting you build your own dependency management system because npm is insecure rather than working on your team's domain problem.
- austincheney 6y ago
  > No one gets fired for using npm Most developers are eager to accept any resulting consequences because they don't own the consequences and because they are fearful of writing original code.
  - ritchiea 6y ago
    Some developers are fearful of writing original code. Others realize it's not going to be appreciated by their colleagues to write their own package manager to solve a problem most of the industry disregards. Imagine arguing for getting the "write our own package manager to replace npm/yarn/pip" ticket into a sprint.

ywei3410 6y ago

Does Cargo resolve transitive dependencies with a hash? So for example, if I have a dependency on tokio (which depends on tokio_core), I don't /think/ the meta-data on tokio forces the exact version of tokio_core on a first download/update? In which case, would you not get the same issue, if you do the same attack, but with a transitive dependency which you haven't specified?

path411 6y ago

I haven't used private packages, but it astonishes me you don't just add private packages with some kind of flag so it knows to not try to pull a public package. Anyone who uses this must have already understood and just overlooked this vulnerability when they realize their private package must have a unique name that doesn't match a public package

thw0rted 6y ago
I was confused by this aspect of the article. I have scoped, private packages in `@myscope`. I set up my `npmrc` with `@myscope:registry=url/to/my/private/repo`. I just checked, and if I try to install `@myscope/commmon-library`, when it's not found on our repo, it will fail, because `npm` has associated `@myscope` with one and only one (private) registry. The only hiccup is that if I'm a new developer, and I haven't made this entry in my user-level `npmrc`, and I'm not using an existing project with a `.npmrc` in the project root, then it will try to hit `@myscope` on the public repo. But if I don't have the registry for `@myscope` configured at all, my actual dependencies won't work, so I should notice that right away. If nothing else, I suppose the takeaway is to grab the group scope for whatever private scope you choose on the public repo before somebody else squats on it. Still, at least for NPM, this seems like a solved problem, you just have to implement the existing solution correctly.