7 ms·

TL;DR: There's nothing fundamentally less worthy of trust about node's supply chain than any other popular mainstream language ecosystem. They've all got some b

TL;DR: There's nothing fundamentally less worthy of trust about node's supply chain than any other popular mainstream language ecosystem. They've all got some badness. --- Here's a trope I'm tired of: Take X general problem that affects a wide range of systems, attribute it to one narrow system Y. Usually because that system Y's general accessibility and success leads to a higher number of high profile incidents related to problem X. One of the practical outcomes of this trope is people trying to solve this problem in narrow, ecosystem-specific, non-portable ways.

hinkley 5y ago
> They've all got some badness. They're all based on the same design, as far as I can see. Is there another architecture we could be using?
mikece 5y ago
Is the trope inaccurate in this case? If so, how? What would a path toward solution look like?
- lucideer 5y ago
  The post has two things to communicate: 1. The Node supply chain has serious problems when it comes to security: this is true 2. The strong implication (throughout the article but most prominently in the title) that these problems are either unique to Node or at least worse in Node than elsewhere. This is definitely not true. --- The solutions cited in the article are good, and definitely benefit the Node ecosystem: --ignore-scripts is a well known option that would be great on-by-default but tends to be enforced in large corp CICD currently. The capabilities model is not too different to what Deno do (this article's idea is somewhat more fine grained but much more complex which could hamper adoption or lead to misuse). Overall both would be of somewhat limited, but still very worthwhile benefit. The hard problem imo is solving more generally for supply chain trust. We implemented --ignore-scripts enforcement last year and have only really see it have a relatively small impact, despite it being relatively new as an idea in the past few years (which motivates more exploits due to its novelty). In reality, most supply chain attacks are either targeting prod envs, or integrated build dependencies, rather than piggybacking CICD repo install steps: the later attack gives you the same env access and has no package-manager-specific mitigation, so it needs a more general solution. Capabilities also have limited use outside install scripts because limiting them also limits the "benign" functionality of your normal application code.
mschuster91 5y ago
The proposed system itself should be fairly portable and in its idea reminds me of SELinux. The problem is SELinux knowledge isn't widespread, it's Linux-only (so no support for OS X and Windows which are the majority of JS developer machines), and especially it operates on a syscall level - which is perfectly fine to limit filesystem access (=for all nodejs processes, limit file I/O to the parent directory of node_modules?), but only barely unusable to sandbox network access. Therefore it makes the most sense to define a standard for configuring capabilities/permission whitelists/grant requests/(however else you want to name it) and leave the implementation up to the platform/language.
nukemaster 5y ago
I'd disagree: The way modern Linux distros work is that a number of volunteers who pay attention to what's going on upstream package software for users and developers. When something gets weird these volunteers change their behavior and prevent the users from being harmed (the recent Audacity mess is a great example of this.) I don't think people pushing eg cargo/node/snap appreciate how safe this has made their OS and languages that rely on distro package managers (such as C.) You get the safety that you might expect from an app store without actually restricting anyone's freedom. It's very much like a church with elders preventing people from being captured by vices and whatnot. Yes it has issues but it works shockingly well, much better than many of the alternatives.
- rlpb 5y ago
  Also, most distros have the concept of stable releases. This provides a very valuable "focus point". It means that we don't just have to rely on the maintainers. Users can review packages for reasonable behavior too, and this isn't made futile by constantly changing packages. Users and maintainers can focus on just the stable release and at a reasonable cadence, and this focus point being the same for all users means that it has value to everyone else, too.
- lucideer 5y ago
  This is ultimately about scale though. It's easier for Linux because of the relative number of contributors to distro repos. Ubuntu has 10s of thousands of packages. NPM has well over a million. The average update frequency is also much higher, as is the number of contributors per-package.
  - infamia 5y ago
    > Ubuntu has 10s of thousands of packages. NPM has well over a million. The average update frequency is also much higher, as is the number of contributors per-package. Node's culture of tiny libraries (partly caused by Javascript's tiny standard lib) is a big part of the problem and increases the number of potential supply chain issues.
zozbot234 5y ago
Other popular mainstream languages don't need a "leftpad" package in the first place.
- onion2k 5y ago
  JS didn't need it either. A dev made the library, shared it, and some developers decided to use it. That's on them. Nothing about JS made it necessary.
  - hombre_fatal 5y ago
    Also, here's how to publish a package: mkdir my-package npm init npm publish
    - colejohnson66 5y ago
      Same with Rust. Any language with a package manager has this issue. cargo new my-crate cargo login ... cargo publish
- ferdowsi 5y ago
  Node has a more robust standard library than Rust, which forces devs to download third party libraries to compute a regular expression or generate a SHA-256. For now Node is a richer target due to its popularity but the same issues will hit any language ecosystem that suffer the same flaws should they become popular.
  - jhugo 5y ago
    "Third-party" isn't really correct for regex: https://github.com/rust-lang/regex https://github.com/rust-lang/regex (note the org) Rust's stdlib is deliberately small in order to allow it to have very strong stability guarantees.
dahfizz 5y ago
The issue is especially bad, and needs to be especially called out, for npm because of the insane dependency bloat. You can write fully featured and useful python apps with just the standard packages that come with python. In JS you need a handful of third party packages just to tell if a number is odd. A difference in magnitude becomes a difference in kind.
- Hamuko 5y ago
  >In JS you need a handful of third party packages just to tell if a number is odd. You really don't, but for some reason JS developers have decided that this is the optimal way to check if a number is odd.
  - bavell 5y ago
    > You really don't, but for some reason JS interns have decided that this is the optimal way to check if a number is odd. FTFY :)
    - Hamuko 5y ago
      is-odd gets about two million downloads a month. is-number gets 278 million downloads a month. I guess the interns have been very busy.
      - pfraze 5y ago
        Those numbers are a little deceiving. It’s likely that those modules are upstream of a popular module or two. It’s not like they are installed by projects directly.
        hombre_fatal 5y ago
        It's only directly depended on by 40 or so npm modules, and most of them look like beginner/joke modules like "is-ten-thousand" where they brag about being featured in a "worst npm libraries" article. Quickly in these sorts of conversations, we're really just making fun of beginners for using dumb packages. With the popularity of Node, it makes sense to me that you can have beginners searching npm/google for how to tell if a number is odd, and for whatever reason they find is-odd or whatever. And perhaps the idea of even collecting packages to do simple things is something fun for them. I remember having a weird maximalist attitude when I was a beginner using Rails. I'd install a Ruby gem and I could use it anywhere in my files without even importing it, usually a single line of abstraction. For some reason that appealed to me even though I could have done it myself. I think I had this idea that people writing libraries were doing things right, and I was right for tapping into them. I think we should save our denigration for the serious, popular projects that use packages rather than the fact that beginners use them. Like, this sort of package has no business being a transitive dep of Express (it isn't) and popular Express middleware, and mainly because x-deps are security issues.
josephg 5y ago
Hi! Author here. I write a lot more Rust than Javascript these days, and I agree - this is absolutely a problem I'd love to eventually see addressed in the rust ecosystem as well. (Although dependency madness hasn't kicked in anywhere near as much in the rust ecosystem). But rust and javascript each have unique challenges. Per-library sandboxing is hard in javascript because of the language's dynamism. Its hard in rust because any code you pull in can always drop down to unsafe, and in unsafe land you can really do whatever you want. We could probably selectively ban eval() in modern server-side javascript pretty easily because its just not used that much in modern code. But we can't ban unsafe in rust because its sprinkled everywhere. > One of the practical outcomes of this trope is people trying to solve this problem in narrow, ecosystem-specific, non-portable ways. I think trying to solve this in every language at the same time would be an exercise in boiling the ocean. There's no need to implement something like this in every language all at once. Much better to start somewhere, experiment, and hopefully if the solution works well we can see how it might apply in other languages.
- zozbot234 5y ago
  > Its hard in rust because any code you pull in can always drop down to unsafe, and in unsafe land you can really do whatever you want There is a quasi-standard "cargo geiger" tool that can manage unsafety requirements in Rust imported crates. (Of course, this should really be an officially provided feature in the first place.)
- lucideer 5y ago
  > I think trying to solve this in every language at the same time would be an exercise in boiling the ocean. I actually think the opposite is true. Capabilities are great but limited in their applicability within actual application code. Ultimately, your main application that may require some "dangerous" APIs to do something benign, may be loading 3rd party code to wrap those APIs, and we need a way to trust that code. That's a general problem, and - while it is a very hard one to solve - coming up with novel and unique solutions in every ecosystem is going to reinvent a lot more wheels (& boil some seas at least) compared to looking at software composition holistically.
goodpoint 5y ago
Distributions like Debian are very trustworthy especially when compared to NPM.
- lucideer 5y ago
  Debian has ~20,000-30,000 packages total. NPM has well over a million. Contribution frequency and overall contributor numbers are also much much higher. NPM is a victim of ease of use and popularity. It's a bigger target. But both systems would benefit from a holistic approach to supply chain security.
  - tonyedgecombe 5y ago
    Part of the problem there is JS doesn't have much of a standard library so everybody and their dog tries to fill that hole.
- codedokode 5y ago
  Sadly, in Debian every package which gets installed, can run a script with root privileges so this is even more dangerous. Although most packages don't need such privileges.
  - goodpoint 5y ago
    Nonsense. Package managers need to be able to run scripts as root to do the installation. And yet, in the last 25 or 30 years there's never been a case of a malicious contributor successfully inserting a backdoor in the installation script of any package in any major distribution. Because there is a vetting process, nothing else. [And yes, of course, it would be possible to sandbox each package installation to access some very specific paths but so far it's really unnecessary]
deleted 5y ago
[deleted]
marcosdumay 5y ago
Of course node is less trustworthy than other platforms. It it wasn't, we would see active exploits on other platforms as well, but they are either incredibly rare or non-existent. But you are right in that it's not a difference of kind. Except for the inane idea of executing random code on library installation, node doesn't have any kind of vulnerability that isn't shared with every package manager out there. The important differences are social on the community, and of exposure area, because JS programs tend to rely on 1 or 2 orders of magnitude more developers than other languages.
- lucideer 5y ago
  100% You've summarised it much more succinctly than I could. Fundamentally it's about misattributing the cause of untrustworthiness.
danShumway 5y ago
> One of the practical outcomes of this trope is people trying to solve this problem in narrow, ecosystem-specific, non-portable ways. In general I agree, but I disagree that conversations about dependency scopes are ecosystem-specific. Frankly, figuring out how to limit the capabilities of imports is a discussion that literally every language with a package manager should be having right now. Sure, the implementation is always going to be platform/language-specific, but this should be a serious consideration that people have any time that they're designing a new package system, we should expect new package managers to have an answer about how they scope dependencies and handle dependency permissions.

hinkley 5y ago

> They've all got some badness. They're all based on the same design, as far as I can see. Is there another architecture we could be using?

mikece 5y ago

Is the trope inaccurate in this case? If so, how? What would a path toward solution look like?

lucideer 5y ago
The post has two things to communicate: 1. The Node supply chain has serious problems when it comes to security: this is true 2. The strong implication (throughout the article but most prominently in the title) that these problems are either unique to Node or at least worse in Node than elsewhere. This is definitely not true. --- The solutions cited in the article are good, and definitely benefit the Node ecosystem: --ignore-scripts is a well known option that would be great on-by-default but tends to be enforced in large corp CICD currently. The capabilities model is not too different to what Deno do (this article's idea is somewhat more fine grained but much more complex which could hamper adoption or lead to misuse). Overall both would be of somewhat limited, but still very worthwhile benefit. The hard problem imo is solving more generally for supply chain trust. We implemented --ignore-scripts enforcement last year and have only really see it have a relatively small impact, despite it being relatively new as an idea in the past few years (which motivates more exploits due to its novelty). In reality, most supply chain attacks are either targeting prod envs, or integrated build dependencies, rather than piggybacking CICD repo install steps: the later attack gives you the same env access and has no package-manager-specific mitigation, so it needs a more general solution. Capabilities also have limited use outside install scripts because limiting them also limits the "benign" functionality of your normal application code.

mschuster91 5y ago

The proposed system itself should be fairly portable and in its idea reminds me of SELinux. The problem is SELinux knowledge isn't widespread, it's Linux-only (so no support for OS X and Windows which are the majority of JS developer machines), and especially it operates on a syscall level - which is perfectly fine to limit filesystem access (=for all nodejs processes, limit file I/O to the parent directory of node_modules?), but only barely unusable to sandbox network access. Therefore it makes the most sense to define a standard for configuring capabilities/permission whitelists/grant requests/(however else you want to name it) and leave the implementation up to the platform/language.

nukemaster 5y ago

I'd disagree: The way modern Linux distros work is that a number of volunteers who pay attention to what's going on upstream package software for users and developers. When something gets weird these volunteers change their behavior and prevent the users from being harmed (the recent Audacity mess is a great example of this.) I don't think people pushing eg cargo/node/snap appreciate how safe this has made their OS and languages that rely on distro package managers (such as C.) You get the safety that you might expect from an app store without actually restricting anyone's freedom. It's very much like a church with elders preventing people from being captured by vices and whatnot. Yes it has issues but it works shockingly well, much better than many of the alternatives.

rlpb 5y ago
Also, most distros have the concept of stable releases. This provides a very valuable "focus point". It means that we don't just have to rely on the maintainers. Users can review packages for reasonable behavior too, and this isn't made futile by constantly changing packages. Users and maintainers can focus on just the stable release and at a reasonable cadence, and this focus point being the same for all users means that it has value to everyone else, too.
lucideer 5y ago
This is ultimately about scale though. It's easier for Linux because of the relative number of contributors to distro repos. Ubuntu has 10s of thousands of packages. NPM has well over a million. The average update frequency is also much higher, as is the number of contributors per-package.
- infamia 5y ago
  > Ubuntu has 10s of thousands of packages. NPM has well over a million. The average update frequency is also much higher, as is the number of contributors per-package. Node's culture of tiny libraries (partly caused by Javascript's tiny standard lib) is a big part of the problem and increases the number of potential supply chain issues.

zozbot234 5y ago

Other popular mainstream languages don't need a "leftpad" package in the first place.

onion2k 5y ago
JS didn't need it either. A dev made the library, shared it, and some developers decided to use it. That's on them. Nothing about JS made it necessary.
- hombre_fatal 5y ago
  Also, here's how to publish a package: mkdir my-package npm init npm publish
  - colejohnson66 5y ago
    Same with Rust. Any language with a package manager has this issue. cargo new my-crate cargo login ... cargo publish
ferdowsi 5y ago
Node has a more robust standard library than Rust, which forces devs to download third party libraries to compute a regular expression or generate a SHA-256. For now Node is a richer target due to its popularity but the same issues will hit any language ecosystem that suffer the same flaws should they become popular.
- jhugo 5y ago
  "Third-party" isn't really correct for regex: https://github.com/rust-lang/regex https://github.com/rust-lang/regex (note the org) Rust's stdlib is deliberately small in order to allow it to have very strong stability guarantees.

dahfizz 5y ago

The issue is especially bad, and needs to be especially called out, for npm because of the insane dependency bloat. You can write fully featured and useful python apps with just the standard packages that come with python. In JS you need a handful of third party packages just to tell if a number is odd. A difference in magnitude becomes a difference in kind.

Hamuko 5y ago
>In JS you need a handful of third party packages just to tell if a number is odd. You really don't, but for some reason JS developers have decided that this is the optimal way to check if a number is odd.
- bavell 5y ago
  > You really don't, but for some reason JS interns have decided that this is the optimal way to check if a number is odd. FTFY :)
  - Hamuko 5y ago
    is-odd gets about two million downloads a month. is-number gets 278 million downloads a month. I guess the interns have been very busy.
    - pfraze 5y ago
      Those numbers are a little deceiving. It’s likely that those modules are upstream of a popular module or two. It’s not like they are installed by projects directly.
      - hombre_fatal 5y ago
        It's only directly depended on by 40 or so npm modules, and most of them look like beginner/joke modules like "is-ten-thousand" where they brag about being featured in a "worst npm libraries" article. Quickly in these sorts of conversations, we're really just making fun of beginners for using dumb packages. With the popularity of Node, it makes sense to me that you can have beginners searching npm/google for how to tell if a number is odd, and for whatever reason they find is-odd or whatever. And perhaps the idea of even collecting packages to do simple things is something fun for them. I remember having a weird maximalist attitude when I was a beginner using Rails. I'd install a Ruby gem and I could use it anywhere in my files without even importing it, usually a single line of abstraction. For some reason that appealed to me even though I could have done it myself. I think I had this idea that people writing libraries were doing things right, and I was right for tapping into them. I think we should save our denigration for the serious, popular projects that use packages rather than the fact that beginners use them. Like, this sort of package has no business being a transitive dep of Express (it isn't) and popular Express middleware, and mainly because x-deps are security issues.

josephg 5y ago

Hi! Author here. I write a lot more Rust than Javascript these days, and I agree - this is absolutely a problem I'd love to eventually see addressed in the rust ecosystem as well. (Although dependency madness hasn't kicked in anywhere near as much in the rust ecosystem). But rust and javascript each have unique challenges. Per-library sandboxing is hard in javascript because of the language's dynamism. Its hard in rust because any code you pull in can always drop down to unsafe, and in unsafe land you can really do whatever you want. We could probably selectively ban eval() in modern server-side javascript pretty easily because its just not used that much in modern code. But we can't ban unsafe in rust because its sprinkled everywhere. > One of the practical outcomes of this trope is people trying to solve this problem in narrow, ecosystem-specific, non-portable ways. I think trying to solve this in every language at the same time would be an exercise in boiling the ocean. There's no need to implement something like this in every language all at once. Much better to start somewhere, experiment, and hopefully if the solution works well we can see how it might apply in other languages.

zozbot234 5y ago
> Its hard in rust because any code you pull in can always drop down to unsafe, and in unsafe land you can really do whatever you want There is a quasi-standard "cargo geiger" tool that can manage unsafety requirements in Rust imported crates. (Of course, this should really be an officially provided feature in the first place.)
lucideer 5y ago
> I think trying to solve this in every language at the same time would be an exercise in boiling the ocean. I actually think the opposite is true. Capabilities are great but limited in their applicability within actual application code. Ultimately, your main application that may require some "dangerous" APIs to do something benign, may be loading 3rd party code to wrap those APIs, and we need a way to trust that code. That's a general problem, and - while it is a very hard one to solve - coming up with novel and unique solutions in every ecosystem is going to reinvent a lot more wheels (& boil some seas at least) compared to looking at software composition holistically.

goodpoint 5y ago

Distributions like Debian are very trustworthy especially when compared to NPM.

lucideer 5y ago
Debian has ~20,000-30,000 packages total. NPM has well over a million. Contribution frequency and overall contributor numbers are also much much higher. NPM is a victim of ease of use and popularity. It's a bigger target. But both systems would benefit from a holistic approach to supply chain security.
- tonyedgecombe 5y ago
  Part of the problem there is JS doesn't have much of a standard library so everybody and their dog tries to fill that hole.
codedokode 5y ago
Sadly, in Debian every package which gets installed, can run a script with root privileges so this is even more dangerous. Although most packages don't need such privileges.
- goodpoint 5y ago
  Nonsense. Package managers need to be able to run scripts as root to do the installation. And yet, in the last 25 or 30 years there's never been a case of a malicious contributor successfully inserting a backdoor in the installation script of any package in any major distribution. Because there is a vetting process, nothing else. [And yes, of course, it would be possible to sandbox each package installation to access some very specific paths but so far it's really unnecessary]

deleted 5y ago

[deleted]

marcosdumay 5y ago

Of course node is less trustworthy than other platforms. It it wasn't, we would see active exploits on other platforms as well, but they are either incredibly rare or non-existent. But you are right in that it's not a difference of kind. Except for the inane idea of executing random code on library installation, node doesn't have any kind of vulnerability that isn't shared with every package manager out there. The important differences are social on the community, and of exposure area, because JS programs tend to rely on 1 or 2 orders of magnitude more developers than other languages.

lucideer 5y ago
100% You've summarised it much more succinctly than I could. Fundamentally it's about misattributing the cause of untrustworthiness.

danShumway 5y ago

> One of the practical outcomes of this trope is people trying to solve this problem in narrow, ecosystem-specific, non-portable ways. In general I agree, but I disagree that conversations about dependency scopes are ecosystem-specific. Frankly, figuring out how to limit the capabilities of imports is a discussion that literally every language with a package manager should be having right now. Sure, the implementation is always going to be platform/language-specific, but this should be a serious consideration that people have any time that they're designing a new package system, we should expect new package managers to have an answer about how they scope dependencies and handle dependency permissions.