4 ms·

What are the actual guarantees that go/Rust make that Python/npm don’t? It seems like it might just be that Python/npm are juicier targets? I’m starting to try

What are the actual guarantees that go/Rust make that Python/npm don’t? It seems like it might just be that Python/npm are juicier targets? I’m starting to try and avoid all third party packages

cookiengineer 5mo ago
I suppose that go's go:generate workflow can also be abused to land a worm like the ones spreading via npm, as you can build programs that just scrape the whole hard drive for git projects and patch the go.mod dependencies there, and you could also just write this in go as a toolchain script, for example. NPM's achilles is the pre/postinstall step which can run arbitrary commands and shell scripts without the user having any way to intervene. Dependencies must be run in isolated chroot sandboxes or better, inside containers. That would be the only way to mitigate this problem, as the filesystem of the operating system must be separated from the filesystem of the development workflow. On top of that most host based firewalls are per-binary instead of per-cmdline. That leads to the warnings and rules relying on that e.g. "python" or "nodejs" getting network access allowlisted, instead of say "nodejs myworm.js". So firewalls in general are pretty useless against this type of malware.
- yegle 5mo ago
  `go:generate` is for the package provider, the command never runs when someone `go install` or `go get` the package.
  - cookiengineer 5mo ago
    Note that the NPM worms are spreading because the package providers are developing on their libraries without them noticing a malicious dependency. It is not users/consumers spreading the worm, it is developers spreading it. Your mismatch is that you think in policies, not assessments here. Nothing in my normal go workflow will ask me if I want to run "curl download whatever from the internet" when I run go build. Though I agree with the difference in workflow, there is not a single mechanism in go catching this. go.mod files can be just patched by the worm, and/or hidden behind a /v123 folder or whatever to play shenanigans on API differences.
- xena 5mo ago
  go:generate is done at dev time, not at build time.
  - cookiengineer 5mo ago
    Actually bindings are usually generated like that, at build time (though with a build cache that nobody knows how it corrupts all the time). Examples that come to mind: webview/webview, webkit, cilium/ebpf and most other CGo projects that I have seen.
    - arccy 5mo ago
      you only run them for your own project, not the generate directives of your dependencies though
jollyllama 5mo ago
> It seems like it might just be that Python/npm are juicier targets? Attackers go where the victims are. Frontend is a monoculture with the vast majority using NPM; backend, less so. This isn't an excuse for NPM, but another strike against it. You could also argue that the attacks make a deeper point about frontend vs backend devs, but I won't go there.
- bichiliad 5mo ago
  Why would you even imply something like that?
  - voidfunc 5mo ago
    [flagged]
    - rembicilious 5mo ago
      Is this a dogfooding joke?
  - llbbdd 5mo ago
    They feel the need to compete given that jokes about "backend" devs write themselves
    - lucketone 5mo ago
      Frontend has lower barrier of entry and more appeal for beginners, so its bell curve might have its left edge is thicker. It impacts the avg of problems and culture of dealing with them
      - llbbdd 5mo ago
        More appeal I agree because it's easier to see useful results and iterate quicker. Lower barrier of entry I disagree with strongly; if the barrier to entry were so low I don't know why I've worked with so many otherwise-talented backend devs that can't wrap their heads around the frontend to save their lives. Frontend forces you to deal with real-world customer problems sooner rather than later; performance is more important, it has to work on more than one environment, you have a frame budget. It's like saying game development has a low barrier to entry; you might be able to get started quickly but you will run into constraints unless you learn fast. On the backend you can just pay another dollar for a VPS twice the size.
panzi 5mo ago
Last I checked npm had 2FA for publishing, but cargo didn't. I don't think cargo is any better than npm, just not that of an attractive target.
lostglass 5mo ago
To be honest Rust has the exact same supply chain attack pattern - it's just newer and more maintained at the moment. Give it a decade.
- nothinkjustai 5mo ago
  Rust doesn’t have post install scripts
  - tasn 5mo ago
    It has build.rs, which has essentially the same problems.
  - deeebug 5mo ago
    They have build.rs (https://doc.rust-lang.org/cargo/reference/build-scripts.html https://doc.rust-lang.org/cargo/reference/build-scripts.html)
  - fabrice_d 5mo ago
    It has build.rs that will run as soon as you compile the dependency. That's not the same thing but pretty close to a post install script: it's very likely to run.
  - est31 5mo ago
    There is build.rs, proc macros are unsandboxed, and lastly you install the binary so that you can run it. Even if the build and install were fully sandboxed, the binary could still do malicious stuff if ran.
    - drdaeman 5mo ago
      Even without post-install script, a malicious payload could be hiding in some function and just wait until the developer invokes `cargo run`. Not that many people audit the crates they pull into their projects.
    - nothinkjustai 5mo ago
jiggawatts 5mo ago
Generally, other package managers aren't great either. Notably, crates.io / cargo has some of the same issues as NPM and the verbiage of their excuses for not fixing these problems is oddly similar. Something fascinating about the design and architecture of programming languages and their surrounding ecosystems is the enormous leverage that they provide to the "core team": For every 1 core language developer[1]... ... there may be 1,000 popular package developers... ... for which there may be 1,000,000 developers writing software... ... for over 1,000,000,000 users. This means that for every corner that is cut at the top of that pyramid, the harms are massively magnified at the lower tiers. A security vulnerability in a "top one thousand" package like log4j can cause billions of dollars in economic damage, man-centuries of remediation effort, etc. However, bizarrely, the funding at the top two levels is essentially a pittance! Most such projects are charities, begging for spare change with hat in hand on a street corner. Some of the most used libraries are often volunteer efforts, despite powering global e-commerce! cough-OpenSSL-cough. The result is that the people most empowered to fix the issues are the least funded to do so. This is why NPM, Crates.io, etc... flatly refuse to do even the most basic security checks like adding namespaces and verifying the identity of major publishers like Google, Microsoft, and the like. That's a non-zero amount of effort, and no matter how trivial to implement technically or how cheap to police, it would likely blow their tiny budget of unreliable donations. The exceptions to this rule are package managers with robust financial backing, such as NuGet, which gets reliable funding from Microsoft and supports their internal (for-profit!) workflows almost as much as it does external "free" users. "Free and open" is wonderful and all, but you get what you pay for. [1] Most of us can name them off the top of our heads: Guido van Rossum, Larry Wall, Kerningham & Richie, etc.
- kibwen 5mo ago
  You appear to have missed that NPM is owned by Microsoft. In addition, crates.io has not flatly refused to support namespaces, there's an entire accepted RFC for it: https://github.com/rust-lang/rfcs/pull/3243 https://github.com/rust-lang/rfcs/pull/3243 At the same time, note that namespacing does nothing to prevent any sort of problem here. Namespacing is great for package organization and making provenance more deliberately obvious, but beyond that it's not a security measure.
brunoborges 5mo ago
It is 100% up to the package manager's steward to control how ownership of packages and namespaces are granted. Maven Central exists for decades the amount of incidents of people stealing namespaces is minimal. One can't simply publish a package under the groupId "com.ycombinator" without having some way to verify that they own the domain ycombinator.com. Then, once a package is published, it is 100% immutable, even if it has malicious code in it. Certainly, that library is flagged everywhere as vulnerable. It baffles me that NPM for so long couldn't replicate the same guardrails as Maven Central.
- cluckindan 5mo ago
  How does that protect against credential theft? MFA required to sign published releases?
  - brunoborges 5mo ago
    That is another important layer. Maven Central is not immune to credential theft. If a publisher token is stolen, an attacker may still be able to publish a malicious new version until the token is revoked or the account is suspended after reporting the problem to Sonatype. But in the Maven/Gradle ecosystem, most projects pin exact dependency versions. Support for version ranges and dynamic versions exist, but they are generally avoided because they hurt reproducible builds. That means a malicious new release does not automatically flow into most consumers’ builds just because it was published. I'd go as far to say that NPM should: 1. Enforce scope (namespace) requirement, and require external verification (reverse DNS for example). 2. Disable version range support out of the box. User must --enable this setting from the command line at all times. 3. Remove support for install scripts completely. If someone wants to publish a ready-to-run software, there are plenty of other mechanisms.
    - com2kid 5mo ago
      > Enforce scope (namespace) requirement, and require external verification (reverse DNS for example). Who the heck says everyone who publishes a library has a domain? That seems absurd.
raggi 5mo ago
none. they just have smaller target populations.
nirvdrum 5mo ago
Part of the point the article makes is that most other popular languages have a comprehensive standard library. JS has an astonishingly small on. Rather than have one vetted set of libraries that ship with the language, applications either need to roll it themselves or pull from a 3rd party package repository. We've drilled NIH into people, so they tend to reach for packages. That's not necessarily a bad thing, but it often means they're pulling in more code than they need. The JS ecosystem has also favored smaller modules, so you need many of them. And everyone builds on top of that, leading to massive growth in dependency graphs. It's a huge surface area for things to go wrong, intentionally or not. With many other languages, you have a lot of functionality out of the box. Certainly, there have been bugs and security issues, but they're a drop in the bucket compared to what you see in the JS ecosystem. With other languages, you have a much smaller external dependency graph and the core functionality is coming from a trusted 3rd party.
- apothegm 5mo ago
  Why Python, tho, in that case? Its stdlib is quite robust. Surprisingly so in some areas.
  - saghm 5mo ago
    I'm not convinced that Python should be the standard for package management either. Earlier this week I was trying to publish a Python package for the first time wrapping a Rust library I wrote (for use only on Linux and Python 3.12+), and it literally took me hours to get from "I have a wheel that I can import and it works on my system" to "I have published that wheel and can install the package from PyPI on the set of systems that I'm trying to support and it actually works". Everything I've heard about this indicates that the situation for Python packaging is literally better than it ever has been before with the current tooling, so I can't even imagine how bad it was for the decades before. In comparison, having literally never touched npm before, I was able to publish a wrapper around the same library and validate that it was working in maybe 10 minutes (most of which were spent from not realizing that a certain tool was failing with a vague "file not found" error because I hadn't installed npm yet). I'm not saying that npm is doing everything right, but I suspect that beyond the obvious low-hanging fruit that we hear about pretty consistently with npm there's probably a long tail of less obvious stuff that can be exploited that will not be specific to npm. The fundamental problems with supply-chain vulnerabilities aren't going to go away if npm magically became pip or go modules overnight.
0xbadcafebee 5mo ago
"What are the actual guarantees that <guy leaving his keys on his dashboard> make that <guy leaving his keys on an illuminated blinking sign outside his house> don't make?"

What are the actual guarantees that go/Rust make that Python/npm don’t? It seems like it might just be that Python/npm are juicier targets? I’m starting to try and avoid all third party packages

cookiengineer 5mo ago

I suppose that go's go:generate workflow can also be abused to land a worm like the ones spreading via npm, as you can build programs that just scrape the whole hard drive for git projects and patch the go.mod dependencies there, and you could also just write this in go as a toolchain script, for example. NPM's achilles is the pre/postinstall step which can run arbitrary commands and shell scripts without the user having any way to intervene. Dependencies must be run in isolated chroot sandboxes or better, inside containers. That would be the only way to mitigate this problem, as the filesystem of the operating system must be separated from the filesystem of the development workflow. On top of that most host based firewalls are per-binary instead of per-cmdline. That leads to the warnings and rules relying on that e.g. "python" or "nodejs" getting network access allowlisted, instead of say "nodejs myworm.js". So firewalls in general are pretty useless against this type of malware.

yegle 5mo ago
`go:generate` is for the package provider, the command never runs when someone `go install` or `go get` the package.
- cookiengineer 5mo ago
  Note that the NPM worms are spreading because the package providers are developing on their libraries without them noticing a malicious dependency. It is not users/consumers spreading the worm, it is developers spreading it. Your mismatch is that you think in policies, not assessments here. Nothing in my normal go workflow will ask me if I want to run "curl download whatever from the internet" when I run go build. Though I agree with the difference in workflow, there is not a single mechanism in go catching this. go.mod files can be just patched by the worm, and/or hidden behind a /v123 folder or whatever to play shenanigans on API differences.
xena 5mo ago
go:generate is done at dev time, not at build time.
- cookiengineer 5mo ago
  Actually bindings are usually generated like that, at build time (though with a build cache that nobody knows how it corrupts all the time). Examples that come to mind: webview/webview, webkit, cilium/ebpf and most other CGo projects that I have seen.
  - arccy 5mo ago
    you only run them for your own project, not the generate directives of your dependencies though

jollyllama 5mo ago

> It seems like it might just be that Python/npm are juicier targets? Attackers go where the victims are. Frontend is a monoculture with the vast majority using NPM; backend, less so. This isn't an excuse for NPM, but another strike against it. You could also argue that the attacks make a deeper point about frontend vs backend devs, but I won't go there.

bichiliad 5mo ago
Why would you even imply something like that?
- voidfunc 5mo ago
  [flagged]
  - rembicilious 5mo ago
    Is this a dogfooding joke?
- llbbdd 5mo ago
  They feel the need to compete given that jokes about "backend" devs write themselves
  - lucketone 5mo ago
    Frontend has lower barrier of entry and more appeal for beginners, so its bell curve might have its left edge is thicker. It impacts the avg of problems and culture of dealing with them
    - llbbdd 5mo ago
      More appeal I agree because it's easier to see useful results and iterate quicker. Lower barrier of entry I disagree with strongly; if the barrier to entry were so low I don't know why I've worked with so many otherwise-talented backend devs that can't wrap their heads around the frontend to save their lives. Frontend forces you to deal with real-world customer problems sooner rather than later; performance is more important, it has to work on more than one environment, you have a frame budget. It's like saying game development has a low barrier to entry; you might be able to get started quickly but you will run into constraints unless you learn fast. On the backend you can just pay another dollar for a VPS twice the size.

panzi 5mo ago

Last I checked npm had 2FA for publishing, but cargo didn't. I don't think cargo is any better than npm, just not that of an attractive target.

lostglass 5mo ago

To be honest Rust has the exact same supply chain attack pattern - it's just newer and more maintained at the moment. Give it a decade.

nothinkjustai 5mo ago
Rust doesn’t have post install scripts
- tasn 5mo ago
  It has build.rs, which has essentially the same problems.
- deeebug 5mo ago
  They have build.rs (https://doc.rust-lang.org/cargo/reference/build-scripts.html https://doc.rust-lang.org/cargo/reference/build-scripts.html)
- fabrice_d 5mo ago
  It has build.rs that will run as soon as you compile the dependency. That's not the same thing but pretty close to a post install script: it's very likely to run.
- est31 5mo ago
  There is build.rs, proc macros are unsandboxed, and lastly you install the binary so that you can run it. Even if the build and install were fully sandboxed, the binary could still do malicious stuff if ran.
  - drdaeman 5mo ago
    Even without post-install script, a malicious payload could be hiding in some function and just wait until the developer invokes `cargo run`. Not that many people audit the crates they pull into their projects.
  - nothinkjustai 5mo ago

jiggawatts 5mo ago

Generally, other package managers aren't great either. Notably, crates.io / cargo has some of the same issues as NPM and the verbiage of their excuses for not fixing these problems is oddly similar. Something fascinating about the design and architecture of programming languages and their surrounding ecosystems is the enormous leverage that they provide to the "core team": For every 1 core language developer[1]... ... there may be 1,000 popular package developers... ... for which there may be 1,000,000 developers writing software... ... for over 1,000,000,000 users. This means that for every corner that is cut at the top of that pyramid, the harms are massively magnified at the lower tiers. A security vulnerability in a "top one thousand" package like log4j can cause billions of dollars in economic damage, man-centuries of remediation effort, etc. However, bizarrely, the funding at the top two levels is essentially a pittance! Most such projects are charities, begging for spare change with hat in hand on a street corner. Some of the most used libraries are often volunteer efforts, despite powering global e-commerce! cough-OpenSSL-cough. The result is that the people most empowered to fix the issues are the least funded to do so. This is why NPM, Crates.io, etc... flatly refuse to do even the most basic security checks like adding namespaces and verifying the identity of major publishers like Google, Microsoft, and the like. That's a non-zero amount of effort, and no matter how trivial to implement technically or how cheap to police, it would likely blow their tiny budget of unreliable donations. The exceptions to this rule are package managers with robust financial backing, such as NuGet, which gets reliable funding from Microsoft and supports their internal (for-profit!) workflows almost as much as it does external "free" users. "Free and open" is wonderful and all, but you get what you pay for. [1] Most of us can name them off the top of our heads: Guido van Rossum, Larry Wall, Kerningham & Richie, etc.

kibwen 5mo ago
You appear to have missed that NPM is owned by Microsoft. In addition, crates.io has not flatly refused to support namespaces, there's an entire accepted RFC for it: https://github.com/rust-lang/rfcs/pull/3243 https://github.com/rust-lang/rfcs/pull/3243 At the same time, note that namespacing does nothing to prevent any sort of problem here. Namespacing is great for package organization and making provenance more deliberately obvious, but beyond that it's not a security measure.

brunoborges 5mo ago

It is 100% up to the package manager's steward to control how ownership of packages and namespaces are granted. Maven Central exists for decades the amount of incidents of people stealing namespaces is minimal. One can't simply publish a package under the groupId "com.ycombinator" without having some way to verify that they own the domain ycombinator.com. Then, once a package is published, it is 100% immutable, even if it has malicious code in it. Certainly, that library is flagged everywhere as vulnerable. It baffles me that NPM for so long couldn't replicate the same guardrails as Maven Central.

cluckindan 5mo ago
How does that protect against credential theft? MFA required to sign published releases?
- brunoborges 5mo ago
  That is another important layer. Maven Central is not immune to credential theft. If a publisher token is stolen, an attacker may still be able to publish a malicious new version until the token is revoked or the account is suspended after reporting the problem to Sonatype. But in the Maven/Gradle ecosystem, most projects pin exact dependency versions. Support for version ranges and dynamic versions exist, but they are generally avoided because they hurt reproducible builds. That means a malicious new release does not automatically flow into most consumers’ builds just because it was published. I'd go as far to say that NPM should: 1. Enforce scope (namespace) requirement, and require external verification (reverse DNS for example). 2. Disable version range support out of the box. User must --enable this setting from the command line at all times. 3. Remove support for install scripts completely. If someone wants to publish a ready-to-run software, there are plenty of other mechanisms.
  - com2kid 5mo ago
    > Enforce scope (namespace) requirement, and require external verification (reverse DNS for example). Who the heck says everyone who publishes a library has a domain? That seems absurd.

raggi 5mo ago

none. they just have smaller target populations.

nirvdrum 5mo ago

Part of the point the article makes is that most other popular languages have a comprehensive standard library. JS has an astonishingly small on. Rather than have one vetted set of libraries that ship with the language, applications either need to roll it themselves or pull from a 3rd party package repository. We've drilled NIH into people, so they tend to reach for packages. That's not necessarily a bad thing, but it often means they're pulling in more code than they need. The JS ecosystem has also favored smaller modules, so you need many of them. And everyone builds on top of that, leading to massive growth in dependency graphs. It's a huge surface area for things to go wrong, intentionally or not. With many other languages, you have a lot of functionality out of the box. Certainly, there have been bugs and security issues, but they're a drop in the bucket compared to what you see in the JS ecosystem. With other languages, you have a much smaller external dependency graph and the core functionality is coming from a trusted 3rd party.

apothegm 5mo ago
Why Python, tho, in that case? Its stdlib is quite robust. Surprisingly so in some areas.
- saghm 5mo ago
  I'm not convinced that Python should be the standard for package management either. Earlier this week I was trying to publish a Python package for the first time wrapping a Rust library I wrote (for use only on Linux and Python 3.12+), and it literally took me hours to get from "I have a wheel that I can import and it works on my system" to "I have published that wheel and can install the package from PyPI on the set of systems that I'm trying to support and it actually works". Everything I've heard about this indicates that the situation for Python packaging is literally better than it ever has been before with the current tooling, so I can't even imagine how bad it was for the decades before. In comparison, having literally never touched npm before, I was able to publish a wrapper around the same library and validate that it was working in maybe 10 minutes (most of which were spent from not realizing that a certain tool was failing with a vague "file not found" error because I hadn't installed npm yet). I'm not saying that npm is doing everything right, but I suspect that beyond the obvious low-hanging fruit that we hear about pretty consistently with npm there's probably a long tail of less obvious stuff that can be exploited that will not be specific to npm. The fundamental problems with supply-chain vulnerabilities aren't going to go away if npm magically became pip or go modules overnight.

0xbadcafebee 5mo ago

"What are the actual guarantees that <guy leaving his keys on his dashboard> make that <guy leaving his keys on an illuminated blinking sign outside his house> don't make?"