4 ms·

Okay then, tell me a way to prevent this.

by AndroTux 11mo ago

blueflow 11mo ago
The same way it always has been done - vendor your deps.
- hu3 11mo ago
  that's what I do whenever feasible. Which is often
- sph 11mo ago
  To be fair this does only work in ecosystems where libraries are stable and don't break every 3 months as it often happens on the JS world. You can vendor your left-pad, but good luck doing that with a third-party SDK.
  - CGamesPlay 11mo ago
    ... you vendor the third-party SDK? Nobody worth working with is breaking their SaaS APIs with that cadence.
- joshstrange 11mo ago
  That literally makes no difference at all. You’ll just vendor the malicious versions. No, a lock file with only exact versions is the safe path here. We haven’t seen a compromise to existing versions that I know of, only patch/minor updates with new malicious code. I maintain that the flexibility in npm package versions is the main issue here.
  - blueflow 11mo ago
    You are using the word "vendoring" differently than i do, i mean some kind of private fork of the repository.
    - joshstrange 11mo ago
      You are using the word differently than everyone else I think. I’ve never heard someone using that word to mean maintain private forks. Then again, even private forks don’t protect you much more than package lock files and they are way more overhead IMHO. You still need some out-of-band process to pull upstream updates and aside from a built-in “cool down” (until you merge changes) I see that method as having a huge amount of downside. Yes, you sidestep malicious versions pushed to npm but now you own the build process for all your dependencies and you have to find time to update (and fix builds if they break) all your dependencies. Locking to a specific version and waiting some period of time (cool down) before updating is way easier and jus as safe IMHO.
      - Yokohiii 11mo ago
        Vendoring literally just means grabbing the source code from origin and commit it to your repo after a review. The expectation that every repo has important regular updates for you is pure FOMO. And if I don't do random updates for fun, nothing will every break. [redacted bullshit!]
        joshstrange 11mo ago
        > Version locking wont help you all the time, i.e. if you build fresh envs from scratch. I'm confused on this. I would imagine it would protect/help you as long as releases are immutable which they are for most package managers (like npm). > Vendoring literally just means grabbing the source code from origin and commit it to your repo after a review. Hmm, I don't think it always necessarily means grabbing the source, it can also mean grabbing the built artifacts in my experience. My biggest issue with vendoring dependencies is it allows for editing of said dependencies. Almost everywhere I've worked that vendored dependencies (copied source or built versions in and committed them) felt the siren song of modifying said dependencies which is hell to deal with later.
        Yokohiii 11mo ago
        You are right about version locking, bullshit on my side, not sure what I was thinking. I personally don't have a problem with the general ability to change vendor code. The question is whether you want it in an specific case or not. If you update frequently then certainly not. But that decision should be deliberate team policy.
amiga386 11mo ago
An example: Java Maven artifacts typically name the exact version of their dependencies. They rarely write "1.2.3 or any newer version in the 1.2.x series", as is the de-facto standard in NPM dependencies. Therefore, it's up to each dependency-user to validate newer versions of dependencies before publishing a new version of their own package. Lots of manual attention needed, so a slower pace of releases. This is a good thing! Another example: all Debian packages are published to unstable, but cannot enter testing for at least 2-10 days, and also have to meet a slew of conditions, including that they can be and are built for all supported architectures, and that they don't cause themselves or anything else to become uninstallable. This allows for the most egregious bugs to be spotted before anyone not directly developing Debian starts using it.
- jml78 11mo ago
  You forgot to mention it is also tied to provable namespaces. People keep saying that NPM is just the biggest target... Hate to break it to you but from targeting enterprises, java maven artifacts would be a MASSIVE target. It is just harder to compromise because NPM is such shit.
  - redwall_hp 11mo ago
    Maven Central verifies the domain used for the package namespace, too. You need to create a DNS TXT entry with a key. This adds a bit more overhead to typo squatting, and a paper trail, since a domain registrar can have identity/billing information subpoenaed. Versus changing a config file and running a publish command...
    - panny 11mo ago
      Maven central also requires package signing. You're not stealing my signing key. It's on a yubikey. Game over, you can't publish malware in my name using my key.
- philipwhiuk 11mo ago
  > An example: Java Maven artifacts typically name the exact version of their dependencies. They rarely write "1.2.3 or any newer version in the 1.2.x series" You can definitely do this. To be honest, you just end up with the same thing via dependabot/renovate.
pabs3 11mo ago
Build packages from source without any binaries (all the way down) and socially audit the source before building. https://bootstrappable.org/ https://bootstrappable.org/ https://reproducible-builds.org/ https://reproducible-builds.org/ https://github.com/crev-dev https://github.com/crev-dev
Balinares 11mo ago
Other languages seem to publish dependencies as self-contained packages whose installation does not require running arbitrary shell scripts. This does not prevent said package from shipping with malware built in, but it does prevent arbitrary shell execution on install and therefore automated worm-like propagation.
tmvnty 11mo ago
Other than general security practices, here are few NPM ecosystem specific ones: https://github.com/bodadotsh/npm-security-best-practices https://github.com/bodadotsh/npm-security-best-practices
codedokode 11mo ago
Hire an antivirus company to provide a safe and verified feed of packages. Use ML and automatic scanners to send packages to manual review. While Halting problem prevents us from 100% reliably detecting malware, at least we can block everything suspicious.
seethishat 11mo ago
I think some system would need to dynamically analyze the code (as it runs) and record what it does. Even then, that may not catch all malicious activity. It's sort of hard to define what malicious activity is. Any file read or network conn could, in theory, be malicious. As a SW developer, you may be able to limit the damage from these attacks by using a MAC (like SELinux or Tomoyo) to ensure that your node app cannot read secrets that it is not intended to read, conns that it should not make, etc. and log attempts to do those things. You could also reduce your use of external packages. Until slowly, over time you have very little external dependencies.
venturecruelty 11mo ago
You have separate people called "maintainers", and they're the ones who build and upload packages to the repository. Crucially, they're not the people who write the software. You know, like Linux has been doing since forever. https://wiki.debian.org/DebianMaintainer https://wiki.debian.org/DebianMaintainer Instead of treating your package repository like a trash can at a music festival, you can treat it more like a museum, curated by experts. Unfortunately, this isn't quite the devil-may-care attitude the Node ecosystem is so accustomed to, and will be met with a lot of whining, so it never happens. See y'all in two weeks when this happens again.

Okay then, tell me a way to prevent this.

blueflow 11mo ago

The same way it always has been done - vendor your deps.

hu3 11mo ago
that's what I do whenever feasible. Which is often
sph 11mo ago
To be fair this does only work in ecosystems where libraries are stable and don't break every 3 months as it often happens on the JS world. You can vendor your left-pad, but good luck doing that with a third-party SDK.
- CGamesPlay 11mo ago
  ... you vendor the third-party SDK? Nobody worth working with is breaking their SaaS APIs with that cadence.
joshstrange 11mo ago
That literally makes no difference at all. You’ll just vendor the malicious versions. No, a lock file with only exact versions is the safe path here. We haven’t seen a compromise to existing versions that I know of, only patch/minor updates with new malicious code. I maintain that the flexibility in npm package versions is the main issue here.
- blueflow 11mo ago
  You are using the word "vendoring" differently than i do, i mean some kind of private fork of the repository.
  - joshstrange 11mo ago
    You are using the word differently than everyone else I think. I’ve never heard someone using that word to mean maintain private forks. Then again, even private forks don’t protect you much more than package lock files and they are way more overhead IMHO. You still need some out-of-band process to pull upstream updates and aside from a built-in “cool down” (until you merge changes) I see that method as having a huge amount of downside. Yes, you sidestep malicious versions pushed to npm but now you own the build process for all your dependencies and you have to find time to update (and fix builds if they break) all your dependencies. Locking to a specific version and waiting some period of time (cool down) before updating is way easier and jus as safe IMHO.
    - Yokohiii 11mo ago
      Vendoring literally just means grabbing the source code from origin and commit it to your repo after a review. The expectation that every repo has important regular updates for you is pure FOMO. And if I don't do random updates for fun, nothing will every break. [redacted bullshit!]
      - joshstrange 11mo ago
        > Version locking wont help you all the time, i.e. if you build fresh envs from scratch. I'm confused on this. I would imagine it would protect/help you as long as releases are immutable which they are for most package managers (like npm). > Vendoring literally just means grabbing the source code from origin and commit it to your repo after a review. Hmm, I don't think it always necessarily means grabbing the source, it can also mean grabbing the built artifacts in my experience. My biggest issue with vendoring dependencies is it allows for editing of said dependencies. Almost everywhere I've worked that vendored dependencies (copied source or built versions in and committed them) felt the siren song of modifying said dependencies which is hell to deal with later.
        Yokohiii 11mo ago
        You are right about version locking, bullshit on my side, not sure what I was thinking. I personally don't have a problem with the general ability to change vendor code. The question is whether you want it in an specific case or not. If you update frequently then certainly not. But that decision should be deliberate team policy.

amiga386 11mo ago

An example: Java Maven artifacts typically name the exact version of their dependencies. They rarely write "1.2.3 or any newer version in the 1.2.x series", as is the de-facto standard in NPM dependencies. Therefore, it's up to each dependency-user to validate newer versions of dependencies before publishing a new version of their own package. Lots of manual attention needed, so a slower pace of releases. This is a good thing! Another example: all Debian packages are published to unstable, but cannot enter testing for at least 2-10 days, and also have to meet a slew of conditions, including that they can be and are built for all supported architectures, and that they don't cause themselves or anything else to become uninstallable. This allows for the most egregious bugs to be spotted before anyone not directly developing Debian starts using it.

jml78 11mo ago
You forgot to mention it is also tied to provable namespaces. People keep saying that NPM is just the biggest target... Hate to break it to you but from targeting enterprises, java maven artifacts would be a MASSIVE target. It is just harder to compromise because NPM is such shit.
- redwall_hp 11mo ago
  Maven Central verifies the domain used for the package namespace, too. You need to create a DNS TXT entry with a key. This adds a bit more overhead to typo squatting, and a paper trail, since a domain registrar can have identity/billing information subpoenaed. Versus changing a config file and running a publish command...
  - panny 11mo ago
    Maven central also requires package signing. You're not stealing my signing key. It's on a yubikey. Game over, you can't publish malware in my name using my key.
philipwhiuk 11mo ago
> An example: Java Maven artifacts typically name the exact version of their dependencies. They rarely write "1.2.3 or any newer version in the 1.2.x series" You can definitely do this. To be honest, you just end up with the same thing via dependabot/renovate.

pabs3 11mo ago

Build packages from source without any binaries (all the way down) and socially audit the source before building. https://bootstrappable.org/ https://bootstrappable.org/ https://reproducible-builds.org/ https://reproducible-builds.org/ https://github.com/crev-dev https://github.com/crev-dev

Balinares 11mo ago

Other languages seem to publish dependencies as self-contained packages whose installation does not require running arbitrary shell scripts. This does not prevent said package from shipping with malware built in, but it does prevent arbitrary shell execution on install and therefore automated worm-like propagation.

tmvnty 11mo ago

Other than general security practices, here are few NPM ecosystem specific ones: https://github.com/bodadotsh/npm-security-best-practices https://github.com/bodadotsh/npm-security-best-practices

codedokode 11mo ago

Hire an antivirus company to provide a safe and verified feed of packages. Use ML and automatic scanners to send packages to manual review. While Halting problem prevents us from 100% reliably detecting malware, at least we can block everything suspicious.

seethishat 11mo ago

I think some system would need to dynamically analyze the code (as it runs) and record what it does. Even then, that may not catch all malicious activity. It's sort of hard to define what malicious activity is. Any file read or network conn could, in theory, be malicious. As a SW developer, you may be able to limit the damage from these attacks by using a MAC (like SELinux or Tomoyo) to ensure that your node app cannot read secrets that it is not intended to read, conns that it should not make, etc. and log attempts to do those things. You could also reduce your use of external packages. Until slowly, over time you have very little external dependencies.

venturecruelty 11mo ago

You have separate people called "maintainers", and they're the ones who build and upload packages to the repository. Crucially, they're not the people who write the software. You know, like Linux has been doing since forever. https://wiki.debian.org/DebianMaintainer https://wiki.debian.org/DebianMaintainer Instead of treating your package repository like a trash can at a music festival, you can treat it more like a museum, curated by experts. Unfortunately, this isn't quite the devil-may-care attitude the Node ecosystem is so accustomed to, and will be met with a lot of whining, so it never happens. See y'all in two weeks when this happens again.