3 ms·

Wow. Really bad news. Is there a way to automate looking at packages published pre September 2020 and compare the contents to publicly available repositories? I

Wow. Really bad news. Is there a way to automate looking at packages published pre September 2020 and compare the contents to publicly available repositories? It wouldn’t cover all possible malicious packages but it seems like it would be a start.

captn3m0 5y ago
Focusing on the most popular packages could make this feasible.
seniorsassycat 5y ago
Package content can't be compared to a repository. 1. npm doesn't record the repo, branch, or commit so it doesn't know what to compare with. 2. Published content is usually a transformation of the repo content -compiled, minified, bundled. You would have to run the same transformation on the repo source and it would have to be a deterministic build. npm could require that popular packages are published via GitHub actions, then it could strongly associate the published version with a source commit and the build that produced the artifact. This has some downsides like tie in to the GitHub ecosystem. Maybe that could be offset with sponsored builds?
- silverwind 5y ago
  For packages without a build, they could just do a checkout on the repo and verify that the file hashes match the ones about to be published. Certainly not a complete solution but a good first step.
  - dane-pgp 5y ago
    A research paper "Investigating The Reproducibility of NPM Packages"[0] found that "Among the 3,390 versions of the 226 packages, only 2,087 versions are reproducible". (Coincidentally that paper was published in September 2020 too). There was an earlier project called "Trust But Verify"[1] which tried to detect discrepancies between published NPM packages and the corresponding (inferred) tag/commit in the source repository, but sadly it doesn't seem to have gained traction. Interestingly, the "Gold" standard of the Core Infrastructure Initiative (CII) Best Practices guide[2] is to have a reproducible build, but it is only "SUGGESTED that projects identify each release within their version control system", so presumably packages are free to indicate which commit they are built from in an ad hoc out-of-band way, which may not be amenable to automated third-party checking. [0] https://people.cs.vt.edu/nm8247/publications/icsme-camera-ready-2020.pdf https://people.cs.vt.edu/nm8247/publications/icsme-camera-re... [1] https://hackernoon.com/what-if-we-could-verify-npm-packages-c2a319cff758 https://hackernoon.com/what-if-we-could-verify-npm-packages-... [2] https://bestpractices.coreinfrastructure.org/en/criteria https://bestpractices.coreinfrastructure.org/en/criteria

captn3m0 5y ago

Focusing on the most popular packages could make this feasible.

seniorsassycat 5y ago

Package content can't be compared to a repository. 1. npm doesn't record the repo, branch, or commit so it doesn't know what to compare with. 2. Published content is usually a transformation of the repo content -compiled, minified, bundled. You would have to run the same transformation on the repo source and it would have to be a deterministic build. npm could require that popular packages are published via GitHub actions, then it could strongly associate the published version with a source commit and the build that produced the artifact. This has some downsides like tie in to the GitHub ecosystem. Maybe that could be offset with sponsored builds?

silverwind 5y ago
For packages without a build, they could just do a checkout on the repo and verify that the file hashes match the ones about to be published. Certainly not a complete solution but a good first step.
- dane-pgp 5y ago
  A research paper "Investigating The Reproducibility of NPM Packages"[0] found that "Among the 3,390 versions of the 226 packages, only 2,087 versions are reproducible". (Coincidentally that paper was published in September 2020 too). There was an earlier project called "Trust But Verify"[1] which tried to detect discrepancies between published NPM packages and the corresponding (inferred) tag/commit in the source repository, but sadly it doesn't seem to have gained traction. Interestingly, the "Gold" standard of the Core Infrastructure Initiative (CII) Best Practices guide[2] is to have a reproducible build, but it is only "SUGGESTED that projects identify each release within their version control system", so presumably packages are free to indicate which commit they are built from in an ad hoc out-of-band way, which may not be amenable to automated third-party checking. [0] https://people.cs.vt.edu/nm8247/publications/icsme-camera-ready-2020.pdf https://people.cs.vt.edu/nm8247/publications/icsme-camera-re... [1] https://hackernoon.com/what-if-we-could-verify-npm-packages-c2a319cff758 https://hackernoon.com/what-if-we-could-verify-npm-packages-... [2] https://bestpractices.coreinfrastructure.org/en/criteria https://bestpractices.coreinfrastructure.org/en/criteria