6 ms·

I never understood why these package repositories don't include some (opt-in?) integrity checking option using digital signatures. If I download code that execu

I never understood why these package repositories don't include some (opt-in?) integrity checking option using digital signatures. If I download code that executes on my machine there should be at least the option to establish some level of trust. We have been doing that with linux distro package managers for decades. Seems like common sense to me.

BillinghamJ 6y ago
They largely do in various forms. Both npm and yarn, by default, record hashes of the dependencies you're using and check them when redownloading. I think the issue tends to be more that there's just so many packages (often nested 10+ deep) and it's best practice to keep them as up to date as possible. When it's fairly typical for a JS project to have thousands of dependencies, there isn't really any practical way to both stay up to date and carefully review everything you pull in. I think the only viable solution for companies taking this issue really seriously is to keep their numbers of dependencies down and avoid having significant deep/indirect dependencies. Edit: as an example, in my company's Node stack (for 10 services) - there's >900 dependencies. In our React stack (for 2 sites), more than 1600. Contrary to what you might think, these are actually pretty small, lightweight systems. So really whatever you might have thought was the worst-case scenario on numbers of deps, the reality is more like 10x that in the modern JS ecosystem. In many ways, the vast number of tiny dependencies are one of the strongest points of the JS ecosystem. But it doesn't come without caveats.
magicalhippo 6y ago
I was thinking the same thing. Surely PayPal's packages should be signed by a certificate only PayPal has, and they would want to verify that before using their packages?
- angry_octet 6y ago
  If the signature reqt is attached to the package metadata, the new package just removed it. If it's part of their custom build system, what signs third party packages? Would it just sign the new one anyway, because how does it know which ones should have PayPal internal signing? Or are you proposing manual controls? shudder Channels and priorities embedded in the package tools are a better approach, combined with something like Artifactory. Some channels might require packages are signed, and possibly monotonically versioned.
  - magicalhippo 6y ago
    > Or are you proposing manual controls? That's what we do. Dependencies are committed to our repository, so changes show up as diffs which are looked through then. Then again we don't use libraries for padding strings and such...
    - angry_octet 6y ago
      I've found that it is easy for discipline to slide on manual controls. It starts off rigourous and tails off into being done infrequently (which makes it a big job) and perfunctorily. This will save you from things that hit the bleeding edge, the idiots who pull from latest on a prod instance, but leaves you exposed to the patched bugs, with increasingly good exploits. Diff inspection will catch some obviously bad things, but it will rarely catch anything clever. So it would be down to luck, if you had merged in this patch before it was spotted/announced. Unless you have something to separate the namespaces? Check for conflicts? I guess CI might work, hoping your CI machines are sandboxed.
sp332 6y ago
The package integrity would be fine in this case. The packages downloaded from PyPI would be legitimately signed by PyPI, and the internal packages would be signed by the local package server. The issue is not knowing which source to use for each package, and you'd have the same issue with not knowing which certificate to use to check them.

BillinghamJ 6y ago

They largely do in various forms. Both npm and yarn, by default, record hashes of the dependencies you're using and check them when redownloading. I think the issue tends to be more that there's just so many packages (often nested 10+ deep) and it's best practice to keep them as up to date as possible. When it's fairly typical for a JS project to have thousands of dependencies, there isn't really any practical way to both stay up to date and carefully review everything you pull in. I think the only viable solution for companies taking this issue really seriously is to keep their numbers of dependencies down and avoid having significant deep/indirect dependencies. Edit: as an example, in my company's Node stack (for 10 services) - there's >900 dependencies. In our React stack (for 2 sites), more than 1600. Contrary to what you might think, these are actually pretty small, lightweight systems. So really whatever you might have thought was the worst-case scenario on numbers of deps, the reality is more like 10x that in the modern JS ecosystem. In many ways, the vast number of tiny dependencies are one of the strongest points of the JS ecosystem. But it doesn't come without caveats.

magicalhippo 6y ago

I was thinking the same thing. Surely PayPal's packages should be signed by a certificate only PayPal has, and they would want to verify that before using their packages?

angry_octet 6y ago
If the signature reqt is attached to the package metadata, the new package just removed it. If it's part of their custom build system, what signs third party packages? Would it just sign the new one anyway, because how does it know which ones should have PayPal internal signing? Or are you proposing manual controls? shudder Channels and priorities embedded in the package tools are a better approach, combined with something like Artifactory. Some channels might require packages are signed, and possibly monotonically versioned.
- magicalhippo 6y ago
  > Or are you proposing manual controls? That's what we do. Dependencies are committed to our repository, so changes show up as diffs which are looked through then. Then again we don't use libraries for padding strings and such...
  - angry_octet 6y ago
    I've found that it is easy for discipline to slide on manual controls. It starts off rigourous and tails off into being done infrequently (which makes it a big job) and perfunctorily. This will save you from things that hit the bleeding edge, the idiots who pull from latest on a prod instance, but leaves you exposed to the patched bugs, with increasingly good exploits. Diff inspection will catch some obviously bad things, but it will rarely catch anything clever. So it would be down to luck, if you had merged in this patch before it was spotted/announced. Unless you have something to separate the namespaces? Check for conflicts? I guess CI might work, hoping your CI machines are sandboxed.

sp332 6y ago

The package integrity would be fine in this case. The packages downloaded from PyPI would be legitimately signed by PyPI, and the internal packages would be signed by the local package server. The issue is not knowing which source to use for each package, and you'd have the same issue with not knowing which certificate to use to check them.