4 ms·

It seems like the fundamental problem is that clients don't actually verify the signatures. It seems to me like the first step is pip and other clients should b

It seems like the fundamental problem is that clients don't actually verify the signatures. It seems to me like the first step is pip and other clients should be updated to have a verify mode, and PyPy should start flagging packages that don't verify. There are a lot of arguments in here that PGP is fundamentally broken, but it's a mature PKI system, and it seems to me like PyPy should stick with it and use it properly. If someone has a different PKI system that they think works better, by all means put together a PEP but "let's just do away with PKI because our implementation is half-assed and bad" doesn't really seem like a good path.

woodruffw 3y ago
It’s PyPI, not PyPy. The latter is a Python implementation, not the package index. Adding verification to pip makes the underlying problem worse: the keys themselves are weak and misconstructed, meaning that “verification” provides a false sense of security. Finally: PGP is not a mature or successful PKI ecosystem, in any sense of either word. The only marginally successful PKI schemes using PGP are those in Linux distributions, and these benefit from constraints (trusted signer sets, baked in key deployment) that PyPI and other ecosystems can not provide.
- lukeschlather 3y ago
  I'd argue that the benefit of package signing is only ever going to be marginal, but that's not a reason not to do it, it's a reason for PyPI to fix their processes. Weak and misconstructed keys should cause the packages to fail to upload to PyPI. Obviously we can't just flip that switch today but that's the direction we should be thinking.
  - woodruffw 3y ago
    > Weak and misconstructed keys should cause the packages to fail to upload to PyPI. This is much easier said than done: keys aren’t uploaded to PyPI at all, only signatures are. This would imply either (1) requiring users to upload their public key ahead of time, or (2) PyPI trawling public key servers in hopes of finding a matching fingerprint. Neither of these is acceptable from an operational overhead standpoint, and the latter is both insecure (birthday attacks on PGP fingerprints are trivial) and requires participation in a half-dead keyserver ecosystem.
chha 3y ago
This is a universal problem in most of package repositories regardless of language. PKI is difficult, and for it to work the users have to do some homework and place trust somewhere.
- XorNot 3y ago
  In a healthier OSS ecosystem, we'd have people counter-signing packages after they've vetted them. If Google approves something through to production, then it's probably okay for you too. This wouldn't be a bad extension to the GNU license really - requiring reciprocal review.
  - chha 3y ago
    Yeah, that could work, at least for larger orgs. Not so sure the majority of users would comply with such sn extension, or even have the knowledge to review for eg. security issues. Having a working PKI solution that could work regardless of ecosystem would be awesome. If nothing else, I can research which key eg. Microsoft uses, and then allow anything signed by that key as an initial threshold.
    - XorNot 3y ago
      Making it a requirement of using something commercially would add a lot of transparency though. The concept of "software bill of materials" has increased interest now, and this would be a part of it: if you're using something then you sign it and publish the signature which then declares an acknowledgment that it was reviewed in some way.
      - chha 3y ago
        Absolutely, but I fear such a solution would lead to a lot of people signing just to be compliant, not beacause they did a thourough job reviewing. If we could connect it to a reputation somehow, it might have something going for it
        3y ago
crabbone 3y ago
You can already (and since a while ago) verify GPG signatures when installing Python packages. Nobody does that. The biggest problem the author didn't really relate to is that most Python packages come with no signature what so ever. So, what are you going to do about those packages if you wanted to verify the rest? -- just verify those that have signatures, and leave those without signatures alone? -- That just sounds ridiculous... By and large, PyPI is the same story we had with HTML: incompetence that breeds more incompetence. The same way how Web browsers encouraged crappy Web developers to generate ever more broken HTML by "helpfully" fixing the broken parts, PyPI by not enforcing the good practices created a community of people who now don't even understand what the problem is. Fighting this is hopeless, because security is by far not the biggest reason why people download and use Python packages. So, even if security sucks, the users, who, ultimately should've been the ones responsible for their security won't do squat. Instead, maybe, at best, there'll be some broken / partial solution from PyPI.
- donaldstufft 3y ago
  gpg signatures are not, and have never been a meaningful "best practice". The only secure package signing schemes that have ever used gpg signatures, uses gpg as a mechanism to access crypto primitives and has built their own secure system on top of it. They're using gpg in spite of its features, not because of them, and would almost certainly be better served by using something else.
  - crabbone 3y ago
    Come on. On what planet are you living? What other signing options are the best practices? Hint: "best" doesn't mean "ideal", it simply means "better than anything else".
    - donaldstufft 3y ago
      Also, minisign, signify, sigstore are all much better than gpg for the problem of signing packages.
    - 3y ago

woodruffw 3y ago

It’s PyPI, not PyPy. The latter is a Python implementation, not the package index. Adding verification to pip makes the underlying problem worse: the keys themselves are weak and misconstructed, meaning that “verification” provides a false sense of security. Finally: PGP is not a mature or successful PKI ecosystem, in any sense of either word. The only marginally successful PKI schemes using PGP are those in Linux distributions, and these benefit from constraints (trusted signer sets, baked in key deployment) that PyPI and other ecosystems can not provide.

lukeschlather 3y ago
I'd argue that the benefit of package signing is only ever going to be marginal, but that's not a reason not to do it, it's a reason for PyPI to fix their processes. Weak and misconstructed keys should cause the packages to fail to upload to PyPI. Obviously we can't just flip that switch today but that's the direction we should be thinking.
- woodruffw 3y ago
  > Weak and misconstructed keys should cause the packages to fail to upload to PyPI. This is much easier said than done: keys aren’t uploaded to PyPI at all, only signatures are. This would imply either (1) requiring users to upload their public key ahead of time, or (2) PyPI trawling public key servers in hopes of finding a matching fingerprint. Neither of these is acceptable from an operational overhead standpoint, and the latter is both insecure (birthday attacks on PGP fingerprints are trivial) and requires participation in a half-dead keyserver ecosystem.

chha 3y ago

This is a universal problem in most of package repositories regardless of language. PKI is difficult, and for it to work the users have to do some homework and place trust somewhere.

XorNot 3y ago
In a healthier OSS ecosystem, we'd have people counter-signing packages after they've vetted them. If Google approves something through to production, then it's probably okay for you too. This wouldn't be a bad extension to the GNU license really - requiring reciprocal review.
- chha 3y ago
  Yeah, that could work, at least for larger orgs. Not so sure the majority of users would comply with such sn extension, or even have the knowledge to review for eg. security issues. Having a working PKI solution that could work regardless of ecosystem would be awesome. If nothing else, I can research which key eg. Microsoft uses, and then allow anything signed by that key as an initial threshold.
  - XorNot 3y ago
    Making it a requirement of using something commercially would add a lot of transparency though. The concept of "software bill of materials" has increased interest now, and this would be a part of it: if you're using something then you sign it and publish the signature which then declares an acknowledgment that it was reviewed in some way.
    - chha 3y ago
      Absolutely, but I fear such a solution would lead to a lot of people signing just to be compliant, not beacause they did a thourough job reviewing. If we could connect it to a reputation somehow, it might have something going for it
      - 3y ago

crabbone 3y ago

You can already (and since a while ago) verify GPG signatures when installing Python packages. Nobody does that. The biggest problem the author didn't really relate to is that most Python packages come with no signature what so ever. So, what are you going to do about those packages if you wanted to verify the rest? -- just verify those that have signatures, and leave those without signatures alone? -- That just sounds ridiculous... By and large, PyPI is the same story we had with HTML: incompetence that breeds more incompetence. The same way how Web browsers encouraged crappy Web developers to generate ever more broken HTML by "helpfully" fixing the broken parts, PyPI by not enforcing the good practices created a community of people who now don't even understand what the problem is. Fighting this is hopeless, because security is by far not the biggest reason why people download and use Python packages. So, even if security sucks, the users, who, ultimately should've been the ones responsible for their security won't do squat. Instead, maybe, at best, there'll be some broken / partial solution from PyPI.

donaldstufft 3y ago
gpg signatures are not, and have never been a meaningful "best practice". The only secure package signing schemes that have ever used gpg signatures, uses gpg as a mechanism to access crypto primitives and has built their own secure system on top of it. They're using gpg in spite of its features, not because of them, and would almost certainly be better served by using something else.
- crabbone 3y ago
  Come on. On what planet are you living? What other signing options are the best practices? Hint: "best" doesn't mean "ideal", it simply means "better than anything else".
  - donaldstufft 3y ago
    Also, minisign, signify, sigstore are all much better than gpg for the problem of signing packages.
  - 3y ago