11 ms·

PGP signatures on PyPI: worse than useless

93 points by woodruffw 3y ago

greatgib 3y ago
I don't think that this complaint is completely meritless but I don't buy most of the arguments. - keys not in a public keyserver? Does not mean that the signature is useless. - half of all keys used to sign on PyPI since 2020 are already expired. In theory you can't easily sign with an expired key, so I guess what he means is that he found packages using expired keys, but not that the keys were expired at the time the package was signed/pushed. Since them it is normal and does not mean that it will not verify once you take the signature date into account. I guess that it is heaven more common if you follow the good practice of a dedicated short life subkey of 2 years. In my own personal experience, I already got reports of an user of my package asking if everything was ok because I accidentally signed a release with a wrong key.
- Avamander 3y ago
  > Does not mean that the signature is useless. This has been discussed throughly in this and the previous thread. The gist is that without the public key being discoverable, it's not much more than a hash. Which makes it quite useless. > but not that the keys were expired at the time the package was signed/pushed. We can't know that with PGP. There's no validity stapling showing that a key was valid (or invalid) during the time of signing. All we have is that at the time of verifying the signature it was invalid, thus it's generally useless/invalid. This is further worsened by the fact that revocations (or extensions) are hard to have an up-to-date eye on, so you might be verifying a signature, finding it valid but they key had been already revoked. Yikes.
kibwen 3y ago
Are there any public package repos that do package signing well, or is this still an unsolved problem?
- taeric 3y ago
  I'm tempted to give in to the claim that a problem that is fundamentally a liability concern, and therefore needs legal fallback, is unlikely to have a technical solution. Closest you can get is probably the Apple model?
- udifuutdvbkx 3y ago
  [flagged]
  - woodruffw 3y ago
    Linux distributions that use PGP are effectively closed key ecosystems, meaning that their use of PGP is purely incidental: you trust the key material because they come from trusted sources or baked into your distribution, not because of any of the behavior that PGP supplies. PGP is strictly a liability in these contexts: it's more malleable and allows third-party signers to make mistakes that a reasonable codesigning scheme does not. In other words: each of these distributions could switch to bare signing keys tomorrow, and nothing would change (other than being able to delete a massive pile of insecure GPG code).
  - Avamander 3y ago
    There's little that inspires confidence that what is used is not actually awful, there's plenty that shows how fragile and finicky PGP implementations are, thus how awful they are. Doing things the same way for two decades combined with not taking additional precautions (like transport crypto) puts a lot of weight on a single thing. That deserves all the possible scrutiny it possibly can get.
    - XorNot 3y ago
      Why would transport cryptography be in scope for package signing? It's literally an anti-feature, because any practical environment needs to MITM and inspect the incoming package stream to enforce policy, or provide caches. The abuse of HTTPS to "verify" content on the internet has been one of the biggest missteps.
dale_glass 3y ago
PGP is an unfortunate obsolete technology, which sadly failed to evolve with the times. I think the main thing that crippled PGP is tooling. What we have available is GnuPG, and GnuPG is horrible for every use case but what it was made for: A command-line interactive application, made for a single person's personal usage. GnuPG insists on a model where it manages everything: key parsing, a key database, and the entire signing/verification process. There's no proper libgnupg either, what gpgme does is calling gpg, and presenting a library interface. The problem with this is that GnuPG is very unfriendly towards attempts to build something new from the pieces. It really wants to be a commandline application that deals with a single person's encryption/decryption activities. So if you want to say, run statistics on PGP keys, or write a keyserver, or use it to implement signatures for packages, either you do it the GnuPG way, or you very laboriously trick it into doing what you need, or you write your own crypto code. Option 1 isn't great if you want anything weird like statistics gathering. Option 2 is just awkward and messy. Both 1 and 2 quickly run into limits. GPG for instance doesn't like trying to work with a database of a million keys. It has heavy startup costs. It's made for humans, not for any kind of heavy lifting. Option 3 sucks because OpenPGP is extremely complex, and writing crypto code is very ill advised for most people. And I think that's kinda what crippled the ecosystem. Doing anything but what gnupg wants to do has extreme startup costs, so few people ever try, and most people do it badly. Doing what the author did here involves really going out of your way -- in a better world we'd have simple to use tooling to do this work, and PyPI would just do validation and reject bad stuff, but doing so is extremely non-trivial in PGP. *Edit:* Case in point, I wondered how the author did this analysis. They made a tool for it, in April 2023: https://blog.yossarian.net/2023/04/14/Introducing-pgpkeydump https://blog.yossarian.net/2023/04/14/Introducing-pgpkeydump This is a tool that GPG should have had decades ago, and yet it didn't exist until now, when pretty much everyone has given up on PGP signing. Edit 2: A nice blog post on how painful it is to deal with gpg if you want to actually build on top of it: https://www.mailpile.is/blog/2014-10-07_Some_Thoughts_on_GnuPG.html https://www.mailpile.is/blog/2014-10-07_Some_Thoughts_on_Gnu...
- WesolyKubeczek 3y ago
  Isn't there that new thing called Sequoia which is supposed to have all advantages of PGP while having none of its drawbacks?
exabrial 3y ago
> signatures that can be correlated, many are generated from weak keys or malformed certificates. The results suggest widespread misuse of GPG and other PGP implementations by Python packagers So a bunch of python packages don't implement the spec correctly... and this is pgp's fault.
- woodruffw 3y ago
  Yes. As the post says: well-designed signing schemes need to be misuse resistant. (And to be clear: it's not the "spec" they're failing to implement. The breakages here indicate that people aren't generating strong keys or doing signing operations correctly with the `gpg` CLI; poor design and documentation there is absolutely the tool's fault.)
  - tedivm 3y ago
    This has been a problem with so many cryptographic systems over the years. For various reasons they prefer to be backwards compatible over being secure. As the best practices shift, the default values of these applications stay the same. I understand why, to a certain extent. If your defaults change and people are expecting the old values it could break existing scripts and tool chains. For most applications this argument makes sense, but for security focused tools I think this is a mistake. Defaults should always represent best practices, and people who want specific flags should specify them.
  - waynesonfire 3y ago
    Instead of throwing the baby out with the bath water, try posting on some python discord channel and educate ur users.
    - woodruffw 3y ago
      As the post says: PGP has been discouraged on PyPI for years, because using it correctly is difficult. The goal here is not to try and carve out the "good" bits, but to drive home that even the remaining users (who were expected to be experts, given that it's no longer advertised) are continuing to use it insufficiently for its stated goals.
hklrc 3y ago
Snowden used gpg and taught it to non-programmers: https://theintercept.com/2014/10/28/smuggling-snowden-secrets/ https://theintercept.com/2014/10/28/smuggling-snowden-secret... I'm surprised by the number of people who say that gpg is insecure. Is it disliked because it is decentralized and everything has to be owned and tracked by the four MAGA (formerly FAANG) corporations? A bullet proof HOWTO can fix all of this.
- Avamander 3y ago
  > Is it disliked because it is decentralized and everything has to be owned and tracked by the four MAGA (formerly FAANG) corporations? It's shit to use and instead of improvements we see deflections like this. Sure, there are better implementations being worked on now, but where were they in the past 10, 20 years?
- donaldstufft 3y ago
  > A bullet proof HOWTO can fix all of this You cannot document your way out of a usability nightmare.
  - prepend 3y ago
    The “usability nightmare” isn’t PGP/GPG fault, it’s PKI. PKI is hard to use because of its decentralized and free nature. The “fix” is to channel everything through google, or a government, or some third party that makes it easy and usable, but removes the main benefits of security, transparency, and independence. I think the reason GPG is all we have and has been for 20+ years is because that’s as good as it gets and better than nothing. It lets people who know how to use it communicate securely. So you can document your way out of an impossible to communicate securely nightmare.
    - donaldstufft 3y ago
      There's not a singular thing at fault. Though there is nothing inherently free or decentralized about "PKI", and given your conflation of those concepts I suspect that you're not actually aware of where the lines are drawn from. Certainly the decentralized nature of PGP adds some challenges to good usability. However, a large number of the problems come from the PGP spec itself (some of which is defensible in a 20+ year old spec, but not in anything in a modern system) and from GPG's poor implementation.
Avamander 3y ago
> Of the 1067 keys IDs collected through signatures on PyPI, a full 308 (or roughly 29%) had no publicly discoverable key on the major remaining keyservers Even if they existed on Ubuntu's or OpenPGP's, they'd still be annoying to discover due to the fact that GnuPG only supports a single key server at the same time. You want to have a correspondence with both Protonmail and Mailvelope users? Sucks to be you. Not to mention the fact that if I wanted to maliciously sign something, I'd upload a wholly forged key to those keyservers and you'd happily discover them. It would be a step forward if we'd have more operators like Protonmail that would certify new keys, at least that way in many cases we'd know a key is actually old, rather than generated recently for an attack.
- woodruffw 3y ago
  This is a great observation! There's a whole extended digression (which I cut out of the post) on how even the keyserver's response is insufficient, for exactly this reason. PGP as an ecosystem completely lacks any strong identity guarantees of the type that PyPI wants; without those guarantees, this is all just fragile hashing with more steps.
8organicbits 3y ago
I've been watching sigstore [1] as a better approach to signing artifacts. Anyone have experience using it with python? [1] https://www.sigstore.dev/ https://www.sigstore.dev/
- ewdurbin 3y ago
  Python release files have been signed with sigstore for the last couple versions. You can peruse the release tooling that uses it at https://github.com/python/release-tools https://github.com/python/release-tools
debatem1 3y ago
IME, many people want a very simple "sign this file" interface with a few complicating bells and whistles tacked on: certificate chains (maybe authorities, maybe TOFU/HPKP-style pinning), weak cipher rejection, and crypto hardware support. That's not that hard in the scheme of things but it's definitely not trivial either, especially supporting weird HSMs. Until something well supported, modern, and easy to use can do the above uncontroversially for 5-10 years gpg is going to see lots of misuse.
- growse 3y ago
  Where does signify fall short on these?
progval 3y ago
> That existing PGP signatures on PyPI serve no security purpose, and that all evidence points to nobody ever attempting to verify them; Some Debian package maintainers do check them when downloading packages from PyPI. For example, in the .dsc archive for https://packages.debian.org/bullseye/limnoria https://packages.debian.org/bullseye/limnoria, debian/upstream/signing-key.asc contains the key used to sign releases of https://pypi.org/project/limnoria/ https://pypi.org/project/limnoria/
- donaldstufft 3y ago
  Debian is basically the reason we didn't delete PGP signature support in 2018, because in theory some handful of Debian packages might be better off with this support. In practice I think the handful of packages that this happens on isn't particularly meaningful or valuable, and in my experience what happens when Debian finds a new key signing packages is.. highly variable. I've seen them just disable signatures when a new key shows up (on major packages even), I've seen them just blanket copy whatever the new key is, I've seen them look at release notes for what the new key is. In one or two cases I've seen them actually track down the project and ask for verification of the new key. To me, the PGP support in Debian's uscan feels more like security theater than actual security controls, given my experience with the varied responses to a new release being made by a different key.
  - lukeschlather 3y ago
    Even just as a fancy checksum I think it has some value, and I think it does serve some security goals. I feel like this article also kind of amounts to listing off a bunch of threats that aren't being properly mitigated, and it doesn't really do a good job of separating which threats are not mitigated because of intrinsic deficiencies in PGP from which threats are not mitigated because other parts of the system haven't even made an attempt to mitigate the threat. And then there's the third category, which is "threats which could be mitigated but PGP makes this hard so people tried and gave up." Just giving up entirely doesn't seem like the right reaction here, and that seems to be what the PGP downers are advocating.
codeguro 3y ago
I disagree. As far as technicality goes, PGP is a pretty strong decentralized model. GnuPG is a pretty solid implementation of PGP. Are they perfect? No. Do they merit this much criticism? Obviously not. This whole thing reads like an opinion piece criticizing implementation details and attributing it to the whole design. I did a spot check on the guy's authoritative sources and found them _extremely biased_, with one referring to an article disparaging DSA, who prefixes his technicals with "more important articles" arguing that white people somehow cannot experience racism even if they experienced racial prejudice. And they expect to be taken seriously? It's insanity. But back to the technical aspect - _Of Course_ you should not use 1024-bi RSA or DSA keys. Of course you should choose prime numbers. If someone chooses to use a non-prime field you're going to have problems - that's not a problem with the PGP protocol or with RSA or DSA. If you're trusting public keys from complete strangers, _you're doing it wrong_. If you're not verifying that the fingerprint matches, _you're doing it wrong_. For a journalist trying to report his findings in an unfriendly country to a secured third party, PGP is an excellent choice. For a client to secure communications with his lawyer, PGP is an excellent choice. And for anyone who would argue the contrary, I challenge them to forge a signature for the following identity. Go on, I dare you. -----BEGIN PGP PUBLIC KEY BLOCK----- mQENBGRqTLABCADPc27v3wK4EhBAuoWn5GAVz15r/6osSf41eQ5W83dAmUp4IeIq bHLOvI44nVGiTUrawnpk1oHdOyZxthdo14KL0mplLTAWI0oYzcFClrhNOWa1cB5a K8ZYL0lwlM2YvhGQMaLzJKqEjO+JxLnqKIZeV9iYt9c16igvatVm37DP0NBVFF+V /58g3haNDjvQg4fPOvVIJ8SuIK1vPuTrb1ob2PLFBmHwwmzNGWfWHBBm6izxXZ85 ad56Rp/gca7j02qjnSE1X8S9s0OAJ6AAYRkWhtr/EUw3mjOjGSv32lFrsB7qvSu4 kEl/GY0PhcmnpSwi79vvva82fHZrvfKnbuoHABEBAAG0CUFub255bW91c4kBTgQT AQoAOBYhBNXzRmmKCQC9zIAegYhSrtq97HJWBQJkakywAhsDBQsJCAcCBhUKCQgL AgQWAgMBAh4BAheAAAoJEIhSrtq97HJWMUQIALwWlE1MyiL2vOjG3srY2m6gnE/p iDI9YzQECRr88R0CTSFOvGU8rFwwGNBavtMHfl9BJyJFVk82VK/mWCW7j37CJNu9 ObxqIuvZc35Op9G5LHlEFj/Hal4B3LqyIzm0Kb2IsY7HPMGNmLEQW8gw+PvkNJHr jaEF8eQ/vhboh+rxYn1COiAlBcxpHr3vvCtrsrlqhc1bPRb49ItJ5ybooDaLkl9T prHruCUzsCobYSiT7A85i6wwFvkICCIQZpPCR7cgVqyffjmjLlTPSWpu/+aCp1GP c6532MLqPGhEaFGrTWJzq9ApdQnbbNOt9kkMGCL1GLPGkPNQq0k2R0LNkmA= =A+3o -----END PGP PUBLIC KEY BLOCK----- Only I own this private key, and only I can sign with it. It's stored offline, secured and airgapped. It'd take more energy than there is on the planet to crack it. But I can easily prove I'm the owner. Trapdoor functions are a wonderful thing. If I were given your public key, and it _followed protocol_ and was a strong key, I'd be able secure communications to you in such a way _even I wouldn't be able to decrypt them_. ----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Go on, forge something. I'm waiting. -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEE1fNGaYoJAL3MgB6BiFKu2r3sclYFAmRqTYgACgkQiFKu2r3s clYdUAf+IBIVIxZ1tSbNsKc/Cay0f6weJwUxmr59v7Q/UNzDUcn468Ei0CO9WSVi klMob9uxBrnB9nGYeHcYULUxGZxAwbooQU8XDrE4x4btvARUaRgP0x+ikw6XY/1t N60SeHG9BVxL0EULrlQpNAbbBEQVk59jcKHTHQg8yTcKujw0DiBlJnmQTW5UOaoq UMJRKDBRJAOsrRBDLO2JjWRuFB8dhcVRNv5gqqPg+YyIdcsd/XQyXObyIyHznoAl gjyu9R8hi2Y0vrpNW8mniapDXsm09hOzpaMZqkNSGDRQJVLYWaUMJMs7epwKnL4z x3Virwk8u9NDj4NqjUmzaw2wpvI5iQ== =f62G -----END PGP SIGNATURE-----
aborsy 3y ago
Sequoia PGP is pretty nice, but the issue is not so much the signature/encryption library, or even the interface, but the key management (after all, as far as the end users are concerned, PGP signing and encryption are simple commands). If this was easy, there would have been many solutions by now. Integration with a hardware-based KMS service will go a long way in improving the security. I don’t think the algorithm or file format is the main issue here.
zimmerfrei 3y ago
The real question: how can pip still not allow me to verify a downloaded package with a public key of my choice (PGP key or others, the format is irrelevant)? In most cases, I don't even want to know who the author is, but just be warned if the signing key changed since last time (possibly because pypi has been compromised, which is the elephant in the python ecosystem room). This minor feature has been blocked for years on ideological basis (and matched to questionable decisions like hiding on purpose the signatures from the pypi UI), on the grounds that keys are not discoverable and not attached to a strong identity (which happens to be an argument close to what this article say). Just add that possibility, make it opt in, and allow the ecosystem to figure how out solve the key management/trust/identity problem.
- lukeschlather 3y ago
  IIRC Google solved this recently in the Play Store by signing packages themselves. But the cost was that it's much harder (impossible?) for me as a package maintainer to be sure that the package I built is the one I'm distributing; it means now Google can alter the package and nobody will be any the wiser. I think a lot of the trouble with this article (and the article it links about why PGP is broken) is that it doesn't specify a threat model. It makes some good arguments why specific threats are not mitigated, but rather than attempting to fix broken processes or replace specific components it suggests just throwing out the whole idea. And some of the arguments seem to point toward using a Google-style central packager/verifier but there are good reasons PyPy doesn't do this and many wouldn't want them to.
  - yjftsjthsd-h 3y ago
    Surely the correct solution is having both parties sign the same artifact?
lukeschlather 3y ago
It seems like the fundamental problem is that clients don't actually verify the signatures. It seems to me like the first step is pip and other clients should be updated to have a verify mode, and PyPy should start flagging packages that don't verify. There are a lot of arguments in here that PGP is fundamentally broken, but it's a mature PKI system, and it seems to me like PyPy should stick with it and use it properly. If someone has a different PKI system that they think works better, by all means put together a PEP but "let's just do away with PKI because our implementation is half-assed and bad" doesn't really seem like a good path.
- woodruffw 3y ago
  It’s PyPI, not PyPy. The latter is a Python implementation, not the package index. Adding verification to pip makes the underlying problem worse: the keys themselves are weak and misconstructed, meaning that “verification” provides a false sense of security. Finally: PGP is not a mature or successful PKI ecosystem, in any sense of either word. The only marginally successful PKI schemes using PGP are those in Linux distributions, and these benefit from constraints (trusted signer sets, baked in key deployment) that PyPI and other ecosystems can not provide.
  - lukeschlather 3y ago
    I'd argue that the benefit of package signing is only ever going to be marginal, but that's not a reason not to do it, it's a reason for PyPI to fix their processes. Weak and misconstructed keys should cause the packages to fail to upload to PyPI. Obviously we can't just flip that switch today but that's the direction we should be thinking.
    - woodruffw 3y ago
      > Weak and misconstructed keys should cause the packages to fail to upload to PyPI. This is much easier said than done: keys aren’t uploaded to PyPI at all, only signatures are. This would imply either (1) requiring users to upload their public key ahead of time, or (2) PyPI trawling public key servers in hopes of finding a matching fingerprint. Neither of these is acceptable from an operational overhead standpoint, and the latter is both insecure (birthday attacks on PGP fingerprints are trivial) and requires participation in a half-dead keyserver ecosystem.
hackan 3y ago
Indeed. I stopped using gpg for my packages, and started with minisign. Unfortunately, PyPi doesn't support any kind of metadata file, like a signature file, so I can't upload them :( Unless maybe u name it like a pgp file, but them u are hurting UX. Anyway, I simple upload them to my releases anyway, in Gitlab or Github.
remram 3y ago
I think the problem is always the same: there is a difference between verfying a library and verifying its authorship. PGP is made strictly to verify authors not artifacts. If I have people I trust, I want to check whether they trust the library I'm about to download. In PGP like most web-of-trust solutions, what is checked is that they trust that the name of the author is correct. That does nothing for security. This is the same for sigstore, etc. Other solutions like crev do the right thing, allowing people to assign trust into library releases, not just author's identities. In PGP terms, it means I don't want PyPI to have PGP signatures from their author, so I can verify their author via my web-of-trust. I want PyPI to have PGP-signed reviews from people I trust, so I can trust the reviews.

93 points by woodruffw 3y ago

I don't think that this complaint is completely meritless but I don't buy most of the arguments. - keys not in a public keyserver? Does not mean that the signature is useless. - half of all keys used to sign on PyPI since 2020 are already expired. In theory you can't easily sign with an expired key, so I guess what he means is that he found packages using expired keys, but not that the keys were expired at the time the package was signed/pushed. Since them it is normal and does not mean that it will not verify once you take the signature date into account. I guess that it is heaven more common if you follow the good practice of a dedicated short life subkey of 2 years. In my own personal experience, I already got reports of an user of my package asking if everything was ok because I accidentally signed a release with a wrong key.

Avamander 3y ago
> Does not mean that the signature is useless. This has been discussed throughly in this and the previous thread. The gist is that without the public key being discoverable, it's not much more than a hash. Which makes it quite useless. > but not that the keys were expired at the time the package was signed/pushed. We can't know that with PGP. There's no validity stapling showing that a key was valid (or invalid) during the time of signing. All we have is that at the time of verifying the signature it was invalid, thus it's generally useless/invalid. This is further worsened by the fact that revocations (or extensions) are hard to have an up-to-date eye on, so you might be verifying a signature, finding it valid but they key had been already revoked. Yikes.

kibwen 3y ago

Are there any public package repos that do package signing well, or is this still an unsolved problem?

taeric 3y ago
I'm tempted to give in to the claim that a problem that is fundamentally a liability concern, and therefore needs legal fallback, is unlikely to have a technical solution. Closest you can get is probably the Apple model?
udifuutdvbkx 3y ago
[flagged]
- woodruffw 3y ago
  Linux distributions that use PGP are effectively closed key ecosystems, meaning that their use of PGP is purely incidental: you trust the key material because they come from trusted sources or baked into your distribution, not because of any of the behavior that PGP supplies. PGP is strictly a liability in these contexts: it's more malleable and allows third-party signers to make mistakes that a reasonable codesigning scheme does not. In other words: each of these distributions could switch to bare signing keys tomorrow, and nothing would change (other than being able to delete a massive pile of insecure GPG code).
- Avamander 3y ago
  There's little that inspires confidence that what is used is not actually awful, there's plenty that shows how fragile and finicky PGP implementations are, thus how awful they are. Doing things the same way for two decades combined with not taking additional precautions (like transport crypto) puts a lot of weight on a single thing. That deserves all the possible scrutiny it possibly can get.
  - XorNot 3y ago
    Why would transport cryptography be in scope for package signing? It's literally an anti-feature, because any practical environment needs to MITM and inspect the incoming package stream to enforce policy, or provide caches. The abuse of HTTPS to "verify" content on the internet has been one of the biggest missteps.

dale_glass 3y ago

PGP is an unfortunate obsolete technology, which sadly failed to evolve with the times. I think the main thing that crippled PGP is tooling. What we have available is GnuPG, and GnuPG is horrible for every use case but what it was made for: A command-line interactive application, made for a single person's personal usage. GnuPG insists on a model where it manages everything: key parsing, a key database, and the entire signing/verification process. There's no proper libgnupg either, what gpgme does is calling gpg, and presenting a library interface. The problem with this is that GnuPG is very unfriendly towards attempts to build something new from the pieces. It really wants to be a commandline application that deals with a single person's encryption/decryption activities. So if you want to say, run statistics on PGP keys, or write a keyserver, or use it to implement signatures for packages, either you do it the GnuPG way, or you very laboriously trick it into doing what you need, or you write your own crypto code. Option 1 isn't great if you want anything weird like statistics gathering. Option 2 is just awkward and messy. Both 1 and 2 quickly run into limits. GPG for instance doesn't like trying to work with a database of a million keys. It has heavy startup costs. It's made for humans, not for any kind of heavy lifting. Option 3 sucks because OpenPGP is extremely complex, and writing crypto code is very ill advised for most people. And I think that's kinda what crippled the ecosystem. Doing anything but what gnupg wants to do has extreme startup costs, so few people ever try, and most people do it badly. Doing what the author did here involves really going out of your way -- in a better world we'd have simple to use tooling to do this work, and PyPI would just do validation and reject bad stuff, but doing so is extremely non-trivial in PGP. *Edit:* Case in point, I wondered how the author did this analysis. They made a tool for it, in April 2023: https://blog.yossarian.net/2023/04/14/Introducing-pgpkeydump https://blog.yossarian.net/2023/04/14/Introducing-pgpkeydump This is a tool that GPG should have had decades ago, and yet it didn't exist until now, when pretty much everyone has given up on PGP signing. Edit 2: A nice blog post on how painful it is to deal with gpg if you want to actually build on top of it: https://www.mailpile.is/blog/2014-10-07_Some_Thoughts_on_GnuPG.html https://www.mailpile.is/blog/2014-10-07_Some_Thoughts_on_Gnu...

WesolyKubeczek 3y ago
Isn't there that new thing called Sequoia which is supposed to have all advantages of PGP while having none of its drawbacks?

exabrial 3y ago

> signatures that can be correlated, many are generated from weak keys or malformed certificates. The results suggest widespread misuse of GPG and other PGP implementations by Python packagers So a bunch of python packages don't implement the spec correctly... and this is pgp's fault.

woodruffw 3y ago
Yes. As the post says: well-designed signing schemes need to be misuse resistant. (And to be clear: it's not the "spec" they're failing to implement. The breakages here indicate that people aren't generating strong keys or doing signing operations correctly with the `gpg` CLI; poor design and documentation there is absolutely the tool's fault.)
- tedivm 3y ago
  This has been a problem with so many cryptographic systems over the years. For various reasons they prefer to be backwards compatible over being secure. As the best practices shift, the default values of these applications stay the same. I understand why, to a certain extent. If your defaults change and people are expecting the old values it could break existing scripts and tool chains. For most applications this argument makes sense, but for security focused tools I think this is a mistake. Defaults should always represent best practices, and people who want specific flags should specify them.
- waynesonfire 3y ago
  Instead of throwing the baby out with the bath water, try posting on some python discord channel and educate ur users.
  - woodruffw 3y ago
    As the post says: PGP has been discouraged on PyPI for years, because using it correctly is difficult. The goal here is not to try and carve out the "good" bits, but to drive home that even the remaining users (who were expected to be experts, given that it's no longer advertised) are continuing to use it insufficiently for its stated goals.

hklrc 3y ago

Snowden used gpg and taught it to non-programmers: https://theintercept.com/2014/10/28/smuggling-snowden-secrets/ https://theintercept.com/2014/10/28/smuggling-snowden-secret... I'm surprised by the number of people who say that gpg is insecure. Is it disliked because it is decentralized and everything has to be owned and tracked by the four MAGA (formerly FAANG) corporations? A bullet proof HOWTO can fix all of this.

Avamander 3y ago
> Is it disliked because it is decentralized and everything has to be owned and tracked by the four MAGA (formerly FAANG) corporations? It's shit to use and instead of improvements we see deflections like this. Sure, there are better implementations being worked on now, but where were they in the past 10, 20 years?
donaldstufft 3y ago
> A bullet proof HOWTO can fix all of this You cannot document your way out of a usability nightmare.
- prepend 3y ago
  The “usability nightmare” isn’t PGP/GPG fault, it’s PKI. PKI is hard to use because of its decentralized and free nature. The “fix” is to channel everything through google, or a government, or some third party that makes it easy and usable, but removes the main benefits of security, transparency, and independence. I think the reason GPG is all we have and has been for 20+ years is because that’s as good as it gets and better than nothing. It lets people who know how to use it communicate securely. So you can document your way out of an impossible to communicate securely nightmare.
  - donaldstufft 3y ago
    There's not a singular thing at fault. Though there is nothing inherently free or decentralized about "PKI", and given your conflation of those concepts I suspect that you're not actually aware of where the lines are drawn from. Certainly the decentralized nature of PGP adds some challenges to good usability. However, a large number of the problems come from the PGP spec itself (some of which is defensible in a 20+ year old spec, but not in anything in a modern system) and from GPG's poor implementation.

Avamander 3y ago

> Of the 1067 keys IDs collected through signatures on PyPI, a full 308 (or roughly 29%) had no publicly discoverable key on the major remaining keyservers Even if they existed on Ubuntu's or OpenPGP's, they'd still be annoying to discover due to the fact that GnuPG only supports a single key server at the same time. You want to have a correspondence with both Protonmail and Mailvelope users? Sucks to be you. Not to mention the fact that if I wanted to maliciously sign something, I'd upload a wholly forged key to those keyservers and you'd happily discover them. It would be a step forward if we'd have more operators like Protonmail that would certify new keys, at least that way in many cases we'd know a key is actually old, rather than generated recently for an attack.

woodruffw 3y ago
This is a great observation! There's a whole extended digression (which I cut out of the post) on how even the keyserver's response is insufficient, for exactly this reason. PGP as an ecosystem completely lacks any strong identity guarantees of the type that PyPI wants; without those guarantees, this is all just fragile hashing with more steps.

8organicbits 3y ago

I've been watching sigstore [1] as a better approach to signing artifacts. Anyone have experience using it with python? [1] https://www.sigstore.dev/ https://www.sigstore.dev/

ewdurbin 3y ago
Python release files have been signed with sigstore for the last couple versions. You can peruse the release tooling that uses it at https://github.com/python/release-tools https://github.com/python/release-tools

debatem1 3y ago

IME, many people want a very simple "sign this file" interface with a few complicating bells and whistles tacked on: certificate chains (maybe authorities, maybe TOFU/HPKP-style pinning), weak cipher rejection, and crypto hardware support. That's not that hard in the scheme of things but it's definitely not trivial either, especially supporting weird HSMs. Until something well supported, modern, and easy to use can do the above uncontroversially for 5-10 years gpg is going to see lots of misuse.

growse 3y ago
Where does signify fall short on these?

progval 3y ago

> That existing PGP signatures on PyPI serve no security purpose, and that all evidence points to nobody ever attempting to verify them; Some Debian package maintainers do check them when downloading packages from PyPI. For example, in the .dsc archive for https://packages.debian.org/bullseye/limnoria https://packages.debian.org/bullseye/limnoria, debian/upstream/signing-key.asc contains the key used to sign releases of https://pypi.org/project/limnoria/ https://pypi.org/project/limnoria/

donaldstufft 3y ago
Debian is basically the reason we didn't delete PGP signature support in 2018, because in theory some handful of Debian packages might be better off with this support. In practice I think the handful of packages that this happens on isn't particularly meaningful or valuable, and in my experience what happens when Debian finds a new key signing packages is.. highly variable. I've seen them just disable signatures when a new key shows up (on major packages even), I've seen them just blanket copy whatever the new key is, I've seen them look at release notes for what the new key is. In one or two cases I've seen them actually track down the project and ask for verification of the new key. To me, the PGP support in Debian's uscan feels more like security theater than actual security controls, given my experience with the varied responses to a new release being made by a different key.
- lukeschlather 3y ago
  Even just as a fancy checksum I think it has some value, and I think it does serve some security goals. I feel like this article also kind of amounts to listing off a bunch of threats that aren't being properly mitigated, and it doesn't really do a good job of separating which threats are not mitigated because of intrinsic deficiencies in PGP from which threats are not mitigated because other parts of the system haven't even made an attempt to mitigate the threat. And then there's the third category, which is "threats which could be mitigated but PGP makes this hard so people tried and gave up." Just giving up entirely doesn't seem like the right reaction here, and that seems to be what the PGP downers are advocating.

codeguro 3y ago

I disagree. As far as technicality goes, PGP is a pretty strong decentralized model. GnuPG is a pretty solid implementation of PGP. Are they perfect? No. Do they merit this much criticism? Obviously not. This whole thing reads like an opinion piece criticizing implementation details and attributing it to the whole design. I did a spot check on the guy's authoritative sources and found them _extremely biased_, with one referring to an article disparaging DSA, who prefixes his technicals with "more important articles" arguing that white people somehow cannot experience racism even if they experienced racial prejudice. And they expect to be taken seriously? It's insanity. But back to the technical aspect - _Of Course_ you should not use 1024-bi RSA or DSA keys. Of course you should choose prime numbers. If someone chooses to use a non-prime field you're going to have problems - that's not a problem with the PGP protocol or with RSA or DSA. If you're trusting public keys from complete strangers, _you're doing it wrong_. If you're not verifying that the fingerprint matches, _you're doing it wrong_. For a journalist trying to report his findings in an unfriendly country to a secured third party, PGP is an excellent choice. For a client to secure communications with his lawyer, PGP is an excellent choice. And for anyone who would argue the contrary, I challenge them to forge a signature for the following identity. Go on, I dare you. -----BEGIN PGP PUBLIC KEY BLOCK----- mQENBGRqTLABCADPc27v3wK4EhBAuoWn5GAVz15r/6osSf41eQ5W83dAmUp4IeIq bHLOvI44nVGiTUrawnpk1oHdOyZxthdo14KL0mplLTAWI0oYzcFClrhNOWa1cB5a K8ZYL0lwlM2YvhGQMaLzJKqEjO+JxLnqKIZeV9iYt9c16igvatVm37DP0NBVFF+V /58g3haNDjvQg4fPOvVIJ8SuIK1vPuTrb1ob2PLFBmHwwmzNGWfWHBBm6izxXZ85 ad56Rp/gca7j02qjnSE1X8S9s0OAJ6AAYRkWhtr/EUw3mjOjGSv32lFrsB7qvSu4 kEl/GY0PhcmnpSwi79vvva82fHZrvfKnbuoHABEBAAG0CUFub255bW91c4kBTgQT AQoAOBYhBNXzRmmKCQC9zIAegYhSrtq97HJWBQJkakywAhsDBQsJCAcCBhUKCQgL AgQWAgMBAh4BAheAAAoJEIhSrtq97HJWMUQIALwWlE1MyiL2vOjG3srY2m6gnE/p iDI9YzQECRr88R0CTSFOvGU8rFwwGNBavtMHfl9BJyJFVk82VK/mWCW7j37CJNu9 ObxqIuvZc35Op9G5LHlEFj/Hal4B3LqyIzm0Kb2IsY7HPMGNmLEQW8gw+PvkNJHr jaEF8eQ/vhboh+rxYn1COiAlBcxpHr3vvCtrsrlqhc1bPRb49ItJ5ybooDaLkl9T prHruCUzsCobYSiT7A85i6wwFvkICCIQZpPCR7cgVqyffjmjLlTPSWpu/+aCp1GP c6532MLqPGhEaFGrTWJzq9ApdQnbbNOt9kkMGCL1GLPGkPNQq0k2R0LNkmA= =A+3o -----END PGP PUBLIC KEY BLOCK----- Only I own this private key, and only I can sign with it. It's stored offline, secured and airgapped. It'd take more energy than there is on the planet to crack it. But I can easily prove I'm the owner. Trapdoor functions are a wonderful thing. If I were given your public key, and it _followed protocol_ and was a strong key, I'd be able secure communications to you in such a way _even I wouldn't be able to decrypt them_. ----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Go on, forge something. I'm waiting. -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEE1fNGaYoJAL3MgB6BiFKu2r3sclYFAmRqTYgACgkQiFKu2r3s clYdUAf+IBIVIxZ1tSbNsKc/Cay0f6weJwUxmr59v7Q/UNzDUcn468Ei0CO9WSVi klMob9uxBrnB9nGYeHcYULUxGZxAwbooQU8XDrE4x4btvARUaRgP0x+ikw6XY/1t N60SeHG9BVxL0EULrlQpNAbbBEQVk59jcKHTHQg8yTcKujw0DiBlJnmQTW5UOaoq UMJRKDBRJAOsrRBDLO2JjWRuFB8dhcVRNv5gqqPg+YyIdcsd/XQyXObyIyHznoAl gjyu9R8hi2Y0vrpNW8mniapDXsm09hOzpaMZqkNSGDRQJVLYWaUMJMs7epwKnL4z x3Virwk8u9NDj4NqjUmzaw2wpvI5iQ== =f62G -----END PGP SIGNATURE-----

aborsy 3y ago

Sequoia PGP is pretty nice, but the issue is not so much the signature/encryption library, or even the interface, but the key management (after all, as far as the end users are concerned, PGP signing and encryption are simple commands). If this was easy, there would have been many solutions by now. Integration with a hardware-based KMS service will go a long way in improving the security. I don’t think the algorithm or file format is the main issue here.

zimmerfrei 3y ago

The real question: how can pip still not allow me to verify a downloaded package with a public key of my choice (PGP key or others, the format is irrelevant)? In most cases, I don't even want to know who the author is, but just be warned if the signing key changed since last time (possibly because pypi has been compromised, which is the elephant in the python ecosystem room). This minor feature has been blocked for years on ideological basis (and matched to questionable decisions like hiding on purpose the signatures from the pypi UI), on the grounds that keys are not discoverable and not attached to a strong identity (which happens to be an argument close to what this article say). Just add that possibility, make it opt in, and allow the ecosystem to figure how out solve the key management/trust/identity problem.

lukeschlather 3y ago
IIRC Google solved this recently in the Play Store by signing packages themselves. But the cost was that it's much harder (impossible?) for me as a package maintainer to be sure that the package I built is the one I'm distributing; it means now Google can alter the package and nobody will be any the wiser. I think a lot of the trouble with this article (and the article it links about why PGP is broken) is that it doesn't specify a threat model. It makes some good arguments why specific threats are not mitigated, but rather than attempting to fix broken processes or replace specific components it suggests just throwing out the whole idea. And some of the arguments seem to point toward using a Google-style central packager/verifier but there are good reasons PyPy doesn't do this and many wouldn't want them to.
- yjftsjthsd-h 3y ago
  Surely the correct solution is having both parties sign the same artifact?

lukeschlather 3y ago

It seems like the fundamental problem is that clients don't actually verify the signatures. It seems to me like the first step is pip and other clients should be updated to have a verify mode, and PyPy should start flagging packages that don't verify. There are a lot of arguments in here that PGP is fundamentally broken, but it's a mature PKI system, and it seems to me like PyPy should stick with it and use it properly. If someone has a different PKI system that they think works better, by all means put together a PEP but "let's just do away with PKI because our implementation is half-assed and bad" doesn't really seem like a good path.

woodruffw 3y ago
It’s PyPI, not PyPy. The latter is a Python implementation, not the package index. Adding verification to pip makes the underlying problem worse: the keys themselves are weak and misconstructed, meaning that “verification” provides a false sense of security. Finally: PGP is not a mature or successful PKI ecosystem, in any sense of either word. The only marginally successful PKI schemes using PGP are those in Linux distributions, and these benefit from constraints (trusted signer sets, baked in key deployment) that PyPI and other ecosystems can not provide.
- lukeschlather 3y ago
  I'd argue that the benefit of package signing is only ever going to be marginal, but that's not a reason not to do it, it's a reason for PyPI to fix their processes. Weak and misconstructed keys should cause the packages to fail to upload to PyPI. Obviously we can't just flip that switch today but that's the direction we should be thinking.
  - woodruffw 3y ago
    > Weak and misconstructed keys should cause the packages to fail to upload to PyPI. This is much easier said than done: keys aren’t uploaded to PyPI at all, only signatures are. This would imply either (1) requiring users to upload their public key ahead of time, or (2) PyPI trawling public key servers in hopes of finding a matching fingerprint. Neither of these is acceptable from an operational overhead standpoint, and the latter is both insecure (birthday attacks on PGP fingerprints are trivial) and requires participation in a half-dead keyserver ecosystem.

hackan 3y ago

Indeed. I stopped using gpg for my packages, and started with minisign. Unfortunately, PyPi doesn't support any kind of metadata file, like a signature file, so I can't upload them :( Unless maybe u name it like a pgp file, but them u are hurting UX. Anyway, I simple upload them to my releases anyway, in Gitlab or Github.

remram 3y ago

I think the problem is always the same: there is a difference between verfying a library and verifying its authorship. PGP is made strictly to verify authors not artifacts. If I have people I trust, I want to check whether they trust the library I'm about to download. In PGP like most web-of-trust solutions, what is checked is that they trust that the name of the author is correct. That does nothing for security. This is the same for sigstore, etc. Other solutions like crev do the right thing, allowing people to assign trust into library releases, not just author's identities. In PGP terms, it means I don't want PyPI to have PGP signatures from their author, so I can verify their author via my web-of-trust. I want PyPI to have PGP-signed reviews from people I trust, so I can trust the reviews.