5 ms·

Why does this need to allowlist CI providers in first place? Why not publish an open interface any CI provider can integrate against?

SethMLarson 2y ago
Because every CI/ID provider has a different set of claims and behaviors that would constitute a "secure" policy for verification. If there was one singular way to do that then we could, but there isn't yet so PyPI needs to onboard providers piecemeal. The work to add a new provider is not massive, the reason there are not tons of providers isn't because the work is hard but rather because people are voting with their feet so Github and Gitlab make sense as initial providers to support.
woodruffw 2y ago
Because the security benefit of Trusted Publishing via OIDC versus normal API tokens is marginal at small scales, in two senses: 1. The primary benefit of Trusted Publishing over a manual API token is knowing that the underlying OIDC IdP has an on-call staff, proper key management and rotation policies, etc. These can be guaranteed for GitHub, GitLab, etc., but they're harder to prove for one-off self-hosted CI setups. For the latter case, the user is no better off than they would be with a manual API token, which is still (and will always be) supported. 2. If the overwhelming majority of traffic comes from a single CI/CD provider, adding more code to support generic OIDC IdPs increases PyPI's attack surface for only marginal user benefit. There also is no "open interface" for PyPI to really use here: this is all built on OIDC, but each OIDC provider needs to have its unique claims mapped to something intelligible by PyPI. That step requires thoughtful, manual, per-IdP consideration to avoid security issues.
- antononcube 2y ago
  > [...] the user is no better off than they would be with a manual API token, which is still (and will always be) supported. This is good to know. I did not see related statements in of the documents linked to this discussion, though.
  - antononcube 2y ago
    I am not sure why my comment above is downvoted -- if you know where the perpetual optionality of digital attestations is officially stated, please, provide a link.
- freeone3000 2y ago
  I think I would be better off with API key + PGP than API key alone. And that’s being phased out?
  - woodruffw 2y ago
    You can no longer upload a PGP signature to PyPI, if that's what you mean. That was phased out last year (to virtually no complaint since nobody was actually verifying any of the signatures, much less attempting to confirm that their keys were discoverable[1]). [1]: https://blog.yossarian.net/2023/05/21/PGP-signatures-on-PyPI-worse-than-useless https://blog.yossarian.net/2023/05/21/PGP-signatures-on-PyPI...
    - guappa 2y ago
      > to virtually no complaint since nobody was actually verifying any of the signatures And this is in no way a consequence of pypi stopping to host public keys right? Say the whole story at least… Say that there used to be a way to verify the signatures but you dropped it years ago and since then the signatures have been useless.
      - woodruffw 2y ago
        If it did, it was well before I ever began to work on PyPI. By the time I came around, PGP signature support was vestigial twice over and the public key discovery network on the Internet was rapidly imploding. (But also: having PyPI be the keyserver defeats the point, since PyPI could then trivially replace my package's key. If that's the "whole story," it's not a very good one.)
        freeone3000 2y ago
        This attestation doesn’t change a ton with that, though. The point is to provide chain of custody — it got to my computer, from pypi, from ???. The PGP signature, much like a self-signed android app, verifies that it continues to be the same person.
- chippiewill 2y ago
  I still think this is overly strict. Supporting arbitrary OIDC providers is not excessively complex or particularly rare, the major cloud providers all support it in one way or another [1][2][3], as does Hashicorp Vault [4]. I disagree that the primary benefit over a manual API token is _knowing_ that the OIDC IdP is following the best practices you talk about. Having it rely on asymmetric keys makes the process more secure and scalable than API tokens for those that choose to use it. I think there's a separate question around trust. But I think blocking non-trusted publishers from using a more secure form of authentication isn't the answer. Instead I think it makes more sense to use nudges in the PyPI UI and eventually of consumers (e.g. pip) to indicate that packages have come from non-trusted publishers. [1] https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_pr... [2] https://learn.microsoft.com/en-us/graph/api/resources/federatedidentitycredentials-overview?view=graph-rest-1.0 https://learn.microsoft.com/en-us/graph/api/resources/federa... [3] https://cloud.google.com/identity-platform/docs/web/oidc https://cloud.google.com/identity-platform/docs/web/oidc [4] https://developer.hashicorp.com/vault/docs/auth/jwt https://developer.hashicorp.com/vault/docs/auth/jwt
  - guappa 2y ago
    I can create a github account. How does that make me trustworthy? How being able to create a github account prevents me from adding a virus in my module?
    - chippiewill 2y ago
      It's not about the package maintainer, it's about the trustworthiness of the OIDC issuer to prove the identity of a user. A poorly maintained issuer could leak their secret keys, allowing anyone to impersonate any package from their service.
      - guappa 2y ago
        But what use does it serve to prove that I am user "qioaisjqowihjdoaih" on github? I mean it only proves I authenticated successfully. Nothing else.
- deleted 2y ago
  [deleted]

SethMLarson 2y ago

Because every CI/ID provider has a different set of claims and behaviors that would constitute a "secure" policy for verification. If there was one singular way to do that then we could, but there isn't yet so PyPI needs to onboard providers piecemeal. The work to add a new provider is not massive, the reason there are not tons of providers isn't because the work is hard but rather because people are voting with their feet so Github and Gitlab make sense as initial providers to support.

woodruffw 2y ago

Because the security benefit of Trusted Publishing via OIDC versus normal API tokens is marginal at small scales, in two senses: 1. The primary benefit of Trusted Publishing over a manual API token is knowing that the underlying OIDC IdP has an on-call staff, proper key management and rotation policies, etc. These can be guaranteed for GitHub, GitLab, etc., but they're harder to prove for one-off self-hosted CI setups. For the latter case, the user is no better off than they would be with a manual API token, which is still (and will always be) supported. 2. If the overwhelming majority of traffic comes from a single CI/CD provider, adding more code to support generic OIDC IdPs increases PyPI's attack surface for only marginal user benefit. There also is no "open interface" for PyPI to really use here: this is all built on OIDC, but each OIDC provider needs to have its unique claims mapped to something intelligible by PyPI. That step requires thoughtful, manual, per-IdP consideration to avoid security issues.

antononcube 2y ago
> [...] the user is no better off than they would be with a manual API token, which is still (and will always be) supported. This is good to know. I did not see related statements in of the documents linked to this discussion, though.
- antononcube 2y ago
  I am not sure why my comment above is downvoted -- if you know where the perpetual optionality of digital attestations is officially stated, please, provide a link.
freeone3000 2y ago
I think I would be better off with API key + PGP than API key alone. And that’s being phased out?
- woodruffw 2y ago
  You can no longer upload a PGP signature to PyPI, if that's what you mean. That was phased out last year (to virtually no complaint since nobody was actually verifying any of the signatures, much less attempting to confirm that their keys were discoverable[1]). [1]: https://blog.yossarian.net/2023/05/21/PGP-signatures-on-PyPI-worse-than-useless https://blog.yossarian.net/2023/05/21/PGP-signatures-on-PyPI...
  - guappa 2y ago
    > to virtually no complaint since nobody was actually verifying any of the signatures And this is in no way a consequence of pypi stopping to host public keys right? Say the whole story at least… Say that there used to be a way to verify the signatures but you dropped it years ago and since then the signatures have been useless.
    - woodruffw 2y ago
      If it did, it was well before I ever began to work on PyPI. By the time I came around, PGP signature support was vestigial twice over and the public key discovery network on the Internet was rapidly imploding. (But also: having PyPI be the keyserver defeats the point, since PyPI could then trivially replace my package's key. If that's the "whole story," it's not a very good one.)
      - freeone3000 2y ago
        This attestation doesn’t change a ton with that, though. The point is to provide chain of custody — it got to my computer, from pypi, from ???. The PGP signature, much like a self-signed android app, verifies that it continues to be the same person.
chippiewill 2y ago
I still think this is overly strict. Supporting arbitrary OIDC providers is not excessively complex or particularly rare, the major cloud providers all support it in one way or another [1][2][3], as does Hashicorp Vault [4]. I disagree that the primary benefit over a manual API token is _knowing_ that the OIDC IdP is following the best practices you talk about. Having it rely on asymmetric keys makes the process more secure and scalable than API tokens for those that choose to use it. I think there's a separate question around trust. But I think blocking non-trusted publishers from using a more secure form of authentication isn't the answer. Instead I think it makes more sense to use nudges in the PyPI UI and eventually of consumers (e.g. pip) to indicate that packages have come from non-trusted publishers. [1] https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_pr... [2] https://learn.microsoft.com/en-us/graph/api/resources/federatedidentitycredentials-overview?view=graph-rest-1.0 https://learn.microsoft.com/en-us/graph/api/resources/federa... [3] https://cloud.google.com/identity-platform/docs/web/oidc https://cloud.google.com/identity-platform/docs/web/oidc [4] https://developer.hashicorp.com/vault/docs/auth/jwt https://developer.hashicorp.com/vault/docs/auth/jwt
- guappa 2y ago
  I can create a github account. How does that make me trustworthy? How being able to create a github account prevents me from adding a virus in my module?
  - chippiewill 2y ago
    It's not about the package maintainer, it's about the trustworthiness of the OIDC issuer to prove the identity of a user. A poorly maintained issuer could leak their secret keys, allowing anyone to impersonate any package from their service.
    - guappa 2y ago
      But what use does it serve to prove that I am user "qioaisjqowihjdoaih" on github? I mean it only proves I authenticated successfully. Nothing else.
deleted 2y ago
[deleted]