4 ms·
FYI pypi tokens look like pypi-9NX39cdNn0AH1cCl1bMT48eKzf4Rhvw1mipk1FZTPrpR9 The integration means that GitHub knows to recognize this format, and calls some A
by remram 6y ago
FYI pypi tokens look like pypi-9NX39cdNn0AH1cCl1bMT48eKzf4Rhvw1mipk1FZTPrpR9
The integration means that GitHub knows to recognize this format, and calls some API of pypi.org when it finds one so PyPI can revoke it.
As always, please allow me to lament that we don't have a standard for this, such as secret-token:pypi.org/9NX39cdNn0AH1cCl1bMT48eKzf4Rhvw1mipk1FZTPrpR9, which would let any system know that this string is a secret and that pypi.org should be notified (for example via POST pypi.org/.well-know/compromised-secret). See also https://news.ycombinator.com/item?id=25978185 https://news.ycombinator.com/item?id=25978185
- nindalf 6y agoAccording to the documentation (https://docs.github.com/en/developers/overview/secret-scanning https://docs.github.com/en/developers/overview/secret-scanni...), secret issuers specify a regex that can detect secrets they've issued. "Be as precise as possible, because this will reduce the number of false positives" - that's the guideline from GitHub. Github runs the regex on every commit that is uploaded and informs the secret provider when a match occurs.
- kevincox 6y agoI wonder if false-positives often result in GitHub sending secrets to the wrong service.
- danudey 6y agoI wonder if any of those services have a combination of bad regexes and bad validation and could be SQL injected by committing a malicious faux-token to GitHub.
- remram 6y agoI see that they document the alerting endpoint there. The only piece missing is building the URL from the token format. I hope we get there someday, and everyone can deploy this without having to replicate GitHub's registry of token formats. This page also mentions that they "strongly recommend you implement signature validation in your secret alert service", but I'm not sure why. Isn't the fact that they send valid tokens proof that they have really found a leak?
- dragonwriter 6y agoSo, you can submit an overly generous (or specifically crafted) regex to get notified of tokens that someone else issued if you know their format?
- l0b0 6y agoOne cool data format standard I only recently learned about is multihash[1] - a self-describing hash format: the first byte represents the hashing algorithm, the second byte represents the length of the hash, and the subsequent [length] bytes is the actual hash. Something similar for tokens would be really useful. [1] https://multiformats.io/multihash/ https://multiformats.io/multihash/
- woodruffw 6y agoHey there! I designed and implemented PyPI's tokens (although not the secret scanning integration). They're actually just macaroons[1] internally, which means that they could easily be upgraded at some point to include a reporting URL like you mention. Just as a tidbit: they were originally prefixed with "pypi:" rather than "pypi-", but that colon caused problems for a few packaging utilities. Any sort of in-band signaling like that is unlikely to gain widespread adoption for exactly that reason :-) [1]: https://en.wikipedia.org/wiki/Macaroons_(computer_science) https://en.wikipedia.org/wiki/Macaroons_(computer_science)
- remram 6y agoInteresting. I can get the "pypi.org" domain from the base64-encoded part, however I don't see anything about revocation in the paper. Your reporting endpoint seems protected by a secret key that GitHub holds. Any reason PyPI can't accept anonymous submission of compromised tokens? If I find a PyPI token on my own server, can I not post it to https://pypi.org/_/github/disclose-token https://pypi.org/_/github/disclose-token without getting a key from you first?
- woodruffw 6y ago> I don't see anything about revocation in the paper. I don't believe it's something standardized or considered by the original whitepaper. Macaroons have the ability to contain arbitrary data, however, so it wouldn't be difficult to add revocation information to them. > If I find a PyPI token on my own server, can I not post it to https://pypi.org/_/github/disclose-token https://pypi.org/_/github/disclose-token without getting a key from you first? I wasn't part of the design, but my first thought goes to preventing the endpoint's use as an oracle: after a compromise, a malicious agent might find it useful to have an unlimited endpoint to test their stolen credentials against. Restricting use to a limited set of trusted entities avoids that.
- remram 6y agoDon't every other endpoints work as the oracle you describe? Are you worried about rate-limiting specifically? Also, the endpoint sends a 204 with no information about the validity of tokens, making it not much of an oracle. I think the payload is processed in the background too, preventing timing attacks.
- rlpb 6y agoUntil someone stores a secret without the prefix - because it's always the same, right?
- remram 6y agoAs long as the API wrappers don't mess this up, this has no reason to happen.