10 ms·

Leaked admin access token to Python, PyPI, and PSF GitHub repos

114 points by elchief 2y ago

elchief 2y ago
The JFrog Security Research team has recently discovered and reported a leaked access token with administrator access to Python’s, PyPI’s and Python Software Foundation’s GitHub repositories, which was leaked in a public Docker container hosted on Docker Hub. As a community service, the JFrog Security Research team continuously scans public repositories such as Docker Hub, NPM, and PyPI to identify malicious packages and leaked secrets. The team reports any findings to the relevant maintainers before attackers can take advantage of them. Although we encounter many secrets that are leaked in the same manner, this case was exceptional because it is difficult to overestimate the potential consequences if it had fallen into the wrong hands – one could supposedly inject malicious code into PyPI packages (imagine replacing all Python packages with malicious ones), and even to the Python language itself! The JFrog Security Research team identified the leaked secret and immediately reported it to PyPI’s security team, who revoked the token within a mere 17 minutes! This post will explain how we found a GitHub PAT that provided access to the entire Python infrastructure and prevented a supply chain disaster. Using this case, we will discuss the importance of (also) shifting right in secrets detection – searching for secrets in binaries and production artifacts, not just on source code.
- sans_souse 2y ago
  How were these leaked in this specific example? Am I right in assuming it was more user-error than malicious intent, and - if so, shouldn't this be a resolvable problem for the back-end as far as coding practices concerning public vs private keys and such? (note I am by no means an expert on coding or security)
  - deleted 2y ago
    [deleted]
  - ketch 2y ago
    The article has answers > It seems that the original author > - Briefly added the authorization token to their source code > - Ran the source code (Python script), which got compiled into a .pyc binary with the auth token > - Removed the authorization token from the source code, but didn’t clean the .pyc > - Pushed both the clean source code and the unclean .pyc binary into the docker image
    - bheadmaster 2y ago
      > - Pushed both the clean source code and the unclean .pyc binary into the docker image Oof. Honestly, I can't blame the guy for a mistake like this, it's just so easy to make. But then again, deploying images built on a development laptop is generally an error-prone activity. This is why build and deployment servers exist.
      - sitkack 2y ago
        Why does one person have admin on all those repos? Why is Python still running any classic access tokens? Why is the access token EVER in the source code? What other stuff is “running on their laptop”? No pass! People with access to the repos shouldn’t also have access to push bits to the world. It puts those people with that access in grave physical danger. Edit, https://blog.pypi.org/posts/2024-07-08-incident-report-leaked-admin-personal-access-token/ https://blog.pypi.org/posts/2024-07-08-incident-report-leake... This token has been in the wild for 15 months! The JFrog post cannot say that disaster was averted because we do not know.
        andrewSC 2y ago
        This is the correct set of questions to be asking. I’m a little more than surprised there aren’t some defined processes and automation around high viz workflows/stuff like this. When are people going to take cybersec and opsec seriously? Esp. In big projects?
        sitkack 2y ago
        And the logs are gone for both GitHub and docker hub. We should assume anything that could get compromised is compromised.
oakpond 2y ago
Yikes, TIL: never build production artifacts from a development checkout?
- ilc 2y ago
  There is a reason CI/CD servers exist. Heck there's a few pieces of software I work on that I can't build locally, I just push WIPs to CI/CD. I could try to setup a CI/CD pipeline of my own, but it isn't worth the time.
  - arielcostas 2y ago
    Maybe I'm too young, but I can't imagine building stuff locally and then pushing. Too messy later figuring out why stuff worked on your machine but not the end users'. I like way better setting up a CI pipeline on GitHub, building stuff remotely and then downloading the final binary to my own device or a VM to do manual testing.
- vetrom 2y ago
  This, in part, is why distributions like Debian go to the trouble they do to build everything from more-or-less isolated sourcecode artifacts. Sure, its not always perfect, and getting say, blobless, is a challenge of its own, but no distro in their right mind for example, would ship arbitrary .o or ld.so objects in source packages.
throwaway81523 2y ago
Why did a leakable token this powerful even exist?
- viralpraxis 2y ago
  Incompetence
pjungwir 2y ago
How did JFrog know this github token was so powerful, compared to all the other ones I'm sure their scanner detects? What caused a human to get involved?

114 points by elchief 2y ago

The JFrog Security Research team has recently discovered and reported a leaked access token with administrator access to Python’s, PyPI’s and Python Software Foundation’s GitHub repositories, which was leaked in a public Docker container hosted on Docker Hub. As a community service, the JFrog Security Research team continuously scans public repositories such as Docker Hub, NPM, and PyPI to identify malicious packages and leaked secrets. The team reports any findings to the relevant maintainers before attackers can take advantage of them. Although we encounter many secrets that are leaked in the same manner, this case was exceptional because it is difficult to overestimate the potential consequences if it had fallen into the wrong hands – one could supposedly inject malicious code into PyPI packages (imagine replacing all Python packages with malicious ones), and even to the Python language itself! The JFrog Security Research team identified the leaked secret and immediately reported it to PyPI’s security team, who revoked the token within a mere 17 minutes! This post will explain how we found a GitHub PAT that provided access to the entire Python infrastructure and prevented a supply chain disaster. Using this case, we will discuss the importance of (also) shifting right in secrets detection – searching for secrets in binaries and production artifacts, not just on source code.

sans_souse 2y ago
How were these leaked in this specific example? Am I right in assuming it was more user-error than malicious intent, and - if so, shouldn't this be a resolvable problem for the back-end as far as coding practices concerning public vs private keys and such? (note I am by no means an expert on coding or security)
- deleted 2y ago
  [deleted]
- ketch 2y ago
  The article has answers > It seems that the original author > - Briefly added the authorization token to their source code > - Ran the source code (Python script), which got compiled into a .pyc binary with the auth token > - Removed the authorization token from the source code, but didn’t clean the .pyc > - Pushed both the clean source code and the unclean .pyc binary into the docker image
  - bheadmaster 2y ago
    > - Pushed both the clean source code and the unclean .pyc binary into the docker image Oof. Honestly, I can't blame the guy for a mistake like this, it's just so easy to make. But then again, deploying images built on a development laptop is generally an error-prone activity. This is why build and deployment servers exist.
    - sitkack 2y ago
      Why does one person have admin on all those repos? Why is Python still running any classic access tokens? Why is the access token EVER in the source code? What other stuff is “running on their laptop”? No pass! People with access to the repos shouldn’t also have access to push bits to the world. It puts those people with that access in grave physical danger. Edit, https://blog.pypi.org/posts/2024-07-08-incident-report-leaked-admin-personal-access-token/ https://blog.pypi.org/posts/2024-07-08-incident-report-leake... This token has been in the wild for 15 months! The JFrog post cannot say that disaster was averted because we do not know.
      - andrewSC 2y ago
        This is the correct set of questions to be asking. I’m a little more than surprised there aren’t some defined processes and automation around high viz workflows/stuff like this. When are people going to take cybersec and opsec seriously? Esp. In big projects?
        sitkack 2y ago
        And the logs are gone for both GitHub and docker hub. We should assume anything that could get compromised is compromised.

oakpond 2y ago

Yikes, TIL: never build production artifacts from a development checkout?

ilc 2y ago
There is a reason CI/CD servers exist. Heck there's a few pieces of software I work on that I can't build locally, I just push WIPs to CI/CD. I could try to setup a CI/CD pipeline of my own, but it isn't worth the time.
- arielcostas 2y ago
  Maybe I'm too young, but I can't imagine building stuff locally and then pushing. Too messy later figuring out why stuff worked on your machine but not the end users'. I like way better setting up a CI pipeline on GitHub, building stuff remotely and then downloading the final binary to my own device or a VM to do manual testing.
vetrom 2y ago
This, in part, is why distributions like Debian go to the trouble they do to build everything from more-or-less isolated sourcecode artifacts. Sure, its not always perfect, and getting say, blobless, is a challenge of its own, but no distro in their right mind for example, would ship arbitrary .o or ld.so objects in source packages.

throwaway81523 2y ago

Why did a leakable token this powerful even exist?

viralpraxis 2y ago
Incompetence

pjungwir 2y ago

How did JFrog know this github token was so powerful, compared to all the other ones I'm sure their scanner detects? What caused a human to get involved?