15 ms·

PyPI new user and new project registrations temporarily suspended

168 points by jonbaer 3y ago

omginternets 3y ago
I’ve seen multiple comments stating that namespacing solves (or at least mitigates) these kinds of attacks. Could someone kindly explain how? I’m sure it’s straightforward, but I don’t see it.
- bpicolo 3y ago
  It's a lot easier to typo a package (e.g. requests -> request results in the wrong package) than it is to typo a namespace-package (@namespace/requests -> @namespace/request would result in an error). Somewhat likewise, namespaces can build trust in a way that single packages can't
  - jeroenhd 3y ago
    With namespacing you still run the risk of @namespace/requests -> @namsepace/requests typos. These may not be all that obvious. For example, the name of the popular Rust HTTP client "reqwest" is an intentional typo. tokio vs tokyo can also be a less than obvious typo.
    - Too 3y ago
      On the other hand, this opens up for two potential places to typo. Both the name and the namespace. This happens often to me on GitHub, you keep browsing some code, only to later realize you are in someone’s personal fork of the official project. Especially when the original author isn’t a big organization, you only have two equally arbitrary usernames as namespaces to compare against.
      - jeroenhd 3y ago
        If the namespace is handled correctly, you won't accidentally download a malicious dependency if you get the namespace right. You'll get an error, but that's what you want for that kind of typo. The cryptic username problem is indeed a bother, but often popular projects will use readable names for their Github accounts. There's no real fix for that if you depend on a much smaller project. When you think you may be running that risk, you should probably wonder if it's a good idea to depend on a project made by someone small enough not to be instantly recognizable within your specific programming niche.
  - omginternets 3y ago
    Ah right! It was indeed straightforward — thank you!
- woodruffw 3y ago
  It doesn't, at least not directly. Namespaces make typosquatting more difficult, but it doesn't stop the main other incentives for spamming an index with inauthentic users and packages, i.e.: 1. Sneaking an inauthentic dependency into a tree somewhere; 2. Convincing less experienced users to install your package directly. My understanding that (2), in particular, is an increasingly issue in cryptocurrency and other communities: inexperienced users typically talk on Discord and other chats, and may not fully understand that `pip install foo` essentially means "allow a random person to run code on my machine."
  - tiedieconderoga 3y ago
    Can "pip install" actually run code? My understanding was that it doesn't, and a package needs to be imported by a script before anything gets run.
    - woodruffw 3y ago
      > Can "pip install" actually run code? Indeed it can, and in fact has to by necessity in many cases: `pip install` boils down to (a variant of) `python setup.py install` for many packages, where `setup.py` is package-controlled arbitrary Python code. This is also true for subcommands like `pip download`, as well as `pip install --dry-run`. (There's some subtlety here, since this only applies to source distributions and not wheels. But source distributions are still the "baseline" way to distribute Python packages.) > and a package needs to be imported by a script before anything gets run. Even with wheels, this part is unfortunately not true: a package can install a `.pth` file[1], which can be used to auto-load code on Python interpreter startup, even if the malicious package itself is never imported directly. [1]: https://docs.python.org/3/library/site.html https://docs.python.org/3/library/site.html
    - remexre 3y ago
      I thought it runs setup.py
      - dalke 3y ago
        There is a long-term goal to remove setup.py in favor of letting "a thousand flowers bloom". In modern Python setups, the pyproject.toml contains an entry for the 'build-backend'. There is a fall-back to 'setuptools.build_meta:__legacy__' because a large number of projects still use setup.py. PyPI recommends hatchling, as the example used in https://packaging.python.org/en/latest/tutorials/packaging-projects/ https://packaging.python.org/en/latest/tutorials/packaging-p... ("You can choose from a number of backends; this tutorial uses Hatchling by default, but it will work identically with setuptools, Flit, PDM, and others that support the [project] table for metadata.") These do not use setup.py. I use setup.py because the new build backends don't seem to support C extensions all that well. My package also has Cython code, and some Python-based code generation for the C code. Another package I know, RDKit, by default automatically downloads missing packages if not available. (These packages are provided as source from the respective authors.) I think distribution people really don't want the build step to be able to run arbitrary code, but they face a long, up-hill battle.
efitz 3y ago
> The volume of malicious users and malicious projects being created on the index in the past week has outpaced our ability to respond to it in a timely fashion, especially with multiple PyPI administrators on leave. People suck and that is why we cannot have nice things.
- Hnrobert42 3y ago
  The shame of it is that a tiny, tiny, tiny fraction of people really suck. The number of people involved is probably less than a thousand. (This is my totally uninformed speculation.)
  - deleted 3y ago
    [deleted]
- miohtama 3y ago
  It’s calle Tragedy of Commons https://en.m.wikipedia.org/wiki/Tragedy_of_the_commons https://en.m.wikipedia.org/wiki/Tragedy_of_the_commons One solution would be have a minimum payment or a trusted sponsor in order to register a new package. A minimum trust score through well-known community persons, or reduce spam by payments. On the root cause why people suck is that different cultures and different people have different values. For many people making money by spamming is not an issue in their own internal value system. Thus, it is better to be tackled by making spam unprofitable.
  - checkyoursudo 3y ago
    I disagree that this is a case of tragedy of the commons (totc). Totc is about unregulated depletion of resources, just like your link says. This pypi stuff is about maliciousness. Criminal or misanthropic behavior is not totc; it's just antisocial. Also, https://aeon.co/essays/the-tragedy-of-the-commons-is-a-false-and-dangerous-myth https://aeon.co/essays/the-tragedy-of-the-commons-is-a-false...
    - logdap 3y ago
dlor 3y ago
I know folks hate the centralization of identity management to the big identity providers, but as AI gets better and better at defeating captcha it's going to get harder and harder for the small, independent ones to operate reliably and securely.
- adhesive_wombat 3y ago
  "Never use your real name on that internet thing" will soon be a quaint throwback.
  - pixl97 3y ago
    We're moving closer to "when you post on the internet, it is signed and big brother knows you did it. Watch your words"
    - ivanmontillam 3y ago
      We are already there since many years ago.
- vhcr 3y ago
  Open Source captcha libraries are non-viable, you can train a NN on a few hours with a simple CNN architecture, there are some papers that describe how to do it.
  - reaperman 3y ago
    We didn't have a lot of luck with simple CNN's on modern CAPTCHA's. But more advanced techniques work great on pretty much all of them. Shameless plug: http://resolved.dominuslabs.com http://resolved.dominuslabs.com feel free to reach out for custom work - specializing in accessing gated sites for new search engines to do better web crawling, scraping public data, building better API integrations on top of existing services, etc.
- reaperman 3y ago
  I wrote a lot of the systems to break captchas. Currently the "small" (really medium), independent ones are doing way way way better than the big identity providers. Google reCAPTCHA was particularly easy for us to get past. Shameless plug: http://resolved.dominuslabs.com http://resolved.dominuslabs.com feel free to reach out for custom work - specializing in accessing gated sites for new search engines to do better web crawling, scraping public data, building better API integrations on top of existing services, etc.
BozeWolf 3y ago
I wonder if Apple’s trick would work for python packages. Pay a few (5? 10? 20?) bucks to become a Pypi developer/sponsor, it would also pay pypi’s operational bills. It raises the bar for malicious actors. Additionally, if pypi provides keys to developers, pypi can also revoke certificates for developers making malicious packages. It would need a system which checks package signatures on startup of a python app, or maybe there is some other way to do that. Pip —check or some thing which then runs in pipelines, specifically meant to check for malicious packages each day. To decrease the barrier of entry on pypi, students could identify with their student number. Or pypi could work with a system where you have trusted users and packages and untrusted users. A bit like the blue checkmark, but without the negative connotation.
- CameronNemo 3y ago
  Universities could host gitlab instances with pypi registries built-in.
- miohtama 3y ago
  You can always have a trusted community member to waive the payment requirement, so anyone who demostrates genuine effort can have it for free.
  - BozeWolf 3y ago
    This. Also enterprises using specific (still untrusted) packages could pay it for the developer. There must be a way to do this. But in true spirit of python the system should be as open as possible.
- woodruffw 3y ago
  FD: I’ve done some work on PyPI, but I am not an admin and everything below is an independent opinion/understanding. PyPI’s operational costs are, to my understanding, mostly covered: hosting is graciously provided by a sponsor, and the PSF currently funds roles for its develop, administration, and security. More funding is always good and I believe the PyPI admins are looking to enable payments through the newly released “Organizations” feature[1]. Edit: and to make it more clear: payments for Organizations would be principally aimed at corporations and other groups. > Additionally, if pypi provides keys to developers, pypi can also revoke certificates for developers making malicious packages. There are currently plans in progress to allow PyPI users to upload Sigstore[2] signatures for packages. That won’t directly address the spam issue, however — signatures will be opt-in (by necessity, due to the size of the packaging ecosystem), and no codesigning scheme can prevent a spammer from simply assuming a new identity (especially when new identities are “free,” as they normally are.) Separately, revocation itself is a nasty problem for packaging ecosystems to deal with: ecosystems with trillions of dollars of value behind them (like the Web PKI) struggle with it, so it’s not immediately clear that it would be anything other than an additional operational burden. Similarly for reputational systems: they’re difficult to operationalize without additional maintainer burden. That’s not to say that they’re necessarily bad or impractical for PyPI’s purposes; only that I’m not aware of a successful use of them in an open source packaging ecosystem. Compare, for example, PGP’s WoT failures. [1]: https://blog.pypi.org/posts/2023-04-23-introducing-pypi-organizations/ https://blog.pypi.org/posts/2023-04-23-introducing-pypi-orga... [2]: https://www.sigstore.dev/ https://www.sigstore.dev/
benatkin 3y ago
I hope they get a handle on this and don't do anything else to alienate data portability folks like making Microsoft the only "Trusted Publisher". https://news.ycombinator.com/item?id=35646436 https://news.ycombinator.com/item?id=35646436
- ewdurbin 3y ago
  there are no plans to limit trusted publishers. in fact there is another in the works: https://github.com/pypi/warehouse/issues/13551 https://github.com/pypi/warehouse/issues/13551
kccqzy 3y ago
A massive amount of people signing up with malicious intent (the Sybil attack) is a thing afflicting every single online service with a signup. I don't blame PyPI, but I think it's genuinely a hard problem to solve. Big Tech often combats this by running a bunch of heuristics and suspending accounts they detect to be bad, but of course we know how easy it is for even tiny false positives to blow up. I think we might come to the end of the era of free online accounts; only those accounts requiring a method of payment will continue to be viable.
blibble 3y ago
namespacing? what's that
tzhenghao 3y ago
Tragedy of the commons - only need a few bad actors to ruin it all for us. Almost all distributors face this problem, from Docker Hub to PyPI. This also reminded me of official Postgres Docker image running a cryptominer in the background [1] [1] - https://github.com/docker-library/postgres/issues/770 https://github.com/docker-library/postgres/issues/770
- boomanaiden154 3y ago
  It doesn't seem like the linked Github issue really backs up the claim that malware was getting shipped in the official image? The OP kept claiming that (even titling it `[Confirmed]`), but only one other person was able to corroborate the claim and their image hashes didn't match any of the hosted ones.
- brasetvik 3y ago
  Reading that thread it doesn't seem like the official image shipped with any cryptominer at any point, and that it's more likely that the container got compromised in other ways. (A compromised [superuser connection to Postgres can execute shell code](https://medium.com/r3d-buck3t/command-execution-with-postgresql-copy-command-a79aef9c2767 https://medium.com/r3d-buck3t/command-execution-with-postgre...), so that seems more likely than the image shipping with a miner)
  - jeroenhd 3y ago
    Several other issues also mention this problem across versions, with one issue mentioning that changing the default password fixed the issue. It looks like the miner was installed because the Postgres port got exposed with a weak password.
- nonethewiser 3y ago
  That is a very interesting read. The maintainers seem more sane than Antender. Im even sympathetic to many of his points but he cant be sure a maintainer put a cryptominer in the image. Shouldnt be so fast to point fingers. Are there any aggregators of interesting github issues? Links with a bit of context about them make for some fascinating reads.
throwaway892238 3y ago
Methods to deal with malicious actors in your system: - Require toilsome identity verification. Things that are in short supply and are difficult to get and uniquely identify a person. Examples include a phone number, credit card, driver's license, mailed letter, etc. - Require a referral, both for accounts and for new packages. Don't allow a signup unless the user has a referral code generated by another user with good karma. This isn't fool-proof, as a user that does get an account can then generate more accounts. But it makes it easier to revoke them en-masse, and forces users to be more scrupulous about who they refer, as you can block the referrer as well as the malicious user. - Require community review, both of new users, and new packages. New users/packages are sent to a mailing list of moderators and somebody has to approve it, but someone who notices a problem can also reject it, even after it's been mistakenly approved. Slower, but there's more eyeballs to spot malicious stuff. This is more common among open source projects. (Growing your moderator list is important, as well as automating as much of the review process as possible; obviously PyPI currently has a shortage of moderators, but they should have a huge number of them by now!) - Don't allow new users to do certain things until enough time has passed or they have accrued enough karma points. May require fixing bugs, answering questions, etc; work which benefits the community and most malicious actors wouldn't invest the time and effort in. Again not fool-proof, but definitely increases the time and difficulty for successful attacks. - Captchas. These can obviously be worked around, but are a minimum-effort way to avoid bot signups. For better defense-in-depth, combine multiple methods.
- nonethewiser 3y ago
  What about some testnet that allows people to push things up as a test? That seems like a valid need but a source of lots of garbage at the same time.
  - mplewis 3y ago
    This is what https://test.pypi.org/ https://test.pypi.org/ is for.
- Hnrobert42 3y ago
  These are good suggestions. I also like @BozeWolf’s suggestion below about charging for new accounts. You could even do a combo. Like $25 sign up. Free with referral.
Severian 3y ago
I've been very cautious the last couple of years due to these bad actors when looking at packages that might suit my needs. If there is no online presence of the source code (git anything, zips/gzs, etc), multiple packages submitted in a short time frame, or a greater than normal amount, an/or a derivation/plugin of a popular package it's usually a no-go. For those that I do possibly trust, I then download the package (pip download) and review it. Doing a quick regex for URLs or exec() calls helps, but I probably should use something like guarddog (https://github.com/DataDog/guarddog https://github.com/DataDog/guarddog)
- woodruffw 3y ago
  For what it's worth: `pip download` is capable of running arbitrary package-defined code[1], by design. You shouldn't use it as a security boundary. If you're trying to statically analyze a distribution before doing anything else with it, you should download it directly from the PEP 503[2] simple index. [1]: https://yossarian.net/res/pub/hushcon-west-2022.pdf https://yossarian.net/res/pub/hushcon-west-2022.pdf [2]: https://peps.python.org/pep-0503/ https://peps.python.org/pep-0503/
  - Severian 3y ago
    Good to know, I was not aware of this. Thank you!
- Too 3y ago
  > If there is no online presence of the source code Could this be utilized the other way around? Any package that was published using official GitHub action X, would automatically be signed as being created from something with source and link to source at given revision. Of course this source could contain bad contents or download even more bad contents, at least it establishes some level of chain of supply, where the first layer of source can be inspected.
  - woodruffw 3y ago
    > Could this be utilized the other way around? Any package that was published using official GitHub action X, would automatically be signed as being created from something with source and link to source at given revision. That's exactly the idea behind Sigstore[1] -- GitHub Actions (or any other CI system with an OIDC credential) can be leveraged for "machine identities", which in turn are bound to ephemeral signing keys. The end result is a scheme where users never have to touch or maintain a signing key while still getting all the benefits of a normal PKI-esque codesigning scheme. [1]: https://www.sigstore.dev/ https://www.sigstore.dev/
imran-iq 3y ago
It's only a matter of time until it happens to other systems too. Take rust for example where folks can install software via `cargo install <global name package>`. Top that off with fact that during compile (as an attacker) you have full access to your victims computer[0]. Python just happens to have more popularity for now. I think golang gets this right where you need the full package name to have it be usable with get/build/install commands. --- 0: https://doc.rust-lang.org/cargo/reference/build-scripts.html https://doc.rust-lang.org/cargo/reference/build-scripts.html
- woodruffw 3y ago
  Namespacing isn't the problem here: an ecosystem with two (or more) levels of namespacing has the exact same username and package stuffing issues. Namespacing arguably makes typosquatting more difficult, but that has nothing to do with build-time code execution (a common feature of packaging systems) or other package confusion techniques. Go is able to avoid these problems because it (arguably wisely) sidesteps nearly all package index problems by punting them to the source host.
  - fiddlerwoaroof 3y ago
    I think maven central has the right idea, but it could be generalized. Require people submitting packages to prove they own a domain and use the domain as the namespace representing the user. We could then use SRV records or TXT records to specify where a domain’s packages are authoritatively hosted and then have conventions to find a signing key for a given domain that can be used to verify cached packages in non-authoritative hosts.
    - woodruffw 3y ago
      Again: namespacing isn't the problem (or a totalizing solution) here. I admire Maven's approach on a conceptual level, but it leaves a lot to be desired on a technical level: it effectively gatekeeps the entire packaging ecosystem on the ability to purchase and maintain a domain name, which (1) is unnecessarily exclusionary, and (2) assumes trust in a namespace (DNS) that was never really designed to be authenticated or a source of stable, permanent identifiers (domains change hands all the times, including for hostile reasons). A modern version of Maven's scheme would bootstrap on well-known URIs over HTTPS instead, but even this would still represent a significant imposition on PyPI uploaders.
adontz 3y ago
PyPI could integrate with Google/Microsoft/Apple as an authentication system (OAuth?). Almost everyone has one of these IDs and it's hard enough to register new ones.
- woodruffw 3y ago
  > PyPI could integrate with Google/Microsoft/Apple as an authentication system (OAuth?). PyPI supports "trusted publishing,"[1] which provides a variant of this: it doesn't replace a user identity, but instead allows a platform (currently just GitHub, but support for others is on the way) to mint API tokens on a project's behalf. Binding PyPI identities to well-known IdPs would address some of the problems here, but also introduces new ones: it creates a new kind of account lockout state (users who lose access to their IdP service, for whatever reason), introduces regulatory and data collection concerns, may prove excessively restrictive to users in countries with filtered Internet access, etc. [1]: https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/ https://blog.pypi.org/posts/2023-04-20-introducing-trusted-p...
  - adontz 3y ago
    And all of these is already a thing for nuget?
- vhcr 3y ago
  All the spam I receive directly from gmail.com accounts kind of disproves your point.
- jjgreen 3y ago
  That's what Rust does, you can only register with crates.io with a GitHub account.
zyl1n 3y ago
As someone who straddles both Python and Java world, I don't seem hear this kind of malicious packages problem with Maven (central) as I do with PyPI. What could be the difference? Perhaps Java world is full of old farts like me doing mundane thing with a handful of well established libraries.
elashri 3y ago
I like the simple solution that arXiv use (which allow anyone to publish a pre-print) also it has order of magnitude more code (compared to PyPI packages). If you want to be able to publish for the first time, you have to get a referral from a well established author on the platform. And of course, things might be easier because you need to register using your real identify and that is excepted. This is different for PyPI that until now does not expect people to have their real identifies linked to their accounts.
_8j50 3y ago
I think this is the wrong problem they are addressing. They should allow malicious users to be a possibility but what is lacking is package authentication/signing. Sign all packages, new users get the worst grade, users with a lot of downloads over a long period get a slightly better grade then have different levels if trusted third party reviewed grades. For new users' packages you get a big red warning that needs to be confirmed and bypassed in multiple ways with a message like "ERROR:THIS PACKAGE IS FROM AN ILL-REPUTED SOURCE, IT IS LIKELY MALICIOUS AND HASN'T UNDERGONE HUMAN REVIEW" kind of scream at you a little.
- woodruffw 3y ago
  Codesigning is something that's actively being worked on for PyPI. It won't have that kind of "noob" reputation scheme, though: those kinds of schemes (1) effectively punish newcomers for daring to contribute to an ecosystem, and (2) are incredibly hard to build trust policies around (what's the trust gradation between "new package" and "very old package with a new maintainer"?).
  - _8j50 3y ago
    The grade is for the maintainer not the package. It does punish newcomers but that's the point, newcomers cannot be trusted. People don't search for packages directly on pypi, they find them elsewhere and then use pypi to install it. If you are a new maintainer, you build reputation elsewhere and ask users to ignore the warning on pypi. After say 1k unique downloads and 90 day probation, you move to the next level with a less serious warning. To avoid all that though, you can purchase a code signing certificate and verify your identity (with pypi not just the ca) and even as a new comer there will be no warning for your packages. After observing many such systems my view is that you have to make up your mind and have a rigid and reliable reputation system or have none at all. The middle is the most dangerous place for everyone but I fear open source people like to repeat mistakes and sacrifice the safety of their users for the sake of ideology. Newcomer+ no known reputation+unverified = warn people
lolinder 3y ago
Modern package registries implicitly treat all package authors as equally trustworthy—you don't need to know which organization wrote the `zip` package, you just need to know that there's a package by that name and it opens your zip files. It's charmingly egalitarian and quite convenient, but the global namespace both enables typosquatting and trains developers to not think at all about the people who write the stuff in their supply chain. The biggest point of failure in any system—the humans—has no first-class expression in the typical modern package registry. I can't help but wonder if this is something Maven got right. Namespacing packages according to domain names makes typosquatting vastly more difficult to pull off and makes authorship of dependencies a first-class concern for end users. If something is under org.apache, that means something about the contents of the package, and PyPI and company don't front that information nearly as well as Maven does.
NoboruWataya 3y ago
I know `pip search` has been disabled with PyPI for a while now due to a constant deluge of requests overwhelming the backend. Knowing that, and seeing this update, makes me think that PyPI is subject to an exceptional volume of malicious activity. Does anyone know if it's true that PyPI receives more abuse than similar projects and, if so, why?

168 points by jonbaer 3y ago

I’ve seen multiple comments stating that namespacing solves (or at least mitigates) these kinds of attacks. Could someone kindly explain how? I’m sure it’s straightforward, but I don’t see it.

bpicolo 3y ago
It's a lot easier to typo a package (e.g. requests -> request results in the wrong package) than it is to typo a namespace-package (@namespace/requests -> @namespace/request would result in an error). Somewhat likewise, namespaces can build trust in a way that single packages can't
- jeroenhd 3y ago
  With namespacing you still run the risk of @namespace/requests -> @namsepace/requests typos. These may not be all that obvious. For example, the name of the popular Rust HTTP client "reqwest" is an intentional typo. tokio vs tokyo can also be a less than obvious typo.
  - Too 3y ago
    On the other hand, this opens up for two potential places to typo. Both the name and the namespace. This happens often to me on GitHub, you keep browsing some code, only to later realize you are in someone’s personal fork of the official project. Especially when the original author isn’t a big organization, you only have two equally arbitrary usernames as namespaces to compare against.
    - jeroenhd 3y ago
      If the namespace is handled correctly, you won't accidentally download a malicious dependency if you get the namespace right. You'll get an error, but that's what you want for that kind of typo. The cryptic username problem is indeed a bother, but often popular projects will use readable names for their Github accounts. There's no real fix for that if you depend on a much smaller project. When you think you may be running that risk, you should probably wonder if it's a good idea to depend on a project made by someone small enough not to be instantly recognizable within your specific programming niche.
- omginternets 3y ago
  Ah right! It was indeed straightforward — thank you!
woodruffw 3y ago
It doesn't, at least not directly. Namespaces make typosquatting more difficult, but it doesn't stop the main other incentives for spamming an index with inauthentic users and packages, i.e.: 1. Sneaking an inauthentic dependency into a tree somewhere; 2. Convincing less experienced users to install your package directly. My understanding that (2), in particular, is an increasingly issue in cryptocurrency and other communities: inexperienced users typically talk on Discord and other chats, and may not fully understand that `pip install foo` essentially means "allow a random person to run code on my machine."
- tiedieconderoga 3y ago
  Can "pip install" actually run code? My understanding was that it doesn't, and a package needs to be imported by a script before anything gets run.
  - woodruffw 3y ago
    > Can "pip install" actually run code? Indeed it can, and in fact has to by necessity in many cases: `pip install` boils down to (a variant of) `python setup.py install` for many packages, where `setup.py` is package-controlled arbitrary Python code. This is also true for subcommands like `pip download`, as well as `pip install --dry-run`. (There's some subtlety here, since this only applies to source distributions and not wheels. But source distributions are still the "baseline" way to distribute Python packages.) > and a package needs to be imported by a script before anything gets run. Even with wheels, this part is unfortunately not true: a package can install a `.pth` file[1], which can be used to auto-load code on Python interpreter startup, even if the malicious package itself is never imported directly. [1]: https://docs.python.org/3/library/site.html https://docs.python.org/3/library/site.html
  - remexre 3y ago
    I thought it runs setup.py
    - dalke 3y ago
      There is a long-term goal to remove setup.py in favor of letting "a thousand flowers bloom". In modern Python setups, the pyproject.toml contains an entry for the 'build-backend'. There is a fall-back to 'setuptools.build_meta:__legacy__' because a large number of projects still use setup.py. PyPI recommends hatchling, as the example used in https://packaging.python.org/en/latest/tutorials/packaging-projects/ https://packaging.python.org/en/latest/tutorials/packaging-p... ("You can choose from a number of backends; this tutorial uses Hatchling by default, but it will work identically with setuptools, Flit, PDM, and others that support the [project] table for metadata.") These do not use setup.py. I use setup.py because the new build backends don't seem to support C extensions all that well. My package also has Cython code, and some Python-based code generation for the C code. Another package I know, RDKit, by default automatically downloads missing packages if not available. (These packages are provided as source from the respective authors.) I think distribution people really don't want the build step to be able to run arbitrary code, but they face a long, up-hill battle.

efitz 3y ago

> The volume of malicious users and malicious projects being created on the index in the past week has outpaced our ability to respond to it in a timely fashion, especially with multiple PyPI administrators on leave. People suck and that is why we cannot have nice things.

Hnrobert42 3y ago
The shame of it is that a tiny, tiny, tiny fraction of people really suck. The number of people involved is probably less than a thousand. (This is my totally uninformed speculation.)
- deleted 3y ago
  [deleted]
miohtama 3y ago
It’s calle Tragedy of Commons https://en.m.wikipedia.org/wiki/Tragedy_of_the_commons https://en.m.wikipedia.org/wiki/Tragedy_of_the_commons One solution would be have a minimum payment or a trusted sponsor in order to register a new package. A minimum trust score through well-known community persons, or reduce spam by payments. On the root cause why people suck is that different cultures and different people have different values. For many people making money by spamming is not an issue in their own internal value system. Thus, it is better to be tackled by making spam unprofitable.
- checkyoursudo 3y ago
  I disagree that this is a case of tragedy of the commons (totc). Totc is about unregulated depletion of resources, just like your link says. This pypi stuff is about maliciousness. Criminal or misanthropic behavior is not totc; it's just antisocial. Also, https://aeon.co/essays/the-tragedy-of-the-commons-is-a-false-and-dangerous-myth https://aeon.co/essays/the-tragedy-of-the-commons-is-a-false...
  - logdap 3y ago

dlor 3y ago

I know folks hate the centralization of identity management to the big identity providers, but as AI gets better and better at defeating captcha it's going to get harder and harder for the small, independent ones to operate reliably and securely.

adhesive_wombat 3y ago
"Never use your real name on that internet thing" will soon be a quaint throwback.
- pixl97 3y ago
  We're moving closer to "when you post on the internet, it is signed and big brother knows you did it. Watch your words"
  - ivanmontillam 3y ago
    We are already there since many years ago.
vhcr 3y ago
Open Source captcha libraries are non-viable, you can train a NN on a few hours with a simple CNN architecture, there are some papers that describe how to do it.
- reaperman 3y ago
  We didn't have a lot of luck with simple CNN's on modern CAPTCHA's. But more advanced techniques work great on pretty much all of them. Shameless plug: http://resolved.dominuslabs.com http://resolved.dominuslabs.com feel free to reach out for custom work - specializing in accessing gated sites for new search engines to do better web crawling, scraping public data, building better API integrations on top of existing services, etc.
reaperman 3y ago
I wrote a lot of the systems to break captchas. Currently the "small" (really medium), independent ones are doing way way way better than the big identity providers. Google reCAPTCHA was particularly easy for us to get past. Shameless plug: http://resolved.dominuslabs.com http://resolved.dominuslabs.com feel free to reach out for custom work - specializing in accessing gated sites for new search engines to do better web crawling, scraping public data, building better API integrations on top of existing services, etc.

BozeWolf 3y ago

I wonder if Apple’s trick would work for python packages. Pay a few (5? 10? 20?) bucks to become a Pypi developer/sponsor, it would also pay pypi’s operational bills. It raises the bar for malicious actors. Additionally, if pypi provides keys to developers, pypi can also revoke certificates for developers making malicious packages. It would need a system which checks package signatures on startup of a python app, or maybe there is some other way to do that. Pip —check or some thing which then runs in pipelines, specifically meant to check for malicious packages each day. To decrease the barrier of entry on pypi, students could identify with their student number. Or pypi could work with a system where you have trusted users and packages and untrusted users. A bit like the blue checkmark, but without the negative connotation.

CameronNemo 3y ago
Universities could host gitlab instances with pypi registries built-in.
miohtama 3y ago
You can always have a trusted community member to waive the payment requirement, so anyone who demostrates genuine effort can have it for free.
- BozeWolf 3y ago
  This. Also enterprises using specific (still untrusted) packages could pay it for the developer. There must be a way to do this. But in true spirit of python the system should be as open as possible.
woodruffw 3y ago
FD: I’ve done some work on PyPI, but I am not an admin and everything below is an independent opinion/understanding. PyPI’s operational costs are, to my understanding, mostly covered: hosting is graciously provided by a sponsor, and the PSF currently funds roles for its develop, administration, and security. More funding is always good and I believe the PyPI admins are looking to enable payments through the newly released “Organizations” feature[1]. Edit: and to make it more clear: payments for Organizations would be principally aimed at corporations and other groups. > Additionally, if pypi provides keys to developers, pypi can also revoke certificates for developers making malicious packages. There are currently plans in progress to allow PyPI users to upload Sigstore[2] signatures for packages. That won’t directly address the spam issue, however — signatures will be opt-in (by necessity, due to the size of the packaging ecosystem), and no codesigning scheme can prevent a spammer from simply assuming a new identity (especially when new identities are “free,” as they normally are.) Separately, revocation itself is a nasty problem for packaging ecosystems to deal with: ecosystems with trillions of dollars of value behind them (like the Web PKI) struggle with it, so it’s not immediately clear that it would be anything other than an additional operational burden. Similarly for reputational systems: they’re difficult to operationalize without additional maintainer burden. That’s not to say that they’re necessarily bad or impractical for PyPI’s purposes; only that I’m not aware of a successful use of them in an open source packaging ecosystem. Compare, for example, PGP’s WoT failures. [1]: https://blog.pypi.org/posts/2023-04-23-introducing-pypi-organizations/ https://blog.pypi.org/posts/2023-04-23-introducing-pypi-orga... [2]: https://www.sigstore.dev/ https://www.sigstore.dev/

benatkin 3y ago

I hope they get a handle on this and don't do anything else to alienate data portability folks like making Microsoft the only "Trusted Publisher". https://news.ycombinator.com/item?id=35646436 https://news.ycombinator.com/item?id=35646436

ewdurbin 3y ago
there are no plans to limit trusted publishers. in fact there is another in the works: https://github.com/pypi/warehouse/issues/13551 https://github.com/pypi/warehouse/issues/13551

kccqzy 3y ago

A massive amount of people signing up with malicious intent (the Sybil attack) is a thing afflicting every single online service with a signup. I don't blame PyPI, but I think it's genuinely a hard problem to solve. Big Tech often combats this by running a bunch of heuristics and suspending accounts they detect to be bad, but of course we know how easy it is for even tiny false positives to blow up. I think we might come to the end of the era of free online accounts; only those accounts requiring a method of payment will continue to be viable.

blibble 3y ago

namespacing? what's that

tzhenghao 3y ago

Tragedy of the commons - only need a few bad actors to ruin it all for us. Almost all distributors face this problem, from Docker Hub to PyPI. This also reminded me of official Postgres Docker image running a cryptominer in the background [1] [1] - https://github.com/docker-library/postgres/issues/770 https://github.com/docker-library/postgres/issues/770

boomanaiden154 3y ago
It doesn't seem like the linked Github issue really backs up the claim that malware was getting shipped in the official image? The OP kept claiming that (even titling it `[Confirmed]`), but only one other person was able to corroborate the claim and their image hashes didn't match any of the hosted ones.
brasetvik 3y ago
Reading that thread it doesn't seem like the official image shipped with any cryptominer at any point, and that it's more likely that the container got compromised in other ways. (A compromised [superuser connection to Postgres can execute shell code](https://medium.com/r3d-buck3t/command-execution-with-postgresql-copy-command-a79aef9c2767 https://medium.com/r3d-buck3t/command-execution-with-postgre...), so that seems more likely than the image shipping with a miner)
- jeroenhd 3y ago
  Several other issues also mention this problem across versions, with one issue mentioning that changing the default password fixed the issue. It looks like the miner was installed because the Postgres port got exposed with a weak password.
nonethewiser 3y ago
That is a very interesting read. The maintainers seem more sane than Antender. Im even sympathetic to many of his points but he cant be sure a maintainer put a cryptominer in the image. Shouldnt be so fast to point fingers. Are there any aggregators of interesting github issues? Links with a bit of context about them make for some fascinating reads.

throwaway892238 3y ago

Methods to deal with malicious actors in your system: - Require toilsome identity verification. Things that are in short supply and are difficult to get and uniquely identify a person. Examples include a phone number, credit card, driver's license, mailed letter, etc. - Require a referral, both for accounts and for new packages. Don't allow a signup unless the user has a referral code generated by another user with good karma. This isn't fool-proof, as a user that does get an account can then generate more accounts. But it makes it easier to revoke them en-masse, and forces users to be more scrupulous about who they refer, as you can block the referrer as well as the malicious user. - Require community review, both of new users, and new packages. New users/packages are sent to a mailing list of moderators and somebody has to approve it, but someone who notices a problem can also reject it, even after it's been mistakenly approved. Slower, but there's more eyeballs to spot malicious stuff. This is more common among open source projects. (Growing your moderator list is important, as well as automating as much of the review process as possible; obviously PyPI currently has a shortage of moderators, but they should have a huge number of them by now!) - Don't allow new users to do certain things until enough time has passed or they have accrued enough karma points. May require fixing bugs, answering questions, etc; work which benefits the community and most malicious actors wouldn't invest the time and effort in. Again not fool-proof, but definitely increases the time and difficulty for successful attacks. - Captchas. These can obviously be worked around, but are a minimum-effort way to avoid bot signups. For better defense-in-depth, combine multiple methods.

nonethewiser 3y ago
What about some testnet that allows people to push things up as a test? That seems like a valid need but a source of lots of garbage at the same time.
- mplewis 3y ago
  This is what https://test.pypi.org/ https://test.pypi.org/ is for.
Hnrobert42 3y ago
These are good suggestions. I also like @BozeWolf’s suggestion below about charging for new accounts. You could even do a combo. Like $25 sign up. Free with referral.

Severian 3y ago

I've been very cautious the last couple of years due to these bad actors when looking at packages that might suit my needs. If there is no online presence of the source code (git anything, zips/gzs, etc), multiple packages submitted in a short time frame, or a greater than normal amount, an/or a derivation/plugin of a popular package it's usually a no-go. For those that I do possibly trust, I then download the package (pip download) and review it. Doing a quick regex for URLs or exec() calls helps, but I probably should use something like guarddog (https://github.com/DataDog/guarddog https://github.com/DataDog/guarddog)

woodruffw 3y ago
For what it's worth: `pip download` is capable of running arbitrary package-defined code[1], by design. You shouldn't use it as a security boundary. If you're trying to statically analyze a distribution before doing anything else with it, you should download it directly from the PEP 503[2] simple index. [1]: https://yossarian.net/res/pub/hushcon-west-2022.pdf https://yossarian.net/res/pub/hushcon-west-2022.pdf [2]: https://peps.python.org/pep-0503/ https://peps.python.org/pep-0503/
- Severian 3y ago
  Good to know, I was not aware of this. Thank you!
Too 3y ago
> If there is no online presence of the source code Could this be utilized the other way around? Any package that was published using official GitHub action X, would automatically be signed as being created from something with source and link to source at given revision. Of course this source could contain bad contents or download even more bad contents, at least it establishes some level of chain of supply, where the first layer of source can be inspected.
- woodruffw 3y ago
  > Could this be utilized the other way around? Any package that was published using official GitHub action X, would automatically be signed as being created from something with source and link to source at given revision. That's exactly the idea behind Sigstore[1] -- GitHub Actions (or any other CI system with an OIDC credential) can be leveraged for "machine identities", which in turn are bound to ephemeral signing keys. The end result is a scheme where users never have to touch or maintain a signing key while still getting all the benefits of a normal PKI-esque codesigning scheme. [1]: https://www.sigstore.dev/ https://www.sigstore.dev/

imran-iq 3y ago

It's only a matter of time until it happens to other systems too. Take rust for example where folks can install software via `cargo install <global name package>`. Top that off with fact that during compile (as an attacker) you have full access to your victims computer[0]. Python just happens to have more popularity for now. I think golang gets this right where you need the full package name to have it be usable with get/build/install commands. --- 0: https://doc.rust-lang.org/cargo/reference/build-scripts.html https://doc.rust-lang.org/cargo/reference/build-scripts.html

woodruffw 3y ago
Namespacing isn't the problem here: an ecosystem with two (or more) levels of namespacing has the exact same username and package stuffing issues. Namespacing arguably makes typosquatting more difficult, but that has nothing to do with build-time code execution (a common feature of packaging systems) or other package confusion techniques. Go is able to avoid these problems because it (arguably wisely) sidesteps nearly all package index problems by punting them to the source host.
- fiddlerwoaroof 3y ago
  I think maven central has the right idea, but it could be generalized. Require people submitting packages to prove they own a domain and use the domain as the namespace representing the user. We could then use SRV records or TXT records to specify where a domain’s packages are authoritatively hosted and then have conventions to find a signing key for a given domain that can be used to verify cached packages in non-authoritative hosts.
  - woodruffw 3y ago
    Again: namespacing isn't the problem (or a totalizing solution) here. I admire Maven's approach on a conceptual level, but it leaves a lot to be desired on a technical level: it effectively gatekeeps the entire packaging ecosystem on the ability to purchase and maintain a domain name, which (1) is unnecessarily exclusionary, and (2) assumes trust in a namespace (DNS) that was never really designed to be authenticated or a source of stable, permanent identifiers (domains change hands all the times, including for hostile reasons). A modern version of Maven's scheme would bootstrap on well-known URIs over HTTPS instead, but even this would still represent a significant imposition on PyPI uploaders.

adontz 3y ago

PyPI could integrate with Google/Microsoft/Apple as an authentication system (OAuth?). Almost everyone has one of these IDs and it's hard enough to register new ones.

woodruffw 3y ago
> PyPI could integrate with Google/Microsoft/Apple as an authentication system (OAuth?). PyPI supports "trusted publishing,"[1] which provides a variant of this: it doesn't replace a user identity, but instead allows a platform (currently just GitHub, but support for others is on the way) to mint API tokens on a project's behalf. Binding PyPI identities to well-known IdPs would address some of the problems here, but also introduces new ones: it creates a new kind of account lockout state (users who lose access to their IdP service, for whatever reason), introduces regulatory and data collection concerns, may prove excessively restrictive to users in countries with filtered Internet access, etc. [1]: https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/ https://blog.pypi.org/posts/2023-04-20-introducing-trusted-p...
- adontz 3y ago
  And all of these is already a thing for nuget?
vhcr 3y ago
All the spam I receive directly from gmail.com accounts kind of disproves your point.
jjgreen 3y ago
That's what Rust does, you can only register with crates.io with a GitHub account.

zyl1n 3y ago

As someone who straddles both Python and Java world, I don't seem hear this kind of malicious packages problem with Maven (central) as I do with PyPI. What could be the difference? Perhaps Java world is full of old farts like me doing mundane thing with a handful of well established libraries.

elashri 3y ago

I like the simple solution that arXiv use (which allow anyone to publish a pre-print) also it has order of magnitude more code (compared to PyPI packages). If you want to be able to publish for the first time, you have to get a referral from a well established author on the platform. And of course, things might be easier because you need to register using your real identify and that is excepted. This is different for PyPI that until now does not expect people to have their real identifies linked to their accounts.

_8j50 3y ago

I think this is the wrong problem they are addressing. They should allow malicious users to be a possibility but what is lacking is package authentication/signing. Sign all packages, new users get the worst grade, users with a lot of downloads over a long period get a slightly better grade then have different levels if trusted third party reviewed grades. For new users' packages you get a big red warning that needs to be confirmed and bypassed in multiple ways with a message like "ERROR:THIS PACKAGE IS FROM AN ILL-REPUTED SOURCE, IT IS LIKELY MALICIOUS AND HASN'T UNDERGONE HUMAN REVIEW" kind of scream at you a little.

woodruffw 3y ago
Codesigning is something that's actively being worked on for PyPI. It won't have that kind of "noob" reputation scheme, though: those kinds of schemes (1) effectively punish newcomers for daring to contribute to an ecosystem, and (2) are incredibly hard to build trust policies around (what's the trust gradation between "new package" and "very old package with a new maintainer"?).
- _8j50 3y ago
  The grade is for the maintainer not the package. It does punish newcomers but that's the point, newcomers cannot be trusted. People don't search for packages directly on pypi, they find them elsewhere and then use pypi to install it. If you are a new maintainer, you build reputation elsewhere and ask users to ignore the warning on pypi. After say 1k unique downloads and 90 day probation, you move to the next level with a less serious warning. To avoid all that though, you can purchase a code signing certificate and verify your identity (with pypi not just the ca) and even as a new comer there will be no warning for your packages. After observing many such systems my view is that you have to make up your mind and have a rigid and reliable reputation system or have none at all. The middle is the most dangerous place for everyone but I fear open source people like to repeat mistakes and sacrifice the safety of their users for the sake of ideology. Newcomer+ no known reputation+unverified = warn people

lolinder 3y ago

Modern package registries implicitly treat all package authors as equally trustworthy—you don't need to know which organization wrote the `zip` package, you just need to know that there's a package by that name and it opens your zip files. It's charmingly egalitarian and quite convenient, but the global namespace both enables typosquatting and trains developers to not think at all about the people who write the stuff in their supply chain. The biggest point of failure in any system—the humans—has no first-class expression in the typical modern package registry. I can't help but wonder if this is something Maven got right. Namespacing packages according to domain names makes typosquatting vastly more difficult to pull off and makes authorship of dependencies a first-class concern for end users. If something is under org.apache, that means something about the contents of the package, and PyPI and company don't front that information nearly as well as Maven does.

NoboruWataya 3y ago

I know `pip search` has been disabled with PyPI for a while now due to a constant deluge of requests overwhelming the backend. Knowing that, and seeing this update, makes me think that PyPI is subject to an exceptional volume of malicious activity. Does anyone know if it's true that PyPI receives more abuse than similar projects and, if so, why?