4 ms·

I have been using purely setuptools for all of our open source Python libraries at Contentful, but have found that lately I've been getting deprecation warnings

I have been using purely setuptools for all of our open source Python libraries at Contentful, but have found that lately I've been getting deprecation warnings from PyPI not to use `setup.py upload` anymore. What should the alternative be now? Edit: I'm reading about twine right now, but I cannot begin to comprehend why it's not bundled directly if this is what they are intending for us to use to upload packages.

Bogdanp 8y ago
Twine seems to be the recommended[1] way. It's pretty straightforward to use, thankfully. [1]: https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives https://packaging.python.org/tutorials/packaging-projects/#u...
deleted 8y ago
[deleted]
toyg 8y ago
Anything PyPI-related has recently gone into the (terrible) habit of recommending very recent (and often half-baked) tools that live entirely outside of stdlib. It seems pretty silly to me, considering Python core developers made significant efforts to bundle and support pip and virtualenv (venv) in the stdlib precisely to avoid having a lot of de-facto essential libraries outside the core distribution. If the problem is that stdlib cannot move as fast as PyPI-related development requires, maybe that should be fixed, rather than trying to bypass all quality checks and then relying on obscure shared knowledge to navigate the ecosystem. Maybe there should be a system where specific network-sensitive stdlib modules could be updated faster than the rest.
- di 8y ago
  You're mostly right, the problem is also that users don't upgrade their Python distribution very often, so they miss out on new features. > Maybe there should be a system where specific network-sensitive stdlib modules could be updated faster than the rest. This is essentially what `setuptools` does, by putting a package on PyPI that monkeypatches/plugs in to the stdlib.
di 8y ago
Hello, I'm the person who deprecated `setup.py upload`. The warnings should be telling you that `twine` is the preferred tool for uploading. The reason for this is that right now, that command comes from `distutils`, which is part of the standard library. There is a huge disadvantage to bundling this functionality with your Python distribution, namely that it can only get upgraded when you upgrade your Python distribution. A lot of folks are still running versions of Python from several years ago, which is fine, but it means that they are missing out on anything new that's been added in the meantime. For example, earlier this year, we released a new package metadata version which allows people to specify their package descriptions with Markdown. This required a new metadata field, which old versions of `distutils` know nothing about. Upgrading `distutils` to support it would require that these changes go though the long process of making it into a Python release, and even then they would only be available to folks using the latest release. Moving this functionality from `distutils` to a tool like `twine` means that new features can be made available nearly immediately (just have to make a release to PyPI) and that they're available to users on any Python distribution (just have to upgrade from PyPI). The `distutils` standard library module comes from a time when we didn't have PyPI and thus, didn't have a better way to distribute this code to users. We have PyPI now though, so bundling `distutils` with Python is becoming less and less useful.
- 1wd 8y ago
  Why not bundle twine like pip? In fact, why not merge the twine functionality into pip?
  - di 8y ago
    > Why not bundle twine like pip? The `pip` package is not actually bundled with your Python distribution, instead the standard library has `ensurepip` which provides a means of bootstrapping a `pip` installation without `pip` itself. See [0]. > In fact, why not merge the twine functionality into pip? This has been considered and still might happen, see [1], specifically the comment at [2]. [0] https://docs.python.org/3/library/ensurepip.html https://docs.python.org/3/library/ensurepip.html [1] https://github.com/pypa/packaging-problems/issues/60 https://github.com/pypa/packaging-problems/issues/60 [2] https://github.com/pypa/packaging-problems/issues/60#issuecomment-369238296 https://github.com/pypa/packaging-problems/issues/60#issueco...

Bogdanp 8y ago

Twine seems to be the recommended[1] way. It's pretty straightforward to use, thankfully. [1]: https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives https://packaging.python.org/tutorials/packaging-projects/#u...

deleted 8y ago

[deleted]

toyg 8y ago

Anything PyPI-related has recently gone into the (terrible) habit of recommending very recent (and often half-baked) tools that live entirely outside of stdlib. It seems pretty silly to me, considering Python core developers made significant efforts to bundle and support pip and virtualenv (venv) in the stdlib precisely to avoid having a lot of de-facto essential libraries outside the core distribution. If the problem is that stdlib cannot move as fast as PyPI-related development requires, maybe that should be fixed, rather than trying to bypass all quality checks and then relying on obscure shared knowledge to navigate the ecosystem. Maybe there should be a system where specific network-sensitive stdlib modules could be updated faster than the rest.

di 8y ago
You're mostly right, the problem is also that users don't upgrade their Python distribution very often, so they miss out on new features. > Maybe there should be a system where specific network-sensitive stdlib modules could be updated faster than the rest. This is essentially what `setuptools` does, by putting a package on PyPI that monkeypatches/plugs in to the stdlib.

di 8y ago

Hello, I'm the person who deprecated `setup.py upload`. The warnings should be telling you that `twine` is the preferred tool for uploading. The reason for this is that right now, that command comes from `distutils`, which is part of the standard library. There is a huge disadvantage to bundling this functionality with your Python distribution, namely that it can only get upgraded when you upgrade your Python distribution. A lot of folks are still running versions of Python from several years ago, which is fine, but it means that they are missing out on anything new that's been added in the meantime. For example, earlier this year, we released a new package metadata version which allows people to specify their package descriptions with Markdown. This required a new metadata field, which old versions of `distutils` know nothing about. Upgrading `distutils` to support it would require that these changes go though the long process of making it into a Python release, and even then they would only be available to folks using the latest release. Moving this functionality from `distutils` to a tool like `twine` means that new features can be made available nearly immediately (just have to make a release to PyPI) and that they're available to users on any Python distribution (just have to upgrade from PyPI). The `distutils` standard library module comes from a time when we didn't have PyPI and thus, didn't have a better way to distribute this code to users. We have PyPI now though, so bundling `distutils` with Python is becoming less and less useful.

1wd 8y ago
Why not bundle twine like pip? In fact, why not merge the twine functionality into pip?
- di 8y ago
  > Why not bundle twine like pip? The `pip` package is not actually bundled with your Python distribution, instead the standard library has `ensurepip` which provides a means of bootstrapping a `pip` installation without `pip` itself. See [0]. > In fact, why not merge the twine functionality into pip? This has been considered and still might happen, see [1], specifically the comment at [2]. [0] https://docs.python.org/3/library/ensurepip.html https://docs.python.org/3/library/ensurepip.html [1] https://github.com/pypa/packaging-problems/issues/60 https://github.com/pypa/packaging-problems/issues/60 [2] https://github.com/pypa/packaging-problems/issues/60#issuecomment-369238296 https://github.com/pypa/packaging-problems/issues/60#issueco...