4 ms·

I'm very glad we have the wheel and ensurepip now. Yet, I think PyPA has not been taking the best decisions regarding Python packaging. Your Kenneth story is

I'm very glad we have the wheel and ensurepip now. Yet, I think PyPA has not been taking the best decisions regarding Python packaging. Your Kenneth story is not the only "weird event" in their history. E.G: Did you know that we don't need "pyproject.toml" at all ? That there is already a production ready plain text standard to replace setup.py ? Did you know that this standard has been perfectly working for TWO YEARS with regular setuptools, is incredibly simple to use and completly compatible with the standard "setup.py stuff" workflow (and hence the whole legacy tool stack) ? Yep. And nobody talks about it. Let me (re)introduce... Setup.cfg ! Oh, I know... Most people believes it's a useless file. After all, the Python documentation seldom states to use it and for only one tiny option: https://docs.python.org/3.7/distutils/configfile.html https://docs.python.org/3.7/distutils/configfile.html But no. Setup.cfg is awesome !! Put one line in setup.py: import setuptools; setuptools.setup() And you can now put all the rest - yes, everything - in setup.cfg. It's perfectly documented: https://setuptools.readthedocs.io/en/latest/setuptools.html#configuring-setup-using-setup-cfg-files https://setuptools.readthedocs.io/en/latest/setuptools.html#... Not only it has been working since 2016, but it has fantastic goodies: version = attr: src.__version__ will include the version from __init__.py "options.data_files" replaces the MANIFEST "license = file: LICENCE.txt" loads the licence from any text file Try it, it just works. And you can "python setup.py sdist upload" as usual with it. You don't need any new tool. Now why did the PyPA decide to forget about this and create a new format ? The short explanation in PEP 518 is a bad joke. And why does nobody talks about it ? When I asked the PyPA, they told me they were too invested in the new project to stop now. I don't like this answer at all: we suffered enough with python packaging during the "distutils, eggs, etc" fiasco. setup.cfg works. It works now. It's nice. It's compatible. It does what we need. Use it. Talk about it. Write about it. Make sure a lot of people knows so that tool makers and PyPA finally acknowledge that there is not need for the XKCD comics about standard to be true again.

gvalkov 8y ago
> Use it. Talk about it. Write about it. I have a project that converts basic setup.py files to setup.cfg files [1]. Still happily using plain setuptools for library development and pip-tools for application development. [1]: https://github.com/gvalkov/setuptools-py2cfg https://github.com/gvalkov/setuptools-py2cfg
- sametmax 8y ago
  Very cool. I'll share that.
scrollaway 8y ago
I love setup.cfg, I've used it for years and indeed pyproject.toml is useless given that setup.cfg has existed for frickin ever at this point (and is supported by a multitude of tools). TOML is nice but its support isn't even in the stdlib which makes it very awkward to use for a core file like that. Here are some examples of my libs/apps using it in the real world, if someone needs references for how to use setup.cfg with an empty or near-empty setup.py: https://github.com/jleclanche/python-bna/blob/master/setup.cfg https://github.com/jleclanche/python-bna/blob/master/setup.c... https://github.com/dj-stripe/dj-stripe/blob/master/setup.cfg https://github.com/dj-stripe/dj-stripe/blob/master/setup.cfg https://github.com/jazzband/django-oauth-toolkit/blob/master/setup.cfg https://github.com/jazzband/django-oauth-toolkit/blob/master... https://github.com/jazzband/django-push-notifications/blob/master/setup.cfg https://github.com/jazzband/django-push-notifications/blob/m... Edit: I see you mention attr: src.__version__. I personally prefer doing it the other way around, with the version defined in setup.cfg and a pkg_resources snippet in __init__.py (https://github.com/HearthSim/python-hearthstone/blob/master/hearthstone/__init__.py https://github.com/HearthSim/python-hearthstone/blob/master/...). To be honest I wish __version__ were automatically defined like that (but more reliably). Do you know if this was discussed in a PEP?
- sametmax 8y ago
  I like the way you do it too, and I don't think it needs any improvements besides being shared more.
- ATsch 8y ago
  I don't understand the pyproject.toml hate. pyproject.toml exists specifically so you can specify your build system. Without it, you are basically forced to use setuptools/distutils as is currently the case. Hence, pyproject.toml and setup.cfg aren't at all in conflict.
  - scrollaway 8y ago
    They're not in conflict. The problem is pyproject.toml is superfluous. See my comment here: https://news.ycombinator.com/item?id=18614058 https://news.ycombinator.com/item?id=18614058
pseudalopex 8y ago
setup.cfg lets you configure setuptools with declarative syntax. pyproject.toml lets you replace setuptools with something else. The PEP explains why they didn't just reuse setup.cfg.[1] [1] https://www.python.org/dev/peps/pep-0518/#sticking-with-setup-cfg https://www.python.org/dev/peps/pep-0518/#sticking-with-setu...
- scrollaway 8y ago
  The problem is that this is yet another boilerplate file. The reasoning is good, but we were just arriving to the point that every Python tool out there is either compatible with tox.ini, setup.cfg, or both (much like the JS ecosystem has tools reading from package.json). Now we have both Pipfile and pyproject.toml on top of it! For a language that prides itself on its stability and backwards compatibility (especially when compared to the JS ecosystem), we churn through boilerplate files harder than Google churns through instant messaging apps.
- sametmax 8y ago
  > The point of pyproject.toml is to allow other libraries to replace setuptools. This can be done with setup.cfg. Setuptools is only a backend supporting it. You can create other ones. Poetry and pipenv could support it in a week in their authors decided so. > The reasons for not using setup.cfg are explained in the PEP.[1] Those are not reasons, those are excuses. Let me quote it: >> There are two issues with setup.cfg used by setuptools as a general format. One is that they are .ini files which have issues as mentioned in the configparser discussion above. Not only setup.cfg does the job with the current limitations of the ini format (while pyproject.toml still doesn't with its fancy one), but python projects are not so complex they require such a rich content. Besides, nothing prevent PyPA to says that setup.cfg format now has a version header, with the current setup.cfg being headerless version 1, then make the header mandatory for version 2 and increments it to move toward TOML if we ever reach a limitation. That's how formats grow everywhere else in the world. >> The other is that the schema for that file has never been rigorously defined and thus it's unknown which format would be safe to use going forward without potentially confusing setuptools installations. That's incredibly dishonest, since I gave a link to a complete documentation of the format in my previous post. Besides, it's better to actually refine the existing standard if you ever find it lacking than recreating one from scratch. While there are good reasons to do so, the later is rarely a rational engineering decision, and most often driven by ego. >> While keeping with traditional thanks to setup.py, it does not necessarily match what the file may contain in the future So ? How is that a problem ? A standard is not meant to be set in stone. It evolves. But it can't do so if everytime one has an itch, one reinvents the wheel.
IvanAnishchuk 8y ago
This is interesting. Somehow I missed this, using setup.cfg make much more sense than putting everything into python code. I wonder if pip-tools will allow compiling install_requires and test_requires in setup.cfg into nice requirements.txt and dev-requirements.txt without too much magic...
sametmax 8y ago
Because I hate the "this ship has sailed" argument about pyproject.toml, here are some github usage stats: - setup.py: 1,259,007 results (https://github.com/search?q=filename%3Asetup.py https://github.com/search?q=filename%3Asetup.py) - setup.cfg: 165,716 results (https://github.com/search?q=filename%3Asetup.cfg https://github.com/search?q=filename%3Asetup.cfg) - pyproject.toml: 2,137 results (https://github.com/search?q=filename%3Apyproject.toml https://github.com/search?q=filename%3Apyproject.toml) Also, remember that setup.cfg is completly compatible with setup.py, the migration is painless. All the legacy tools work. Not the case with pyproject.toml.
- Kwpolska 8y ago
  setup.cfg isn't always used as a setup.py replacement. I use it for configuring flake8, wheel, pytest and coverage: https://github.com/Kwpolska/python-project-template/blob/master/%7B%7Bcookiecutter.repo_name%7D%7D/setup.cfg https://github.com/Kwpolska/python-project-template/blob/mas...
guggle 8y ago
Your latest blog post say good things about poetry and bad things about pyproject.toml. However the default in poetry seems to be pyproject.toml... I'm confused.
- guggle 8y ago
  oh, nevermind... I'll stick to manual venv/pip: ptest poetry add requests [UnicodeDecodeError] 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
  - sdispater 8y ago
    Author of Poetry here! I've never seen that error before. Which version of Python do you use? And feel free to create an issue on the issue tracker: https://github.com/sdispater/poetry/issues https://github.com/sdispater/poetry/issues
    - sametmax 8y ago
      pyproject debates aside, I love your work. I think poetry is a beautiful piece of software, the source code is very easy to read and the 2/3 compat is well done (although I would not use assert to check things).
  - agateau 8y ago
    Looks like a locale problem. `locale-gen en_US.UTF-8` should fix it.
- sametmax 8y ago
  Because I'm not dogmatic, I can regognize good tools, even if I disagree with the underlying political decision. Computing is not black and white, and perfect purity is only nice in "fizz buzz". Now to be extra fun, poetry uses a custom section ([tool.poetry]) in pyproject.toml, not really the standard itself. What does that say about this format ?

gvalkov 8y ago

> Use it. Talk about it. Write about it. I have a project that converts basic setup.py files to setup.cfg files [1]. Still happily using plain setuptools for library development and pip-tools for application development. [1]: https://github.com/gvalkov/setuptools-py2cfg https://github.com/gvalkov/setuptools-py2cfg

sametmax 8y ago
Very cool. I'll share that.

scrollaway 8y ago

I love setup.cfg, I've used it for years and indeed pyproject.toml is useless given that setup.cfg has existed for frickin ever at this point (and is supported by a multitude of tools). TOML is nice but its support isn't even in the stdlib which makes it very awkward to use for a core file like that. Here are some examples of my libs/apps using it in the real world, if someone needs references for how to use setup.cfg with an empty or near-empty setup.py: https://github.com/jleclanche/python-bna/blob/master/setup.cfg https://github.com/jleclanche/python-bna/blob/master/setup.c... https://github.com/dj-stripe/dj-stripe/blob/master/setup.cfg https://github.com/dj-stripe/dj-stripe/blob/master/setup.cfg https://github.com/jazzband/django-oauth-toolkit/blob/master/setup.cfg https://github.com/jazzband/django-oauth-toolkit/blob/master... https://github.com/jazzband/django-push-notifications/blob/master/setup.cfg https://github.com/jazzband/django-push-notifications/blob/m... Edit: I see you mention attr: src.__version__. I personally prefer doing it the other way around, with the version defined in setup.cfg and a pkg_resources snippet in __init__.py (https://github.com/HearthSim/python-hearthstone/blob/master/hearthstone/__init__.py https://github.com/HearthSim/python-hearthstone/blob/master/...). To be honest I wish __version__ were automatically defined like that (but more reliably). Do you know if this was discussed in a PEP?

sametmax 8y ago
I like the way you do it too, and I don't think it needs any improvements besides being shared more.
ATsch 8y ago
I don't understand the pyproject.toml hate. pyproject.toml exists specifically so you can specify your build system. Without it, you are basically forced to use setuptools/distutils as is currently the case. Hence, pyproject.toml and setup.cfg aren't at all in conflict.
- scrollaway 8y ago
  They're not in conflict. The problem is pyproject.toml is superfluous. See my comment here: https://news.ycombinator.com/item?id=18614058 https://news.ycombinator.com/item?id=18614058

pseudalopex 8y ago

setup.cfg lets you configure setuptools with declarative syntax. pyproject.toml lets you replace setuptools with something else. The PEP explains why they didn't just reuse setup.cfg.[1] [1] https://www.python.org/dev/peps/pep-0518/#sticking-with-setup-cfg https://www.python.org/dev/peps/pep-0518/#sticking-with-setu...

scrollaway 8y ago
The problem is that this is yet another boilerplate file. The reasoning is good, but we were just arriving to the point that every Python tool out there is either compatible with tox.ini, setup.cfg, or both (much like the JS ecosystem has tools reading from package.json). Now we have both Pipfile and pyproject.toml on top of it! For a language that prides itself on its stability and backwards compatibility (especially when compared to the JS ecosystem), we churn through boilerplate files harder than Google churns through instant messaging apps.
sametmax 8y ago
> The point of pyproject.toml is to allow other libraries to replace setuptools. This can be done with setup.cfg. Setuptools is only a backend supporting it. You can create other ones. Poetry and pipenv could support it in a week in their authors decided so. > The reasons for not using setup.cfg are explained in the PEP.[1] Those are not reasons, those are excuses. Let me quote it: >> There are two issues with setup.cfg used by setuptools as a general format. One is that they are .ini files which have issues as mentioned in the configparser discussion above. Not only setup.cfg does the job with the current limitations of the ini format (while pyproject.toml still doesn't with its fancy one), but python projects are not so complex they require such a rich content. Besides, nothing prevent PyPA to says that setup.cfg format now has a version header, with the current setup.cfg being headerless version 1, then make the header mandatory for version 2 and increments it to move toward TOML if we ever reach a limitation. That's how formats grow everywhere else in the world. >> The other is that the schema for that file has never been rigorously defined and thus it's unknown which format would be safe to use going forward without potentially confusing setuptools installations. That's incredibly dishonest, since I gave a link to a complete documentation of the format in my previous post. Besides, it's better to actually refine the existing standard if you ever find it lacking than recreating one from scratch. While there are good reasons to do so, the later is rarely a rational engineering decision, and most often driven by ego. >> While keeping with traditional thanks to setup.py, it does not necessarily match what the file may contain in the future So ? How is that a problem ? A standard is not meant to be set in stone. It evolves. But it can't do so if everytime one has an itch, one reinvents the wheel.

IvanAnishchuk 8y ago

This is interesting. Somehow I missed this, using setup.cfg make much more sense than putting everything into python code. I wonder if pip-tools will allow compiling install_requires and test_requires in setup.cfg into nice requirements.txt and dev-requirements.txt without too much magic...

sametmax 8y ago

Because I hate the "this ship has sailed" argument about pyproject.toml, here are some github usage stats: - setup.py: 1,259,007 results (https://github.com/search?q=filename%3Asetup.py https://github.com/search?q=filename%3Asetup.py) - setup.cfg: 165,716 results (https://github.com/search?q=filename%3Asetup.cfg https://github.com/search?q=filename%3Asetup.cfg) - pyproject.toml: 2,137 results (https://github.com/search?q=filename%3Apyproject.toml https://github.com/search?q=filename%3Apyproject.toml) Also, remember that setup.cfg is completly compatible with setup.py, the migration is painless. All the legacy tools work. Not the case with pyproject.toml.

Kwpolska 8y ago
setup.cfg isn't always used as a setup.py replacement. I use it for configuring flake8, wheel, pytest and coverage: https://github.com/Kwpolska/python-project-template/blob/master/%7B%7Bcookiecutter.repo_name%7D%7D/setup.cfg https://github.com/Kwpolska/python-project-template/blob/mas...

guggle 8y ago

Your latest blog post say good things about poetry and bad things about pyproject.toml. However the default in poetry seems to be pyproject.toml... I'm confused.

guggle 8y ago
oh, nevermind... I'll stick to manual venv/pip: ptest poetry add requests [UnicodeDecodeError] 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
- sdispater 8y ago
  Author of Poetry here! I've never seen that error before. Which version of Python do you use? And feel free to create an issue on the issue tracker: https://github.com/sdispater/poetry/issues https://github.com/sdispater/poetry/issues
  - sametmax 8y ago
    pyproject debates aside, I love your work. I think poetry is a beautiful piece of software, the source code is very easy to read and the 2/3 compat is well done (although I would not use assert to check things).
- agateau 8y ago
  Looks like a locale problem. `locale-gen en_US.UTF-8` should fix it.
sametmax 8y ago
Because I'm not dogmatic, I can regognize good tools, even if I disagree with the underlying political decision. Computing is not black and white, and perfect purity is only nice in "fizz buzz". Now to be extra fun, poetry uses a custom section ([tool.poetry]) in pyproject.toml, not really the standard itself. What does that say about this format ?