7 ms·

I've been planning on packaging a python package recently, and the internet is annoyingly full of guides which are, I think, out of date. They at least suggest

I've been planning on packaging a python package recently, and the internet is annoyingly full of guides which are, I think, out of date. They at least suggest quite different things. I just have a single python file, meant to be treated as an executable (no package at present). There are a whole bunch of tests, but that's obviously separate. Any suggestions on modern best practices welcome!

woodruffw 4y ago
If it's pure Python, the only packaging file you need is `pyproject.toml`. You can fill that file with packaging metadata per PEP 518 and PEP 621, including using modern build tooling like flit[1] for the build backend and build[2] for the frontend. With that, you entire package build (for all distribution types) should be reducible to `python -m build`. Here's an example of a full project doing everything with just `pyproject.toml`[3] (FD: my project). [1]: https://github.com/pypa/flit https://github.com/pypa/flit [2]: https://github.com/pypa/build https://github.com/pypa/build [3]: https://github.com/pypa/pip-audit https://github.com/pypa/pip-audit
- quietbritishjim 4y ago
  > including using modern build tooling like flit[1] for the build backend and build[2] for the frontend. Or you can use setuptools, which is the package that enables old setup.py builds, as the backend with pyproject.toml. This has the advantage of being mature, unlikely to be abandoned, and possibly some familiarity if you've used it before. Even then, you can use build as the front end build tool.
  - woodruffw 4y ago
    Yes, setuptools is also perfectly fine. It had some rough edges around PEP 621 support for a while, but those were mostly smoothed out in 2021. (I'll note that maturity is not strong evidence here: distutils is very mature, but is in the process of being deprecated and removed from Python entirely. I don't think that's likely to happen to setuptools, but the fact that behavioral PEPs now exist for all of these tools means that the decline/abandonment of any poses much less of an ecosystem risk.)
    - quietbritishjim 4y ago
      I suppose I meant maturity as in: actively maintained and recommended for use for a long period, which doesn't apply to distutils. And I should've been more upfront about the real reason for suggesting setuptools: there seem to be a number of build tools that support pyproject.toml, including flit, poetry and setuptools (and I'm sure I've seen at least one other). For me, at least, when I was making a small library recently, it was an overwhelming choice for a part of my project that feels like just admin rather than core business logic. I came close to giving up and just using setup.py with `setup()`. At least setuptools with pyproject.toml is a choice that feels safe; it may not be the best, but it will certainly be good enough that I'm unlikely to regret it later, so I didn't need to spend a lot of time looking at the detailed pros and cons of all the choices.
      - woodruffw 4y ago
        That's very reasonable! I don't mean to disparage that decision at all: setuptools is rock solid and a very safe choice.
- kevin_thibedeau 4y ago
  Good luck finding clear documentation on how to write the pyproject file. For something that is supposed to be the way forward there hasn't been much effort to make it easy to implement. There's also annoyances like the inability to install a script in the search path without implementing it as a module. Something setup.py doesn't require.
  - woodruffw 4y ago
    > Good luck finding clear documentation on how to write the pyproject file. For something that is supposed to be the way forward there hasn't been much effort to make it easy to implement. PEP 621, which I mentioned, covers the format of `pyproject.toml` in detail. I also linked an example which, to the best of my knowledge, covers all current best practices for that file. > There's also annoyances like the inability to install a script in the search path without implementing it as a module. Something setup.py doesn't require. I'm not sure I'm following. A module in Python is just a Python file, so your script is a module. Are you saying that you can't distribute single-module packages with pyproject.toml? Because I don't think that's true.
    - kevin_thibedeau 4y ago
      The PEP in no way explains how to write a usable pyproject for ordinary projects. It's basically just targeted at people developing installers. I meant package. A directory with an __init__.py. You can't install standalone script.py (or a generated wrapper) as /usr/local/bin/script with a pyproject.
      - woodruffw 4y ago
        > The PEP in no way explains how to write a usable pyproject for ordinary projects. It's basically just targeted at people developing installers. Did you look at it[1]? > I meant package. A directory with an __init__.py. You can't install standalone script.py (or a generated wrapper) as /usr/local/bin/script with a pyproject. I still don't think I understand what your expectation is here: a `pyproject.toml` is just a metadata specification. The only difference between it and `setup.py` is that the latter is arbitrary code. There's an old, long deprecated way to use `setup.py`, namely `setup.py install`. But that's been discouraged in favor of `pip install` for years, which behaves precisely the same way with `pyproject.toml`. If you want to install a script into `/usr/local/bin`, `pip install` with a package specified in `pyproject.toml` will work just fine. [1]: https://peps.python.org/pep-0621/#example https://peps.python.org/pep-0621/#example
ensignavenger 4y ago
If you are wanting to release it to pypi as a python package, I would personally use Poetry. But your case- a single pure Python package, is a simple case that won't have many problems like are brought up in the article, whatever tool you use. If you want a stand alone executable, I haven't found a good, single, cross platform tool for that yet... seems like there is a separate tool for each platform.
- quietbritishjim 4y ago
  Nuitka works on Windows, Linux and Mac https://nuitka.net/ https://nuitka.net/
- selcuka 4y ago
  > If you want a stand alone executable, I haven't found a good, single, cross platform tool for that yet. PyInstaller is cross platform, and arguably good.
mixmastamyk 4y ago
Keep it simple and fashion-proof. Been using setup.py for a one-script package for one or two? decades: from setuptools import setup setup( name = 'foobar', scripts = ['foo'], # install a script from current fldr # ... ) A few years ago I had to start using twine to register and upload it to pypi.
- tomrod 4y ago
  Is there a place you recommend where I can learn more about this?
  - mixmastamyk 4y ago
    It’s documented, try google, stackoverflow, and reading other packages’ setup.py for tricks. However the scripts=[], keyword is the key to the case above.
- talideon 4y ago
  I wouldn't recommend that at all these days. setup.py definitely is not future-proof: it's undergoing deprecation. Better would be a basic pyproject.toml file along the lines of the following: [build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" [project] name = "foobar" version = "0.0.1" dependencies = [ "...", ] [project.scripts] foo = "foobar:main" See: https://setuptools.pypa.io/en/latest/userguide/quickstart.html https://setuptools.pypa.io/en/latest/userguide/quickstart.ht...
  - eesmith 4y ago
    > it's undergoing deprecation In the "we would rather people not use this but it's going to stay around for a long time" sense. I strongly doubt it will disappear within the next decade or two. There's a long tail of setup.py-based tools. Last I checked pyproject.toml only supports the simplest of Python/C extensions. Anything fancy, like --with/--without compilation flags to enable/disable optional support, compiler-specific flags (in my case, to add OpenMP), compile-time code generation (like using yacc/lex), etc. requires a setup.py and a bunch of hacking.
japanuspus 4y ago
As detailed in the other answers, there are two parts to this: 1) Creating a python package from your project (and possibly share this on pypi), and 2) Making this package available as an end-user application. For step 2 you can use nuitka or similar, but if your audience is somewhat developer-oriented, you can also propose for them to use pipx: https://github.com/pypa/pipx https://github.com/pypa/pipx.
tpoacher 4y ago
The approach I prefer is to not mess with setuptools etc at all in the first place, and simply make a nice executable package. e.g. https://github.com/tpapastylianou/self-contained-runnable-python-package-template https://github.com/tpapastylianou/self-contained-runnable-py...

woodruffw 4y ago

If it's pure Python, the only packaging file you need is `pyproject.toml`. You can fill that file with packaging metadata per PEP 518 and PEP 621, including using modern build tooling like flit[1] for the build backend and build[2] for the frontend. With that, you entire package build (for all distribution types) should be reducible to `python -m build`. Here's an example of a full project doing everything with just `pyproject.toml`[3] (FD: my project). [1]: https://github.com/pypa/flit https://github.com/pypa/flit [2]: https://github.com/pypa/build https://github.com/pypa/build [3]: https://github.com/pypa/pip-audit https://github.com/pypa/pip-audit

quietbritishjim 4y ago
> including using modern build tooling like flit[1] for the build backend and build[2] for the frontend. Or you can use setuptools, which is the package that enables old setup.py builds, as the backend with pyproject.toml. This has the advantage of being mature, unlikely to be abandoned, and possibly some familiarity if you've used it before. Even then, you can use build as the front end build tool.
- woodruffw 4y ago
  Yes, setuptools is also perfectly fine. It had some rough edges around PEP 621 support for a while, but those were mostly smoothed out in 2021. (I'll note that maturity is not strong evidence here: distutils is very mature, but is in the process of being deprecated and removed from Python entirely. I don't think that's likely to happen to setuptools, but the fact that behavioral PEPs now exist for all of these tools means that the decline/abandonment of any poses much less of an ecosystem risk.)
  - quietbritishjim 4y ago
    I suppose I meant maturity as in: actively maintained and recommended for use for a long period, which doesn't apply to distutils. And I should've been more upfront about the real reason for suggesting setuptools: there seem to be a number of build tools that support pyproject.toml, including flit, poetry and setuptools (and I'm sure I've seen at least one other). For me, at least, when I was making a small library recently, it was an overwhelming choice for a part of my project that feels like just admin rather than core business logic. I came close to giving up and just using setup.py with `setup()`. At least setuptools with pyproject.toml is a choice that feels safe; it may not be the best, but it will certainly be good enough that I'm unlikely to regret it later, so I didn't need to spend a lot of time looking at the detailed pros and cons of all the choices.
    - woodruffw 4y ago
      That's very reasonable! I don't mean to disparage that decision at all: setuptools is rock solid and a very safe choice.
kevin_thibedeau 4y ago
Good luck finding clear documentation on how to write the pyproject file. For something that is supposed to be the way forward there hasn't been much effort to make it easy to implement. There's also annoyances like the inability to install a script in the search path without implementing it as a module. Something setup.py doesn't require.
- woodruffw 4y ago
  > Good luck finding clear documentation on how to write the pyproject file. For something that is supposed to be the way forward there hasn't been much effort to make it easy to implement. PEP 621, which I mentioned, covers the format of `pyproject.toml` in detail. I also linked an example which, to the best of my knowledge, covers all current best practices for that file. > There's also annoyances like the inability to install a script in the search path without implementing it as a module. Something setup.py doesn't require. I'm not sure I'm following. A module in Python is just a Python file, so your script is a module. Are you saying that you can't distribute single-module packages with pyproject.toml? Because I don't think that's true.
  - kevin_thibedeau 4y ago
    The PEP in no way explains how to write a usable pyproject for ordinary projects. It's basically just targeted at people developing installers. I meant package. A directory with an __init__.py. You can't install standalone script.py (or a generated wrapper) as /usr/local/bin/script with a pyproject.
    - woodruffw 4y ago
      > The PEP in no way explains how to write a usable pyproject for ordinary projects. It's basically just targeted at people developing installers. Did you look at it[1]? > I meant package. A directory with an __init__.py. You can't install standalone script.py (or a generated wrapper) as /usr/local/bin/script with a pyproject. I still don't think I understand what your expectation is here: a `pyproject.toml` is just a metadata specification. The only difference between it and `setup.py` is that the latter is arbitrary code. There's an old, long deprecated way to use `setup.py`, namely `setup.py install`. But that's been discouraged in favor of `pip install` for years, which behaves precisely the same way with `pyproject.toml`. If you want to install a script into `/usr/local/bin`, `pip install` with a package specified in `pyproject.toml` will work just fine. [1]: https://peps.python.org/pep-0621/#example https://peps.python.org/pep-0621/#example

ensignavenger 4y ago

If you are wanting to release it to pypi as a python package, I would personally use Poetry. But your case- a single pure Python package, is a simple case that won't have many problems like are brought up in the article, whatever tool you use. If you want a stand alone executable, I haven't found a good, single, cross platform tool for that yet... seems like there is a separate tool for each platform.

quietbritishjim 4y ago
Nuitka works on Windows, Linux and Mac https://nuitka.net/ https://nuitka.net/
selcuka 4y ago
> If you want a stand alone executable, I haven't found a good, single, cross platform tool for that yet. PyInstaller is cross platform, and arguably good.

mixmastamyk 4y ago

Keep it simple and fashion-proof. Been using setup.py for a one-script package for one or two? decades: from setuptools import setup setup( name = 'foobar', scripts = ['foo'], # install a script from current fldr # ... ) A few years ago I had to start using twine to register and upload it to pypi.

tomrod 4y ago
Is there a place you recommend where I can learn more about this?
- mixmastamyk 4y ago
  It’s documented, try google, stackoverflow, and reading other packages’ setup.py for tricks. However the scripts=[], keyword is the key to the case above.
talideon 4y ago
I wouldn't recommend that at all these days. setup.py definitely is not future-proof: it's undergoing deprecation. Better would be a basic pyproject.toml file along the lines of the following: [build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" [project] name = "foobar" version = "0.0.1" dependencies = [ "...", ] [project.scripts] foo = "foobar:main" See: https://setuptools.pypa.io/en/latest/userguide/quickstart.html https://setuptools.pypa.io/en/latest/userguide/quickstart.ht...
- eesmith 4y ago
  > it's undergoing deprecation In the "we would rather people not use this but it's going to stay around for a long time" sense. I strongly doubt it will disappear within the next decade or two. There's a long tail of setup.py-based tools. Last I checked pyproject.toml only supports the simplest of Python/C extensions. Anything fancy, like --with/--without compilation flags to enable/disable optional support, compiler-specific flags (in my case, to add OpenMP), compile-time code generation (like using yacc/lex), etc. requires a setup.py and a bunch of hacking.

japanuspus 4y ago

As detailed in the other answers, there are two parts to this: 1) Creating a python package from your project (and possibly share this on pypi), and 2) Making this package available as an end-user application. For step 2 you can use nuitka or similar, but if your audience is somewhat developer-oriented, you can also propose for them to use pipx: https://github.com/pypa/pipx https://github.com/pypa/pipx.

tpoacher 4y ago

The approach I prefer is to not mess with setuptools etc at all in the first place, and simply make a nice executable package. e.g. https://github.com/tpapastylianou/self-contained-runnable-python-package-template https://github.com/tpapastylianou/self-contained-runnable-py...