6 ms·

Did you ever try poetry[1]? It is a great tool that I use in all my projects. It effectively solves dependency management for me in a very easy to use way. De

Did you ever try poetry[1]? It is a great tool that I use in all my projects. It effectively solves dependency management for me in a very easy to use way. Dependency management using requirements.txt used to give me such a headache, now I just have a pyproject.toml that I know works. [tool.poetry.dependencies] python = ">=3.8,<3.9" pandas = "^1.4.3" This basically means: Use the version of python between 3.8 and 3.9 and use any version higher than 1.4.3 for pandas. What I like about poetry is that it makes sure that the whole dependency graph of the packages that you add is correct. If it can not solve the graph, then it fails fast and it fails hard, which is a good thing. This is probably a very bad explainer of what poetry does, but be sure to check it out! :) [1] https://python-poetry.org/ https://python-poetry.org/

frafra 4y ago
I tried, thanks! :) And I also suggest to evaluate PDM https://pdm.fming.dev/ https://pdm.fming.dev/
- IanCal 4y ago
  I've been using PDM recently and although there have been a few issues I really like just cd'ing into a directory, running "python" and the correct set of packages being available.
  - wyuenho 4y ago
    Take a look at this: https://direnv.net/man/direnv-stdlib.1.html#codelayout-python-ltpythonexegtcode https://direnv.net/man/direnv-stdlib.1.html#codelayout-pytho...
plonk 4y ago
Isn't that the same as adding version constraints to setuptools' setup.cfg? [1] pip will use these in its dependency resolver. You can also constrain the Python version in there if you want. [1] https://setuptools.pypa.io/en/latest/userguide/quickstart.html#dependency-management https://setuptools.pypa.io/en/latest/userguide/quickstart.ht...
- zelphirkalt 4y ago
  Maybe it is the same, but the idea is to take that selected version of the dependency and store a hash checksum for it, so that one can later get the exact same dependencies. Poetry only does not source setup.cfg, but its tool-specific config file. This way it stays out of the way of any other tool.
  - orzig 4y ago
    pip-tools can hash as well: https://pip-tools.readthedocs.io/en/latest/#using-hashes https://pip-tools.readthedocs.io/en/latest/#using-hashes Using a tool specific config file seems like a design choice with upsides and downsides, which I respect
    - westurner 4y ago
      A Pipfile can store hashes for multiple versions of a package built for multiple architectures; whereas requirements.txt can only store the hash of one version of the package on one platform. Can a requirements.txt or a Pipfile store cryptographically-signed hashes for each dependency? Which tool would check that not PyPI-upload but package-builder-signing keys validate? FWIU, nobody ever added GPG .asc signature support to Pip? What keys would it trust for which package? Should twine download after upload and check the publisher and PyPI-upload signatures?
      - zelphirkalt 4y ago
        Are cryptographically-signed hashes really necessary in this case (why?)? What would be the secret, which is used for signing?
        westurner 4y ago
        If the hashes are retrieved over the same channel as the package (i.e. HTTPS), and that channel is unfortunately compromised, why wouldn't a MITM tool change those software package artifact hash checksums too? Only if the key used to sign the package / package manifest with per-file hashes is or was retrieved over a different channel (i.e. WKD, HKP (HTTPS w/w/o Certificate Pinning (*))), and the key is trusted to sign for that package, then install the software package artifact and assign file permissions and extended filesystem attributes.
crucialfelix 4y ago
Hatch is also interesting and very similar to Poetry. https://hatch.pypa.io/latest/ https://hatch.pypa.io/latest/ In comparison to poetry I think it includes more advanced multi-environment and multi-python-version support and a tox-like testing matrix. It probably gets a little too complex there. It also works with pyproject.toml If anyone else has experience with Hatch vs Poetry please share!
- wyuenho 4y ago
  You can't lock your dependencies with Hatch. https://hatch.pypa.io/latest/meta/faq/#libraries-vs-applications https://hatch.pypa.io/latest/meta/faq/#libraries-vs-applicat...
wyuenho 4y ago
Poetry is good, I'll hit a bug in poetry from time to time but it's improving quickly. Direnv is better, direnv layout will automatically create a hidden virtualenv directory on your work tree and sets up your path when you cd into it. The only downside is it doesn't seem to work on Windows.
- athorax 4y ago
  I don't see the link between poetry and direnv? Poetry is about solving python's dependency issues, direnv doesn't seem to have anything to do with that
  - wyuenho 4y ago
    Set up direnv, and then: cd /your/project && echo "layout python" > .envrc && direnv allow Give it a go then tell me what the point is for any of these poetry/pipenv/hatch/flit/pdm/pyflow thingy if neither you or your teammates work on Windows.
    - barefeg 4y ago
      When your project depends on a module version and that module depends on another one (sub dependency), it’s very common that re-installing the same module version in a new environment will cause something to break because the sub dependency was updated. This is not something that direnv solves therefore those other tools are still needed.
      - wyuenho 4y ago
        pip freeze > requirements.txt
        barefeg 4y ago
        Then you lose track of which are actual dependencies and which are sub dependencies
datatrashfire 4y ago
My least favorite part about poetry is how slow it is.
- Tainnor 4y ago
  I seem to remember that some part of poetry's slowness is due to how the python index works and is therefore a problem shared by all such tools. That said, I've used both pipenv and poetry, and I had projects where pipenv would simply time out when trying to resolve packages. I haven't seen the same behaviour with poetry (indeed, that was the reason I migrated one project from pipenv to poetry after I just had to give up with the former).
ak217 4y ago
Poetry has a major issue with its lockfiles when working with active projects. It generates a top level dependency listing checksum, which causes any two PRs/branches that independently update the top level requirements to conflict with each other. The related issue, https://github.com/python-poetry/poetry/issues/496 https://github.com/python-poetry/poetry/issues/496, has been open for 4 years with no movement. The other issue with Poetry is that it uses its own pyproject.toml dependency listing format instead of the one standardized in the relevant PEP (https://peps.python.org/pep-0621/ https://peps.python.org/pep-0621/). This is understandable for historical reasons (Poetry was first written before this was standardized), but Poetry should have been updated to support the standard format. A relatively minor issue, but the poetry shell command is also a footgun. It's presented as a way to configure your shell to activate the virtualenv for the project. In reality it's a very slow, barely functional terminal emulator running on top of your terminal, which will cause problems for any programs that assume a working terminal or talk to the tty directly.
- wyuenho 4y ago
  https://twitter.com/SDisPater/status/1521932867214921728 https://twitter.com/SDisPater/status/1521932867214921728 poetry shell is also not a term emulator, it's just a subshell with some environment variables setup for your project. Once you are in, it's just a regular shell. If anything is slow, it's where you add or remove a dependency, but it's probably faster than you editing requirements.txt, clearing out your virtualenv and then reinstalling everything again.
  - ak217 4y ago
    https://github.com/python-poetry/poetry/blob/master/src/poetry/utils/shell.py https://github.com/python-poetry/poetry/blob/master/src/poet... The process spawned by `poetry shell` is a terminal emulator driven by the pexpect and cleo packages. It hijacks and proxies the user's keystrokes before sending them to the underlying terminal.
    - wyuenho 4y ago
      Cleo creates "subcommands" in a git like manner, whereas pexpect spawns a subshell. That cleo Terminal class you see is only a viewport. A terminal emulator will be something substantially more complex such as libvterm. If it ain't handling terminfo, it ain't a terminal emulator. https://launchpad.net/libvterm https://launchpad.net/libvterm
Doxin 4y ago
> What I like about poetry is that it makes sure that the whole dependency graph of the packages that you add is correct. As does pip these days.

frafra 4y ago

I tried, thanks! :) And I also suggest to evaluate PDM https://pdm.fming.dev/ https://pdm.fming.dev/

IanCal 4y ago
I've been using PDM recently and although there have been a few issues I really like just cd'ing into a directory, running "python" and the correct set of packages being available.
- wyuenho 4y ago
  Take a look at this: https://direnv.net/man/direnv-stdlib.1.html#codelayout-python-ltpythonexegtcode https://direnv.net/man/direnv-stdlib.1.html#codelayout-pytho...

plonk 4y ago

Isn't that the same as adding version constraints to setuptools' setup.cfg? [1] pip will use these in its dependency resolver. You can also constrain the Python version in there if you want. [1] https://setuptools.pypa.io/en/latest/userguide/quickstart.html#dependency-management https://setuptools.pypa.io/en/latest/userguide/quickstart.ht...

zelphirkalt 4y ago
Maybe it is the same, but the idea is to take that selected version of the dependency and store a hash checksum for it, so that one can later get the exact same dependencies. Poetry only does not source setup.cfg, but its tool-specific config file. This way it stays out of the way of any other tool.
- orzig 4y ago
  pip-tools can hash as well: https://pip-tools.readthedocs.io/en/latest/#using-hashes https://pip-tools.readthedocs.io/en/latest/#using-hashes Using a tool specific config file seems like a design choice with upsides and downsides, which I respect
  - westurner 4y ago
    A Pipfile can store hashes for multiple versions of a package built for multiple architectures; whereas requirements.txt can only store the hash of one version of the package on one platform. Can a requirements.txt or a Pipfile store cryptographically-signed hashes for each dependency? Which tool would check that not PyPI-upload but package-builder-signing keys validate? FWIU, nobody ever added GPG .asc signature support to Pip? What keys would it trust for which package? Should twine download after upload and check the publisher and PyPI-upload signatures?
    - zelphirkalt 4y ago
      Are cryptographically-signed hashes really necessary in this case (why?)? What would be the secret, which is used for signing?
      - westurner 4y ago
        If the hashes are retrieved over the same channel as the package (i.e. HTTPS), and that channel is unfortunately compromised, why wouldn't a MITM tool change those software package artifact hash checksums too? Only if the key used to sign the package / package manifest with per-file hashes is or was retrieved over a different channel (i.e. WKD, HKP (HTTPS w/w/o Certificate Pinning (*))), and the key is trusted to sign for that package, then install the software package artifact and assign file permissions and extended filesystem attributes.

crucialfelix 4y ago

Hatch is also interesting and very similar to Poetry. https://hatch.pypa.io/latest/ https://hatch.pypa.io/latest/ In comparison to poetry I think it includes more advanced multi-environment and multi-python-version support and a tox-like testing matrix. It probably gets a little too complex there. It also works with pyproject.toml If anyone else has experience with Hatch vs Poetry please share!

wyuenho 4y ago
You can't lock your dependencies with Hatch. https://hatch.pypa.io/latest/meta/faq/#libraries-vs-applications https://hatch.pypa.io/latest/meta/faq/#libraries-vs-applicat...

wyuenho 4y ago

Poetry is good, I'll hit a bug in poetry from time to time but it's improving quickly. Direnv is better, direnv layout will automatically create a hidden virtualenv directory on your work tree and sets up your path when you cd into it. The only downside is it doesn't seem to work on Windows.

athorax 4y ago
I don't see the link between poetry and direnv? Poetry is about solving python's dependency issues, direnv doesn't seem to have anything to do with that
- wyuenho 4y ago
  Set up direnv, and then: cd /your/project && echo "layout python" > .envrc && direnv allow Give it a go then tell me what the point is for any of these poetry/pipenv/hatch/flit/pdm/pyflow thingy if neither you or your teammates work on Windows.
  - barefeg 4y ago
    When your project depends on a module version and that module depends on another one (sub dependency), it’s very common that re-installing the same module version in a new environment will cause something to break because the sub dependency was updated. This is not something that direnv solves therefore those other tools are still needed.
    - wyuenho 4y ago
      pip freeze > requirements.txt
      - barefeg 4y ago
        Then you lose track of which are actual dependencies and which are sub dependencies

datatrashfire 4y ago

My least favorite part about poetry is how slow it is.

Tainnor 4y ago
I seem to remember that some part of poetry's slowness is due to how the python index works and is therefore a problem shared by all such tools. That said, I've used both pipenv and poetry, and I had projects where pipenv would simply time out when trying to resolve packages. I haven't seen the same behaviour with poetry (indeed, that was the reason I migrated one project from pipenv to poetry after I just had to give up with the former).

ak217 4y ago

Poetry has a major issue with its lockfiles when working with active projects. It generates a top level dependency listing checksum, which causes any two PRs/branches that independently update the top level requirements to conflict with each other. The related issue, https://github.com/python-poetry/poetry/issues/496 https://github.com/python-poetry/poetry/issues/496, has been open for 4 years with no movement. The other issue with Poetry is that it uses its own pyproject.toml dependency listing format instead of the one standardized in the relevant PEP (https://peps.python.org/pep-0621/ https://peps.python.org/pep-0621/). This is understandable for historical reasons (Poetry was first written before this was standardized), but Poetry should have been updated to support the standard format. A relatively minor issue, but the poetry shell command is also a footgun. It's presented as a way to configure your shell to activate the virtualenv for the project. In reality it's a very slow, barely functional terminal emulator running on top of your terminal, which will cause problems for any programs that assume a working terminal or talk to the tty directly.

wyuenho 4y ago
https://twitter.com/SDisPater/status/1521932867214921728 https://twitter.com/SDisPater/status/1521932867214921728 poetry shell is also not a term emulator, it's just a subshell with some environment variables setup for your project. Once you are in, it's just a regular shell. If anything is slow, it's where you add or remove a dependency, but it's probably faster than you editing requirements.txt, clearing out your virtualenv and then reinstalling everything again.
- ak217 4y ago
  https://github.com/python-poetry/poetry/blob/master/src/poetry/utils/shell.py https://github.com/python-poetry/poetry/blob/master/src/poet... The process spawned by `poetry shell` is a terminal emulator driven by the pexpect and cleo packages. It hijacks and proxies the user's keystrokes before sending them to the underlying terminal.
  - wyuenho 4y ago
    Cleo creates "subcommands" in a git like manner, whereas pexpect spawns a subshell. That cleo Terminal class you see is only a viewport. A terminal emulator will be something substantially more complex such as libvterm. If it ain't handling terminfo, it ain't a terminal emulator. https://launchpad.net/libvterm https://launchpad.net/libvterm

Doxin 4y ago

> What I like about poetry is that it makes sure that the whole dependency graph of the packages that you add is correct. As does pip these days.