8 ms·

It took me a while to learn to love pipenv. It also bothers me how many developers "blame pipenv" for any problem they can't easily explain. Anyway, has Poetry

by korijn 7y ago

It took me a while to learn to love pipenv. It also bothers me how many developers "blame pipenv" for any problem they can't easily explain. Anyway, has Poetry caught up to the dependency resolution provided by the pipenv lock command yet? Last time I tried it (~6 months ago), it couldn't produce working environments for my requirements. BTW, for anyone who wants to educate themselves, read the issue description here and follow the links: https://github.com/pypa/pip/issues/988 https://github.com/pypa/pip/issues/988

tduberne 7y ago
That's funny, Poetry's README [1] claims exactly the opposite (that it resolves dependencies better than pipenv). And looking at the git history, that section of the README is 2 years old, so that claim was made when you experimented. Was the problem with dependency resolution you had similar to what Poetry describes in it README? Or are there other problems the README fails to mention? The description they make got me interested, but up to now all alternatives I tried to build moderately complex python projects failed to deliver on their promises, so definitely interested in what you have to say. [1] https://github.com/sdispater/poetry/blob/master/README.md#dependency-resolution https://github.com/sdispater/poetry/blob/master/README.md#de...
guitarbill 7y ago
I do really like the workflow pipenv provides. But last time I checked, the mayor downside of pipenv is it doesn't and won't support multiple Python versions [0]. For libraries, this can be a dealbreaker. For applications, especially ones you're deploying to a controlled environment, this isn't an issue. Been meaning to look into poetry. Not a huge fan of TOML, but hopefully all the tooling supports pyproject.toml now (I know black does, but not sure about flake8, isort, pylint, pytest, coverage). I know there's still some hacks required for tox + poetry though. [0] https://github.com/pypa/pipenv/issues/1050 https://github.com/pypa/pipenv/issues/1050
- korijn 7y ago
  I use pyenv to switch between python versions (or just the `py` launcher when on windows), and pipenv will follow suit. It does require regeneration of the lockfile for each python version/environment in the scenario you describe, but you can also keep them organized in a folder and swap them out for each environment. You're right that it doesn't come included in the tool though.
- ben509 7y ago
  It seems like pipenv and poetry fill different niches: pipenv is intended for application deployment, so you're likely to only have one version of python, while poetry is intended for building libraries. I found the tox support a bit hacky, but not bad, and got 3.5 to 3.8 testing just fine. Poetry does make the publishing process pretty seamless for the simple case. If you're starting a new library, poetry is great. I think poetry needs a "fuzz option" to test random valid solutions to the version constraints, to check if the older versions are really valid.
- meowface 7y ago
  What don't you like about TOML? It seems to be the best language for simple configuration files. I think YAML is often overkill for basic configuration, and JSON isn't ideal. pipenv's Pipfile is also in TOML. What config language would you prefer to use instead?
  - guitarbill 7y ago
    To be honest, I don't know. All I can say is YAML parsers exist and it works well enough for many applications, although I do understand the issues with it. Sure, it isn't perfect, but no need to re-invent the wheel for marginal improvements. I roughly agree with the arguments laid out in PEP 518, even though the array of tables thing is not at all "obvious" to me. And I actually do think an "official"/semi-official YAML subset would be great for loads of use-cases - yaml.safe_load already is that in practice.
flyingdeertown 7y ago
My experience was the opposite. pipenv always told me something couldn't be installed because of version mismatches and poetry has never given me any trouble.
- korijn 7y ago
  When you use `pipenv lock --verbose` you trigger holistic dependency resolution with reporting on the process and I have yet to encounter _any_ issues with that. It is important to apply pipenv as intended; I only use `pipenv lock` and `pipenv sync`, and edit Pipfile manually. The problem with other commands such as `pipenv install` is that they don't apply the holistic dependency resolution as `pipenv lock` does.
  - mcintyre1994 7y ago
    How are you supposed to know you're not meant to use `pipenv install <package>` and should instead manually edit Pipfile? The documentation starts at `pipenv install <package>` and never gets to "but that's not applying pipenv as intended so don't do it".
    - korijn 7y ago
      I don't know. I'm just happy to have stumbled upon an effortless workflow (for my applications) and thought I would share the nugget of knowledge. Perhaps "as intended" is not a correct assumption on my part.
epage 7y ago
Does pipenv create cross-platform lock files? That has been my primary requirement when looking at solutions, more than the usability issue of Python using a bunch of tools and files for the whole development process without a coherent story. iirc Poetry does have cross-platform lock files and a coherent story. I'm just blocked on some PRs due to the original author getting busy and not yet having delegated to lieutenants.
- ben509 7y ago
  Mostly. The lock file format is certainly consistent between platforms, and I've been able to install applications on windows, mac and linux. But it doesn't handle platform-specific dependencies correctly. For instance, `doit` has a watch feature, and it will depend on pyinotify on linux and macfsevents on Mac. If I run pipenv lock on the Mac, it correctly generates this: "macfsevents": { "hashes": [ "sha256:hexhexhex..." ], "markers": "sys_platform == 'darwin'", "version": "==0.8.1" }, But it won't lock the linux version. Similarly, if you lock on Linux, it throws out macfsevents. And `doit` is not doing anything strange[1] to report these extra dependencies. [1]: https://github.com/pydoit/doit/blob/660d3a2e75dc9d861e5a8916a5723d01a14a0945/setup.py#L81 https://github.com/pydoit/doit/blob/660d3a2e75dc9d861e5a8916...
ben509 7y ago
Pipenv's dependency resolution doesn't handle backtracking[1]. I've tried similar configurations with poetry and it doesn't seem to have a problem with them. [1]: https://github.com/pypa/pipenv/issues/2596 https://github.com/pypa/pipenv/issues/2596
- korijn 7y ago
  Not even with the `pipenv lock` command? It seems to work out solutions in multiple rounds, including backtracking, but I might be wrong on that observation...
  - ben509 7y ago
    It does do multiple rounds, but once it has decided on a constraint, it can't revisit that constraint.
theptip 7y ago
In the past Pipenv has failed to resolve dependencies for my project before, where Poetry (run last week) succeeded. Not a fair comparison as I've not tried Pipenv recently, but that at least lets you know that Poetry is now better than where pipenv was a few months ago. $ pip freeze |wc -l 204

by korijn 7y ago

tduberne 7y ago

That's funny, Poetry's README [1] claims exactly the opposite (that it resolves dependencies better than pipenv). And looking at the git history, that section of the README is 2 years old, so that claim was made when you experimented. Was the problem with dependency resolution you had similar to what Poetry describes in it README? Or are there other problems the README fails to mention? The description they make got me interested, but up to now all alternatives I tried to build moderately complex python projects failed to deliver on their promises, so definitely interested in what you have to say. [1] https://github.com/sdispater/poetry/blob/master/README.md#dependency-resolution https://github.com/sdispater/poetry/blob/master/README.md#de...

guitarbill 7y ago

I do really like the workflow pipenv provides. But last time I checked, the mayor downside of pipenv is it doesn't and won't support multiple Python versions [0]. For libraries, this can be a dealbreaker. For applications, especially ones you're deploying to a controlled environment, this isn't an issue. Been meaning to look into poetry. Not a huge fan of TOML, but hopefully all the tooling supports pyproject.toml now (I know black does, but not sure about flake8, isort, pylint, pytest, coverage). I know there's still some hacks required for tox + poetry though. [0] https://github.com/pypa/pipenv/issues/1050 https://github.com/pypa/pipenv/issues/1050

korijn 7y ago
I use pyenv to switch between python versions (or just the `py` launcher when on windows), and pipenv will follow suit. It does require regeneration of the lockfile for each python version/environment in the scenario you describe, but you can also keep them organized in a folder and swap them out for each environment. You're right that it doesn't come included in the tool though.
ben509 7y ago
It seems like pipenv and poetry fill different niches: pipenv is intended for application deployment, so you're likely to only have one version of python, while poetry is intended for building libraries. I found the tox support a bit hacky, but not bad, and got 3.5 to 3.8 testing just fine. Poetry does make the publishing process pretty seamless for the simple case. If you're starting a new library, poetry is great. I think poetry needs a "fuzz option" to test random valid solutions to the version constraints, to check if the older versions are really valid.
meowface 7y ago
What don't you like about TOML? It seems to be the best language for simple configuration files. I think YAML is often overkill for basic configuration, and JSON isn't ideal. pipenv's Pipfile is also in TOML. What config language would you prefer to use instead?
- guitarbill 7y ago
  To be honest, I don't know. All I can say is YAML parsers exist and it works well enough for many applications, although I do understand the issues with it. Sure, it isn't perfect, but no need to re-invent the wheel for marginal improvements. I roughly agree with the arguments laid out in PEP 518, even though the array of tables thing is not at all "obvious" to me. And I actually do think an "official"/semi-official YAML subset would be great for loads of use-cases - yaml.safe_load already is that in practice.

flyingdeertown 7y ago

My experience was the opposite. pipenv always told me something couldn't be installed because of version mismatches and poetry has never given me any trouble.

korijn 7y ago
When you use `pipenv lock --verbose` you trigger holistic dependency resolution with reporting on the process and I have yet to encounter _any_ issues with that. It is important to apply pipenv as intended; I only use `pipenv lock` and `pipenv sync`, and edit Pipfile manually. The problem with other commands such as `pipenv install` is that they don't apply the holistic dependency resolution as `pipenv lock` does.
- mcintyre1994 7y ago
  How are you supposed to know you're not meant to use `pipenv install <package>` and should instead manually edit Pipfile? The documentation starts at `pipenv install <package>` and never gets to "but that's not applying pipenv as intended so don't do it".
  - korijn 7y ago
    I don't know. I'm just happy to have stumbled upon an effortless workflow (for my applications) and thought I would share the nugget of knowledge. Perhaps "as intended" is not a correct assumption on my part.

epage 7y ago

Does pipenv create cross-platform lock files? That has been my primary requirement when looking at solutions, more than the usability issue of Python using a bunch of tools and files for the whole development process without a coherent story. iirc Poetry does have cross-platform lock files and a coherent story. I'm just blocked on some PRs due to the original author getting busy and not yet having delegated to lieutenants.

ben509 7y ago
Mostly. The lock file format is certainly consistent between platforms, and I've been able to install applications on windows, mac and linux. But it doesn't handle platform-specific dependencies correctly. For instance, `doit` has a watch feature, and it will depend on pyinotify on linux and macfsevents on Mac. If I run pipenv lock on the Mac, it correctly generates this: "macfsevents": { "hashes": [ "sha256:hexhexhex..." ], "markers": "sys_platform == 'darwin'", "version": "==0.8.1" }, But it won't lock the linux version. Similarly, if you lock on Linux, it throws out macfsevents. And `doit` is not doing anything strange[1] to report these extra dependencies. [1]: https://github.com/pydoit/doit/blob/660d3a2e75dc9d861e5a8916a5723d01a14a0945/setup.py#L81 https://github.com/pydoit/doit/blob/660d3a2e75dc9d861e5a8916...

ben509 7y ago

Pipenv's dependency resolution doesn't handle backtracking[1]. I've tried similar configurations with poetry and it doesn't seem to have a problem with them. [1]: https://github.com/pypa/pipenv/issues/2596 https://github.com/pypa/pipenv/issues/2596

korijn 7y ago
Not even with the `pipenv lock` command? It seems to work out solutions in multiple rounds, including backtracking, but I might be wrong on that observation...
- ben509 7y ago
  It does do multiple rounds, but once it has decided on a constraint, it can't revisit that constraint.

theptip 7y ago

In the past Pipenv has failed to resolve dependencies for my project before, where Poetry (run last week) succeeded. Not a fair comparison as I've not tried Pipenv recently, but that at least lets you know that Poetry is now better than where pipenv was a few months ago. $ pip freeze |wc -l 204