4 ms·

It's a bit of an old post, and yes Pipenv is not the go-to tool anymore. pip-tools is okay for people that really, really love their requirements.txt; otherwise

by greut 4y ago

It's a bit of an old post, and yes Pipenv is not the go-to tool anymore. pip-tools is okay for people that really, really love their requirements.txt; otherwise we tend to go with Poetry at work. Any folks having a good experience with PDM https://github.com/pdm-project/pdm https://github.com/pdm-project/pdm ?

strbean 4y ago
How is the maturity/stability of Poetry these days? I despise Pipenv, and was hoping to push for a switch to Poetry at my place of work a couple years ago, but I ran into blocking bugs across multiple versions (latest N versions affected by bug A, prior M versions affected by bug B). Had to chock it up to "not yet mature enough" and resign myself to the absurd lock times and countless terrible behaviors of Pipenv.
- aden1ne 4y ago
  We use poetry for all python projects. I haven't seen an actual poetry internal bug in quite a while, but using poetry effectively does require one keep some things in mind that are probably non-obvious to newcomers: 1. Poetry's default assumption on packages respecting semver simply does not hold up in reality. There are very few packages actually sticking to semver. Thus the `^x.y.z` default version range is quite often too loose. I've found that using `~x.y.z` for most packages is far more stable. 2. Imho, `poetry update` is a footgun. Without specifier, it will attempt to update the entire dependency tree. Not only is this slow, together with 1) it's all too likely one ends up with dependencies that actually are incompatible at runtime. I'd much rather have a `poetry update --all` flag instead for the rare instance I do want to update everything. The default behaviour should be to require a list of packages to update. 3. There are some common packages that cause very long resolution times if they are not restricted. Case in point: boto3. Even if one doesn't use boto3 oneself, it's very likely a transient dependency. Many packages simply specify `'*'` as their version dependency (they shouldn't, but it's the unfortunate reality many do). This will cause poetry to consider every possible boto3 version. With hundreds of versions - boto3 has a release every other day - this gets unwieldy fast. So I often end up specifying boto3 myself with some sensible range in my toml file, even when it's not a strict dependency of my own project. 4. The datascience ecosystem needs particular attention. Best to simply pin those, as every pandas update is guaranteed to break something. ABI changes to numpy are a particular nightmare. This is again due to too many packages simply specifying `'*'` for their numpy dependency. Which is further complicated by the fact that most don't distinguish between build-time dependencies and run-time dependencies. The numpy ABI is only forward compatible. Hence one should build with the oldest supported numpy[0]. [0]: https://pypi.org/project/oldest-supported-numpy/ https://pypi.org/project/oldest-supported-numpy/
  - strbean 4y ago
    I'll take the behavior of 2 over Pipenv's "any re-lock aggressively updates everything" approach. What's the point of lock files if you have to peg everything in order to have stability / control over versions??
  - 41b696ef1113 4y ago
    Updating dependencies (#2) does seem needlessly painful. I have wondered if I am missing some obvious workflow.
- bamboozled 4y ago
  Seems very good, except they do seem to have some long running per-releases going which seems troubling. Just ship it already!
gooseyard 4y ago
After a series of bad experiences with Poetry, I switched the packages I maintain to PDM. Although I have hit a few minor snags, the maintainer and other users in the project's github discussions have never failed to help with a fix, workaround, or advice. It's pleasant to use, and I think the only feature I cared about which was in Poetry but not PDM (a publish-to-pypi command), has proven easier to do with the Twine tool anyway. My situation is I think unusual, in that I need to use a private pypi repo which requires mTLS for both fetching and publishing. Had it not been for that I suspect I'd still be using Poetry, but given the experiences I've had with PDM I wouldn't switch back even if the situation with my repo changed.
- bamboozled 4y ago
  See my above comment, something to be aware of.
bamboozled 4y ago
PDM is based on PEP-582, which is only a draft and you will hit edge cases where it's not supported, or projects refuse to support it because of this. I'd avoid it for that reason personally. virtualenvs are a much more supported.
- radus 4y ago
  You can opt out of PEP-582, and in PDM 2.0 (just released), it becomes opt in.
  - bamboozled 4y ago
    What’s the point of using it over out Poetry then ? Why do we need a other ?

by greut 4y ago

strbean 4y ago

How is the maturity/stability of Poetry these days? I despise Pipenv, and was hoping to push for a switch to Poetry at my place of work a couple years ago, but I ran into blocking bugs across multiple versions (latest N versions affected by bug A, prior M versions affected by bug B). Had to chock it up to "not yet mature enough" and resign myself to the absurd lock times and countless terrible behaviors of Pipenv.

aden1ne 4y ago
We use poetry for all python projects. I haven't seen an actual poetry internal bug in quite a while, but using poetry effectively does require one keep some things in mind that are probably non-obvious to newcomers: 1. Poetry's default assumption on packages respecting semver simply does not hold up in reality. There are very few packages actually sticking to semver. Thus the `^x.y.z` default version range is quite often too loose. I've found that using `~x.y.z` for most packages is far more stable. 2. Imho, `poetry update` is a footgun. Without specifier, it will attempt to update the entire dependency tree. Not only is this slow, together with 1) it's all too likely one ends up with dependencies that actually are incompatible at runtime. I'd much rather have a `poetry update --all` flag instead for the rare instance I do want to update everything. The default behaviour should be to require a list of packages to update. 3. There are some common packages that cause very long resolution times if they are not restricted. Case in point: boto3. Even if one doesn't use boto3 oneself, it's very likely a transient dependency. Many packages simply specify `'*'` as their version dependency (they shouldn't, but it's the unfortunate reality many do). This will cause poetry to consider every possible boto3 version. With hundreds of versions - boto3 has a release every other day - this gets unwieldy fast. So I often end up specifying boto3 myself with some sensible range in my toml file, even when it's not a strict dependency of my own project. 4. The datascience ecosystem needs particular attention. Best to simply pin those, as every pandas update is guaranteed to break something. ABI changes to numpy are a particular nightmare. This is again due to too many packages simply specifying `'*'` for their numpy dependency. Which is further complicated by the fact that most don't distinguish between build-time dependencies and run-time dependencies. The numpy ABI is only forward compatible. Hence one should build with the oldest supported numpy[0]. [0]: https://pypi.org/project/oldest-supported-numpy/ https://pypi.org/project/oldest-supported-numpy/
- strbean 4y ago
  I'll take the behavior of 2 over Pipenv's "any re-lock aggressively updates everything" approach. What's the point of lock files if you have to peg everything in order to have stability / control over versions??
- 41b696ef1113 4y ago
  Updating dependencies (#2) does seem needlessly painful. I have wondered if I am missing some obvious workflow.
bamboozled 4y ago
Seems very good, except they do seem to have some long running per-releases going which seems troubling. Just ship it already!

gooseyard 4y ago

After a series of bad experiences with Poetry, I switched the packages I maintain to PDM. Although I have hit a few minor snags, the maintainer and other users in the project's github discussions have never failed to help with a fix, workaround, or advice. It's pleasant to use, and I think the only feature I cared about which was in Poetry but not PDM (a publish-to-pypi command), has proven easier to do with the Twine tool anyway. My situation is I think unusual, in that I need to use a private pypi repo which requires mTLS for both fetching and publishing. Had it not been for that I suspect I'd still be using Poetry, but given the experiences I've had with PDM I wouldn't switch back even if the situation with my repo changed.

bamboozled 4y ago
See my above comment, something to be aware of.

bamboozled 4y ago

PDM is based on PEP-582, which is only a draft and you will hit edge cases where it's not supported, or projects refuse to support it because of this. I'd avoid it for that reason personally. virtualenvs are a much more supported.

radus 4y ago
You can opt out of PEP-582, and in PDM 2.0 (just released), it becomes opt in.
- bamboozled 4y ago
  What’s the point of using it over out Poetry then ? Why do we need a other ?