8 ms·
Use Poetry. It does everything you need and PDM is a bit too new still.
by stavros 5y ago
Use Poetry. It does everything you need and PDM is a bit too new still.
- fb03 5y agoGreat. I have heard "anecdata evidence" that sometimes poetry fails to install a combination of packages or something along those lines, did you find any of those shenanigans in your own experience?
- akdor1154 5y ago(Not GP) Occasionally, but still use poetry. The issues happen because poetry is strict about version constraints. Pip used to be lax, so some packages could get away with poor/overly restrictive constraints.. Now pip is strict as well, and poetry is also getting a lot more common, so maintainers have had pressure to fix most of these issues.
- nbadg 5y agoI've run into this a couple of times. Typically it's when starting a new project and gradually adding dependencies. By default, when adding (poetry add <dep>), poetry adds the most recent version of the library, and in pyproject.toml that becomes the minimum possible version. If you then later try to add something that depends on that library, but isn't up-to-date on the most recent version, poetry will (correctly!) error out because it fails to find a compatible solution. For example, something like this: poetry add ingredients # most recent version: 1.0 poetry add cake # requires ingredients, but <1.0 So here, poetry would add ingredients to your pyproject.toml with version = "^1.0.0" in the first step. In the second step, when you tried to add cake as a new dependency, it would say it can't find a version of cake compatible with your pyproject.toml's requirement for ingredients. The solution is to change pyproject.toml to be more permissive about the version of ingredients, resulting in it downgrading to a compatible version. Or submit an issue and/or PR with the maintainers of cake to bump up the supported versions.
- frostming 5y agoIn PDM, add command supports a `--unconstrained` option to relax the version constraints to find a solution. You don't need to edit the file manually.
- andreareina 5y agoAnother way it happens: I was stuck on python 3.6 for a while and it meant that I couldn't just poetry add pandas because the latest version isn't compatible with 3.6 and it won't look for other versions.
- acidbaseextract 5y agoNo shenanigans for any library-ish code with few dependencies that targets relatively modern Python. I've had one or two fundamental version conflicts with a 5+ year old application with 100+ dependencies and a decent amount of legacy stuff. They were a pain in the ass, and the sdispater's stance on not allowing overrides is a pain in the ass. We ended up forking the upstream libraries to resolve the version conflict. With all of that, poetry is amazing and a huge step forward. I'd advocate it wholeheartedly.
- gooseyard 5y agoI've spent the last year or so comparing these packages. Initially, I liked poetry, but I began to run into issues related to client cert auth with private repositories. I and several other users who encountered these problems submitted pull requests to address them, but found that it was nearly impossible to get any of them reviewed, much less merged. Out of frustration, I decided to try pdm instead, and although pdm doesn't handle pushing to pypi repositories, Twine works fine for it, and I have had a pain-free experience using pdm for dependency management. I like the idea of poetry, but it's unusable if I can't get bugs fixed, and I gave up after nearly a year of trying.
- bb88 5y ago> but I began to run into issues related to client cert auth with private repositories. Yeah, I think that's the worst thing about private infrastructure in 2021 is getting those certs correctly installed and seen by the software and both sides. I'm thinking that tailscale or something like it may be a better alternative in the future where authorization is at the link layer before the tcp connect can happen.
- stavros 5y agoAs others have said, this is related to how Poetry does dependency resolution. If you say your program uses Python "^3.8", you would expect any Python version in that range to work, but instead, ALL Python versions in that range must work. If one of your dependencies won't work with 3.9, the dependency graph cannot resolve. The author of Poetry said that this is because Poetry is meant for libraries, where this makes more sense.
- amethyst 5y agoExcept Poetry's recommendations/decisions around version capping doesn't make sense for libraries. https://iscinumpy.dev/post/bound-version-constraints/ https://iscinumpy.dev/post/bound-version-constraints/
- jreese 5y agoAs a Python infra person, I would refine that to: If you're building an application, use Poetry. If you're building a library, use Flit. Use PEP621 metadata in pyproject.toml regardless. Poetry is much more focused on managing dependencies for applications than dealing with libraries that have to be used by other libraries or applications. See this deep discussion for some timely/relevant examples: https://iscinumpy.dev/post/bound-version-constraints/ https://iscinumpy.dev/post/bound-version-constraints/
- kbumsik 5y agoCan we mix Poetry and PEP621 though? I tried to use project.dependencies and project.optional-dependencies fields in pyproject.toml defined by PEP621. However Poetry seems to ignore it and it tries to stick with its own tools.poetry section.
- jreese 5y agoAh, you're right. I thought Poetry had adopted PEP 621 already, but it looks like there's still a debate about if/how they want to do that. A bit ironic considering the author of Poetry was listed as an author on PEP 621, and that was accepted just over a year ago.
- kbumsik 5y agoIt went final this March though. So Poetry might needs more time to think about the transitioning plan. Python packaging is again in a chaos since 2021 because of the rise of standardized PEPs.
- spurgu 5y agoI know you didn't mean it that way, but "since 2021"..? :D I was working actively with Python (and loving it) between 2012-2015 (during which the 2 vs 3 transition/conflicts were raging) and the couple of times I've tried to get back into the groove since then (in side projects) I keep hitting walls and getting confused about how to even get the basic things running. I've heard good things about Poetry but never tried it. PDM being closer to npm/yarn sounds like an attractive option (since they're very familar), just to tinker around (not to build a complex application with specific requirements). Will give it a shot!
- zmmmmm 5y agoSold everyone on using poetry a few months ago and now red faced as we have a litany of problems and time waste due to it. We are now sitting on some bleeding edge branch because specific dependencies cannot work at all without some new fangled feature and everyone wishes we were just using plain virtualenv as we had much less problems with that.
- smohare 5y agoIn other words, you work with people who have no clue what’s going on.
- spurgu 5y agoThanks, that's good advice!
- zelphirkalt 5y agoYou would also not have any real control over what gets installed with just venv. So everyone might think the grass was greener on the other side, but it might not actually be, if you value reproducible environments.
- funstuff007 5y agoWhat you've never spent hours trying out n! orderings of requirements.txt until you get the results you want/can live with? You have not lived until you've done this. /s
- bb88 5y agoDep Hell is real.
- burgerrito 5y agoIs Poetry used a lot by Python devs? What package management a lot of Python devs use? I mostly code on JavaScript and (obviously) use NPM a lot, and it makes me wonder.