7 ms·

Poetry – Python dependency management and packaging

46 points by reedwolf 6y ago

zelphirkalt 6y ago
Poetry still has to fight with bugs, as you can see looking at the git repository and the issues there. However, in most cases it works fine for me. Dependency resolution might be a hard problem to solve concurrently, but it would be great, if poetry did not only use a single core, to speed up the process.
dagenix 6y ago
Poetry is far and away the most interesting solution when its comes to managing Python dependencies.
- ratherlongname 6y ago
  Care to elaborate for us muggles
kissgyorgy 6y ago
I agree. Never had any problem with it so far.
Animats 6y ago
Another Python packaging system? I thought we were done when "wheels" replaced "eggs". ("Eggs", of course, replaced "setup.py" files.)
- Caligatio 6y ago
  The final output of Poetry is a wheel so it's not an egg/sdist/wheel competitor.
- geofft 6y ago
  Poetry is a tool to install packages, in whatever form they're in. It's roughly in the same space as pip/easy_install or virtualenvwrapper, but it's really a higher-level tool than anything that is currently standard, which is what makes it worthwhile. (It's in the same space as Cargo, if that helps.) Wheels and eggs are packaging formats, which Poetry (and pip) installs. You can think of poetry as like yum/dnf or apt and wheels/eggs as like .rpm or .deb files. (You can think of pip as like the rpm or dpkg command, in the sense that it's a lower-level tool but it also installs wheels/eggs, but that analogy isn't perfect.) The existence of Poetry does not obsolete either wheels or eggs. Neither wheels nor eggs replaced setup.py files - the standard way of building a wheel is to run "setup.py bdist_wheel". You can think of setup.py as like Makefile. You would not say that tarballs have replaced Makefiles.
  - elcomet 6y ago
    Poetry also builds your package to create the wheel file. So it's more than pip. Check the examples on the websites to learn more about it
    - geofft 6y ago
      Yup - I'm familiar with it, I'm just trying to come up with a loose analogy for someone who isn't familiar with Python packaging but has heard things second-hand.
- viraptor 6y ago
  If you're familiar with Ruby, a good mapping would be: gem (file format) is like wheel, gem (the command) is like pip, and bundler is like poetry. It just manages / ties everything together.
nurettin 6y ago
Would have been nice if poetry shell loaded .env I got used to that workflow with pipenv.
- rajasimon 6y ago
  Arguing that doesn't help https://github.com/python-poetry/poetry/issues/337#issuecomment-601945357 https://github.com/python-poetry/poetry/issues/337#issuecomm...
  - geofft 6y ago
    It sounds like there's interest in a plugin system which would handle things like this? I agree, this doesn't seem like it makes sense to be built into a tool like poetry - it's a little too magic. And I'd have a use case for doing different kinds of runtime configuration (not environment variables) if it had a hook system.
  - nurettin 6y ago
    This seems nice, I'm down with that. Still would have been nicer if poetry had it. https://github.com/python-poetry/poetry/issues/337#issuecomment-660549880 https://github.com/python-poetry/poetry/issues/337#issuecomm...
Caligatio 6y ago
I'm a big fan of Poetry (I have a few commits in there) but it's not without its downsides. At least a few months ago, you needed to install an entire compilation suite if you wanted to install a pure-Python source distribution (sdist) that was created with Poetry on Alpine. Somewhat by design, sdists created with Poetry require Poetry to install. Poetry itself is dependent on the cryptography package which, like all like all packages that use native-C extensions, suffers from a lack of musl-compatible wheels.
- geofft 6y ago
  Hmm, this is solvable in theory by defining a platform tag for musl such that authors of C extensions could upload musl wheels, right? I see some discussion at https://github.com/pypa/manylinux/issues/37 https://github.com/pypa/manylinux/issues/37 but I don't totally follow why it died off.
  - deleted 6y ago
    [deleted]
  - Caligatio 6y ago
    You can really go down the rabbit hole trying to figure out where the wheel+Alpine problem should be fixed. https://github.com/pypa/packaging-problems/issues/69 https://github.com/pypa/packaging-problems/issues/69 is a newer bug that is tracking this. Python relies on the "manylinux" specification (https://github.com/pypa/manylinux https://github.com/pypa/manylinux) which originally pinned build environments to CentOS 5, was updated in 2018 to use CentOS 6, and then updated in 2019 to use CentOS 7. It's entirely possible that the Alpine user base isn't as big/important as I would think but this seems like something that needs to be solved.
  - corndoge 6y ago
    The pypa maintainers, in my experience, will engage with such issues but ultimately discussion dies off and they simply ignore corner cases. Realistically speaking if you want to ship impure wheels for anything outside of ppc(64le), armhf, x86 or amd64 you're SOL. Python packaging is an absolute shit show especially where impure wheels are involved. Between the manylinux "standards" that require downloading random Centos docker images because their libc is "probably old enough" to the setuptools vs distutils vs whatever else with none of them having actually mature support for binary extensions (and even less - really zero, and no docs - for raw .so's via ctypes) to the byzantine PyPI upload procedures with its underspecified platform tags (really a manylinux problem though), my Python packaging experience was so bad that I ultimately gave up on the Python ecosystem as a whole. Shipping applications and libraries with this language is simply awful to the point where I'd rather use a compiled language because it is easier to build binaries for 10 different arches under qemu than it is to ship a fucking wheel. I'm so over Python.
turbinerneiter 6y ago
I tried it and liked it, really simple to use, but: * how do I use it to install i.e. service files, config files etc.? * can I use it to generate Deb/RPM packages? I'm not sure if packaging applications is as much of goal as packaging libraries, but I felt that it was missing stuff to do the former.
- makeworld 6y ago
  It's not really for that. It can make Python packages for PyPI and manage deps and that's it. Building final deployment packages like debs are out of scope.
softinio 6y ago
I tried it for the first time couple of months ago and I loved it, its my preferred way of managing python packages given the choice. Would love to see greater adoption.
midrus 6y ago
https://xkcd.com/927/ https://xkcd.com/927/ Finally! A standard for managing dependencies in Python.
ai_ja_nai 6y ago
What's the advantage over pipenv?
- zaro 6y ago
  Are there any advantages with pupenv at all? Pipenv definitely takes the crown in my list of Package managers that are plain stupid.
  - ai_ja_nai 6y ago
    Well, it was the first locking package manager for Python. A definite edge over plain pip. It always worked without issues for me. What's so bad about it?
    - globular-toast 6y ago
      Pip-tools was around a lot longer (more than two years).
    - sirn 6y ago
      Pipenv tries to do so much (e.g. a pipenv command involve poking around procfs to detect shell via shellingham to display colors), used to introduce breaking changes between releases (before they switched the scheme to CalVer), and takes a really long time between each releases (longer for bugs to get fixed). Most people that switched from Pipenv to Poetry were probably done so during the end of 2018 throughout 2019, of which Pipenv has no releases at all[1] (non-alpha/beta version were only released in November 2018 then May 2020). There were a rant from few years ago[2][3] regarding other issues (performance, etc.). [1]: https://news.ycombinator.com/item?id=21781421 https://news.ycombinator.com/item?id=21781421 [2]: https://chriswarrick.com/blog/2018/07/17/pipenv-promises-a-lot-delivers-very-little/ https://chriswarrick.com/blog/2018/07/17/pipenv-promises-a-l... [3]: https://news.ycombinator.com/item?id=18612590 https://news.ycombinator.com/item?id=18612590
    - zaro 6y ago
      It kind of works for my team, but only because nobody touches it anymore. What's so bad about it? Well it's bad at being package manager for starter. Here is a quick question. How do you install packages with pipenv with the versions specified in the lock file?
kelsolaar 6y ago
Poetry is awesome, it has its quirks but really simplified our work at https://www.colour-science.org/ https://www.colour-science.org/. We have dropped Conda entirely as it solved all our previous Pip dependency hell that ended up with the adoption of Conda. As I'm writing that we are coincidentally and unfortunately plagued by random CI failures occurring when Poetry resolves the dependencies: https://github.com/actions/virtual-environments/issues/1343 https://github.com/actions/virtual-environments/issues/1343
- zzleeper 6y ago
  Wow, anything that removes the need for conda is quite promising indeed!
- wdroz 6y ago
  Conda let you install non python stuffs like cudatoolkit and cudnn. Can you do the same with Poetry?
  - kelsolaar 6y ago
    AFAIK, no, Poetry deals with Pypi, Github and urls to Python packages: https://python-poetry.org/docs/dependency-specification/ https://python-poetry.org/docs/dependency-specification/
abalaji 6y ago
Poetry is fantastic but it looks like pip is trying to create their own competitor. [1] I would much prefer if the pypa folks and the poetry folks got together to more tightly integrate the systems. Poetry solves a ton of problems ranging from venv management to making packaging easier to splitting dev and production dependencies. Consider me a huge fan. [1] https://github.com/pypa/pip/issues/8511#issuecomment-665140265 https://github.com/pypa/pip/issues/8511#issuecomment-6651402...
emmanueloga_ 6y ago
I'm a complete outsider to the python ecosystem, from time to time I need to write a script or use a popular python library like pygments and I used to hate having to do anything with python because of how difficult it seemed to install anything (cognitive overload of having 20 different ways to install things...). Anyway, recently I discovered miniconda and it seems like the best, most unambiguous way to install python. It is small (in rel terms...), it works like a charm on Windows, and I can just do something run a single command like `conda install pygments` and I'm all set! Just not sure why I would ever need to look into something like poetry, although I heard conda and poetry can complement each other somehow? (as do pip, easy_install or whatever is called and all the others haha :-p).
- Spiritus 6y ago
  > [...] from time to time I need to write a script or use a popular python library like pygments and I used to hate having to do anything with python because of how difficult it seemed to install anything Not sure I understand why you're having so much trouble. It's literally two commands: $ python3 -m venv venv # Create virtualenv $ ./venv/bin/pip install pygments However! One unfortunate thing with this is that `python3 -m venv` doesn't include the 'wheel' package in the virtualenv out-of-the-box. So it's often a good idea to install/upgrade pip, setuptools and wheel after creating the virtualenv: $ ./venv/bin/pip install -U pip setuptools wheel
  - emmanueloga_ 6y ago
    "Not sure I understands why you're having so much trouble" ... proceeds to explain a procedure that involves understanding various moving parts, including virtualenvs, pip, setuptools, wheel, etc... Even if I'm not gonna get deep into python, I like to feel like I understand what's going on. With python it just involves a lot more effort than with other ecosystems, even if you think that these things are easy to learn. Compare with ruby: `gem install the-gem`. Done. No "virtual envs", no plethora of similar but different tools to do slightly the same thing / whatever, no preconditions, no step by step procedures. Simplicity is important!
ianamartin 6y ago
I'm going to sound like a very grumpy old man—because that's what I am—but if you actually need a tool like poetry, you've really fucked up. I have a ton of respect for the author. It's really good code and solves a hard problem in elegant ways. But it's a problem that we shouldn't let ourselves have. It's like a patient talking to a therapist: patient: I'm really depressed. therapist: Why do you think that is? p: Well, I've been having an affair with this woman, and my wife found out about it. t: How does she feel about that? p: She's pretty angry at me. It's affecting our relationship. t: How is it affecting your relationship? p: Well, she doesn't want to have sex with me anymore, and things have gotten a little weird with the kids. t: How do you feel about not having sex with your wife? p: It's depressing. t: And the kids? How do you feel about them? p: They'll grow up and understand eventually. t: Here's a pill you can take every day that will make you feel better. There are two ways to address this kind of situation. The first way is to give the patient a pill to fix the symptom of depression. The second way is to fix the behavior that's causing the depression. Poetry (and Cargo and other package/dependency managers) are a little pill you can take to make you feel better about stuff. But there's another school of therapy. I'm maybe going to sound a little like Zed Shaw here, but there's the "Don't fucking do that" school of therapy. It's possible to fix the underlying behavior that's causing the pain instead of taking a pill. It's harder and requires more work, sure. But in the long term it is a better, more stable solution. Since I'm already on a bit of a rant, I'll go ahead and say it out loud: agile is the source of many of these types of problems. You start out with good intentions and then one day you end up married to a thing that was only supposed to be a proof-of-concept, but it got shipped because product team and velocity, and now your life is hell, and that POC now has kids, and you're legally responsible for them, and fuck it, just give me a pill, doctor. Poetry is a brilliant solution to a problem we shouldn't create for ourselves.
- LittlePeter 6y ago
  Can you give concrete examples how you would do <insert some scenario> without poetry? Examples of what I really have messed up if I have to use poetry would be also nice. I have trouble coming up with concrete examples myself.
  - 6y ago
dmarchand90 6y ago
Previous discussion: https://news.ycombinator.com/item?id=20672436 https://news.ycombinator.com/item?id=20672436
ris 6y ago
> With x, packaging in Python is a solved problem This is the claim that has been made about every "advancement" in python packaging for the last 20 years. Some of us have been kicking around long enough to remember the great saviour being distutils, setuptools, easy_install, eggs, wheels, pip, pipfiles/pip-tools... The truth being that all people have managed to do is produce a horrific mess with an ever growing tower of components each of which never really solves the problem (and now people want me to add "tox" to the mix too!). I have to say that packaging is really the worst part of the python world. I do my best these days to avoid every component of the python packaging ecosystem I possibly can and use Nix as much as possible, which really Just Solves The Problem.
- drcongo 6y ago
  I agree that packaging is horrible in Python. Poetry is mostly excellent, but I have one project where running `poetry lock` takes over an hour and I have no idea why. I keep seeing this claim about Nix, and I keep trying to understand it, but every time I look into it I come out more confused. Nix appears to be an operating system (?) and packaging things _for_ Nix seems to be something akin to a dockerfile, but one that will only work on a machine with the Nix operating system? If I've got all of that right (which is a big "if"), I can see how that's useful for building a system's dependencies and deploying it to a Nix OS server, but I can never work out how any of it is useful if I don't run Nix OS on the servers or I want to distribute a Python package.
  - ris 6y ago
    So... Nix is just a package manager, but it's a package manager that tries to be as generic as possible and only have the attitude that a "package" is just "a bunch of (immutable) files that depend on the presence of another bunch of files" and so should be able to take over the job of all package mangers on your system (no matter how much of a special snowflake language x or application y thinks it is and deserves its own, inevitably crappy, package manager). One of the many advantages this brings is that your system's software installation makes sense "as a whole" and isn't the result of 7 different package managers or "auto updating" applications fighting each other. But because of this, Nix can only have things depend on packages it has provided itself. It doesn't assume anything about the host system apart from the kernel. So yes, installing Nix on an existing Linux system can feel like installing another distribution. But you only ever have to use as much of it as you want to. Everything lives in /nix, and to get rid of everything you just have to `rm -rf /nix`. NixOS is a full distribution that just uses Nix for everything, including configuration management.
merricksb 6y ago
Discussed here 7 months ago: https://news.ycombinator.com/item?id=21779191 https://news.ycombinator.com/item?id=21779191
jayaram 6y ago
I ended up with PyScaffold and never turned back again!

46 points by reedwolf 6y ago

Poetry still has to fight with bugs, as you can see looking at the git repository and the issues there. However, in most cases it works fine for me. Dependency resolution might be a hard problem to solve concurrently, but it would be great, if poetry did not only use a single core, to speed up the process.

dagenix 6y ago

Poetry is far and away the most interesting solution when its comes to managing Python dependencies.

ratherlongname 6y ago
Care to elaborate for us muggles

kissgyorgy 6y ago

I agree. Never had any problem with it so far.

Animats 6y ago

Another Python packaging system? I thought we were done when "wheels" replaced "eggs". ("Eggs", of course, replaced "setup.py" files.)

Caligatio 6y ago
The final output of Poetry is a wheel so it's not an egg/sdist/wheel competitor.
geofft 6y ago
Poetry is a tool to install packages, in whatever form they're in. It's roughly in the same space as pip/easy_install or virtualenvwrapper, but it's really a higher-level tool than anything that is currently standard, which is what makes it worthwhile. (It's in the same space as Cargo, if that helps.) Wheels and eggs are packaging formats, which Poetry (and pip) installs. You can think of poetry as like yum/dnf or apt and wheels/eggs as like .rpm or .deb files. (You can think of pip as like the rpm or dpkg command, in the sense that it's a lower-level tool but it also installs wheels/eggs, but that analogy isn't perfect.) The existence of Poetry does not obsolete either wheels or eggs. Neither wheels nor eggs replaced setup.py files - the standard way of building a wheel is to run "setup.py bdist_wheel". You can think of setup.py as like Makefile. You would not say that tarballs have replaced Makefiles.
- elcomet 6y ago
  Poetry also builds your package to create the wheel file. So it's more than pip. Check the examples on the websites to learn more about it
  - geofft 6y ago
    Yup - I'm familiar with it, I'm just trying to come up with a loose analogy for someone who isn't familiar with Python packaging but has heard things second-hand.
viraptor 6y ago
If you're familiar with Ruby, a good mapping would be: gem (file format) is like wheel, gem (the command) is like pip, and bundler is like poetry. It just manages / ties everything together.

nurettin 6y ago

Would have been nice if poetry shell loaded .env I got used to that workflow with pipenv.

rajasimon 6y ago
Arguing that doesn't help https://github.com/python-poetry/poetry/issues/337#issuecomment-601945357 https://github.com/python-poetry/poetry/issues/337#issuecomm...
- geofft 6y ago
  It sounds like there's interest in a plugin system which would handle things like this? I agree, this doesn't seem like it makes sense to be built into a tool like poetry - it's a little too magic. And I'd have a use case for doing different kinds of runtime configuration (not environment variables) if it had a hook system.
- nurettin 6y ago
  This seems nice, I'm down with that. Still would have been nicer if poetry had it. https://github.com/python-poetry/poetry/issues/337#issuecomment-660549880 https://github.com/python-poetry/poetry/issues/337#issuecomm...

Caligatio 6y ago

I'm a big fan of Poetry (I have a few commits in there) but it's not without its downsides. At least a few months ago, you needed to install an entire compilation suite if you wanted to install a pure-Python source distribution (sdist) that was created with Poetry on Alpine. Somewhat by design, sdists created with Poetry require Poetry to install. Poetry itself is dependent on the cryptography package which, like all like all packages that use native-C extensions, suffers from a lack of musl-compatible wheels.

geofft 6y ago
Hmm, this is solvable in theory by defining a platform tag for musl such that authors of C extensions could upload musl wheels, right? I see some discussion at https://github.com/pypa/manylinux/issues/37 https://github.com/pypa/manylinux/issues/37 but I don't totally follow why it died off.
- deleted 6y ago
  [deleted]
- Caligatio 6y ago
  You can really go down the rabbit hole trying to figure out where the wheel+Alpine problem should be fixed. https://github.com/pypa/packaging-problems/issues/69 https://github.com/pypa/packaging-problems/issues/69 is a newer bug that is tracking this. Python relies on the "manylinux" specification (https://github.com/pypa/manylinux https://github.com/pypa/manylinux) which originally pinned build environments to CentOS 5, was updated in 2018 to use CentOS 6, and then updated in 2019 to use CentOS 7. It's entirely possible that the Alpine user base isn't as big/important as I would think but this seems like something that needs to be solved.
- corndoge 6y ago
  The pypa maintainers, in my experience, will engage with such issues but ultimately discussion dies off and they simply ignore corner cases. Realistically speaking if you want to ship impure wheels for anything outside of ppc(64le), armhf, x86 or amd64 you're SOL. Python packaging is an absolute shit show especially where impure wheels are involved. Between the manylinux "standards" that require downloading random Centos docker images because their libc is "probably old enough" to the setuptools vs distutils vs whatever else with none of them having actually mature support for binary extensions (and even less - really zero, and no docs - for raw .so's via ctypes) to the byzantine PyPI upload procedures with its underspecified platform tags (really a manylinux problem though), my Python packaging experience was so bad that I ultimately gave up on the Python ecosystem as a whole. Shipping applications and libraries with this language is simply awful to the point where I'd rather use a compiled language because it is easier to build binaries for 10 different arches under qemu than it is to ship a fucking wheel. I'm so over Python.

turbinerneiter 6y ago

I tried it and liked it, really simple to use, but: * how do I use it to install i.e. service files, config files etc.? * can I use it to generate Deb/RPM packages? I'm not sure if packaging applications is as much of goal as packaging libraries, but I felt that it was missing stuff to do the former.

makeworld 6y ago
It's not really for that. It can make Python packages for PyPI and manage deps and that's it. Building final deployment packages like debs are out of scope.

softinio 6y ago

I tried it for the first time couple of months ago and I loved it, its my preferred way of managing python packages given the choice. Would love to see greater adoption.

midrus 6y ago

https://xkcd.com/927/ https://xkcd.com/927/ Finally! A standard for managing dependencies in Python.

ai_ja_nai 6y ago

What's the advantage over pipenv?

zaro 6y ago
Are there any advantages with pupenv at all? Pipenv definitely takes the crown in my list of Package managers that are plain stupid.
- ai_ja_nai 6y ago
  Well, it was the first locking package manager for Python. A definite edge over plain pip. It always worked without issues for me. What's so bad about it?
  - globular-toast 6y ago
    Pip-tools was around a lot longer (more than two years).
  - sirn 6y ago
    Pipenv tries to do so much (e.g. a pipenv command involve poking around procfs to detect shell via shellingham to display colors), used to introduce breaking changes between releases (before they switched the scheme to CalVer), and takes a really long time between each releases (longer for bugs to get fixed). Most people that switched from Pipenv to Poetry were probably done so during the end of 2018 throughout 2019, of which Pipenv has no releases at all[1] (non-alpha/beta version were only released in November 2018 then May 2020). There were a rant from few years ago[2][3] regarding other issues (performance, etc.). [1]: https://news.ycombinator.com/item?id=21781421 https://news.ycombinator.com/item?id=21781421 [2]: https://chriswarrick.com/blog/2018/07/17/pipenv-promises-a-lot-delivers-very-little/ https://chriswarrick.com/blog/2018/07/17/pipenv-promises-a-l... [3]: https://news.ycombinator.com/item?id=18612590 https://news.ycombinator.com/item?id=18612590
  - zaro 6y ago
    It kind of works for my team, but only because nobody touches it anymore. What's so bad about it? Well it's bad at being package manager for starter. Here is a quick question. How do you install packages with pipenv with the versions specified in the lock file?

kelsolaar 6y ago

Poetry is awesome, it has its quirks but really simplified our work at https://www.colour-science.org/ https://www.colour-science.org/. We have dropped Conda entirely as it solved all our previous Pip dependency hell that ended up with the adoption of Conda. As I'm writing that we are coincidentally and unfortunately plagued by random CI failures occurring when Poetry resolves the dependencies: https://github.com/actions/virtual-environments/issues/1343 https://github.com/actions/virtual-environments/issues/1343

zzleeper 6y ago
Wow, anything that removes the need for conda is quite promising indeed!
wdroz 6y ago
Conda let you install non python stuffs like cudatoolkit and cudnn. Can you do the same with Poetry?
- kelsolaar 6y ago
  AFAIK, no, Poetry deals with Pypi, Github and urls to Python packages: https://python-poetry.org/docs/dependency-specification/ https://python-poetry.org/docs/dependency-specification/

abalaji 6y ago

Poetry is fantastic but it looks like pip is trying to create their own competitor. [1] I would much prefer if the pypa folks and the poetry folks got together to more tightly integrate the systems. Poetry solves a ton of problems ranging from venv management to making packaging easier to splitting dev and production dependencies. Consider me a huge fan. [1] https://github.com/pypa/pip/issues/8511#issuecomment-665140265 https://github.com/pypa/pip/issues/8511#issuecomment-6651402...

emmanueloga_ 6y ago

I'm a complete outsider to the python ecosystem, from time to time I need to write a script or use a popular python library like pygments and I used to hate having to do anything with python because of how difficult it seemed to install anything (cognitive overload of having 20 different ways to install things...). Anyway, recently I discovered miniconda and it seems like the best, most unambiguous way to install python. It is small (in rel terms...), it works like a charm on Windows, and I can just do something run a single command like `conda install pygments` and I'm all set! Just not sure why I would ever need to look into something like poetry, although I heard conda and poetry can complement each other somehow? (as do pip, easy_install or whatever is called and all the others haha :-p).

Spiritus 6y ago
> [...] from time to time I need to write a script or use a popular python library like pygments and I used to hate having to do anything with python because of how difficult it seemed to install anything Not sure I understand why you're having so much trouble. It's literally two commands: $ python3 -m venv venv # Create virtualenv $ ./venv/bin/pip install pygments However! One unfortunate thing with this is that `python3 -m venv` doesn't include the 'wheel' package in the virtualenv out-of-the-box. So it's often a good idea to install/upgrade pip, setuptools and wheel after creating the virtualenv: $ ./venv/bin/pip install -U pip setuptools wheel
- emmanueloga_ 6y ago
  "Not sure I understands why you're having so much trouble" ... proceeds to explain a procedure that involves understanding various moving parts, including virtualenvs, pip, setuptools, wheel, etc... Even if I'm not gonna get deep into python, I like to feel like I understand what's going on. With python it just involves a lot more effort than with other ecosystems, even if you think that these things are easy to learn. Compare with ruby: `gem install the-gem`. Done. No "virtual envs", no plethora of similar but different tools to do slightly the same thing / whatever, no preconditions, no step by step procedures. Simplicity is important!

ianamartin 6y ago

I'm going to sound like a very grumpy old man—because that's what I am—but if you actually need a tool like poetry, you've really fucked up. I have a ton of respect for the author. It's really good code and solves a hard problem in elegant ways. But it's a problem that we shouldn't let ourselves have. It's like a patient talking to a therapist: patient: I'm really depressed. therapist: Why do you think that is? p: Well, I've been having an affair with this woman, and my wife found out about it. t: How does she feel about that? p: She's pretty angry at me. It's affecting our relationship. t: How is it affecting your relationship? p: Well, she doesn't want to have sex with me anymore, and things have gotten a little weird with the kids. t: How do you feel about not having sex with your wife? p: It's depressing. t: And the kids? How do you feel about them? p: They'll grow up and understand eventually. t: Here's a pill you can take every day that will make you feel better. There are two ways to address this kind of situation. The first way is to give the patient a pill to fix the symptom of depression. The second way is to fix the behavior that's causing the depression. Poetry (and Cargo and other package/dependency managers) are a little pill you can take to make you feel better about stuff. But there's another school of therapy. I'm maybe going to sound a little like Zed Shaw here, but there's the "Don't fucking do that" school of therapy. It's possible to fix the underlying behavior that's causing the pain instead of taking a pill. It's harder and requires more work, sure. But in the long term it is a better, more stable solution. Since I'm already on a bit of a rant, I'll go ahead and say it out loud: agile is the source of many of these types of problems. You start out with good intentions and then one day you end up married to a thing that was only supposed to be a proof-of-concept, but it got shipped because product team and velocity, and now your life is hell, and that POC now has kids, and you're legally responsible for them, and fuck it, just give me a pill, doctor. Poetry is a brilliant solution to a problem we shouldn't create for ourselves.

LittlePeter 6y ago
Can you give concrete examples how you would do <insert some scenario> without poetry? Examples of what I really have messed up if I have to use poetry would be also nice. I have trouble coming up with concrete examples myself.
- 6y ago

dmarchand90 6y ago

Previous discussion: https://news.ycombinator.com/item?id=20672436 https://news.ycombinator.com/item?id=20672436

ris 6y ago

> With x, packaging in Python is a solved problem This is the claim that has been made about every "advancement" in python packaging for the last 20 years. Some of us have been kicking around long enough to remember the great saviour being distutils, setuptools, easy_install, eggs, wheels, pip, pipfiles/pip-tools... The truth being that all people have managed to do is produce a horrific mess with an ever growing tower of components each of which never really solves the problem (and now people want me to add "tox" to the mix too!). I have to say that packaging is really the worst part of the python world. I do my best these days to avoid every component of the python packaging ecosystem I possibly can and use Nix as much as possible, which really Just Solves The Problem.

drcongo 6y ago
I agree that packaging is horrible in Python. Poetry is mostly excellent, but I have one project where running `poetry lock` takes over an hour and I have no idea why. I keep seeing this claim about Nix, and I keep trying to understand it, but every time I look into it I come out more confused. Nix appears to be an operating system (?) and packaging things _for_ Nix seems to be something akin to a dockerfile, but one that will only work on a machine with the Nix operating system? If I've got all of that right (which is a big "if"), I can see how that's useful for building a system's dependencies and deploying it to a Nix OS server, but I can never work out how any of it is useful if I don't run Nix OS on the servers or I want to distribute a Python package.
- ris 6y ago
  So... Nix is just a package manager, but it's a package manager that tries to be as generic as possible and only have the attitude that a "package" is just "a bunch of (immutable) files that depend on the presence of another bunch of files" and so should be able to take over the job of all package mangers on your system (no matter how much of a special snowflake language x or application y thinks it is and deserves its own, inevitably crappy, package manager). One of the many advantages this brings is that your system's software installation makes sense "as a whole" and isn't the result of 7 different package managers or "auto updating" applications fighting each other. But because of this, Nix can only have things depend on packages it has provided itself. It doesn't assume anything about the host system apart from the kernel. So yes, installing Nix on an existing Linux system can feel like installing another distribution. But you only ever have to use as much of it as you want to. Everything lives in /nix, and to get rid of everything you just have to `rm -rf /nix`. NixOS is a full distribution that just uses Nix for everything, including configuration management.

merricksb 6y ago

Discussed here 7 months ago: https://news.ycombinator.com/item?id=21779191 https://news.ycombinator.com/item?id=21779191

jayaram 6y ago

I ended up with PyScaffold and never turned back again!