24 ms·

If anyone hasn't seen it, now is a good time to look at https://python-poetry.org/ https://python-poetry.org/ It is rapidly becoming _the_ package manager to us

If anyone hasn't seen it, now is a good time to look at https://python-poetry.org/ https://python-poetry.org/ It is rapidly becoming _the_ package manager to use. I've used it in a bunch of personal and professional projects with zero issues. It's been rock solid so far, and I'm definitely a massive fan.

skrtskrt 6y ago
Pasting this from another comment: Poetry is still rough around the edges but I think the core experience is great and it's well on its way to be a very popular tool. Before using it at work, we are waiting on waiting on https://github.com/python-poetry/poetry/issues/2610 https://github.com/python-poetry/poetry/issues/2610 (alternate repository not getting used for transitive dependencies) and ideally this https://github.com/python-poetry/poetry/issues/1556 https://github.com/python-poetry/poetry/issues/1556 (disable SSL verify for alternate repositories) If you don't use your own PyPi for a bunch of internal packages, it works great imo. One more wish item would be having absolute path dependencies instead of only relative path.
- wp381640 6y ago
  I hope the second is never changed - it’s 2020, time to start authenticating the servers you’re downloading and executing code from Internal doesn’t mean secure
  - MrOxiMoron 6y ago
    Have to agree, and if it is truly internal why are you doing SSL?
    - zzzeek 6y ago
      SSL for internal services seems to be becoming common these days and that's not a bad thing. Corporate/institutional information warfare is becoming a pretty big deal now (see all the US hospitals being infiltrated) so hardening on the inside to at least slow down an internal threat is not a bad thing at all.
ghshephard 6y ago
I occasionally ask our principle / sr. Python engineers about this, and their response is always, "These things come and go, virtualenv/wrappers + pip + requirements.txt works fine - no need to look at anything else." We've got about 15 repos, with the largest repo containing about 1575 files and 34MBytes of .py source, 14 current developers (with about 40 over the last 10 years) - and they really are quite proficient, but haven't demonstrated any interest at looking at anything outside pip/virtualenv. Is there a reason to look at poetry if you've got the pip/virtualenv combination working fine? People who use poetry seem to love it - so I'm interested in whether it provides any new abilities / flexibility that pip doesn't.
- mplewis 6y ago
  You're losing the ability to easily update using fuzzy specs and a lockfile if you're using pip vanilla without pip-tools or poetry.
  - skrtskrt 6y ago
    To clarify further - lockfile == reproducible builds without having to pip freeze every single dependency in the tree. Fuzzy specs == effortless upgrades according to your risk tolerance for a given library (major version for boto3, minor version for pandas, something like that). Poetry gets you the combination of the two: Let your dep versions float, and easily revert back to a previous deterministic build using the version-controlled lockfile if something breaks.
    - xapata 6y ago
      setup.cfg vs requirements.txt offers this.
      - skrtskrt 6y ago
        Poetry is by no means the only option for this. Lots of people like pip-tools, it would feel a lot more lightweight and closer to pip than Poetry does. Pipenv exists but... steer clear for a multitude of reasons. Personally I like that Poetry centers itself around the pyproject.toml standard. I also think that its usability and the enthusiasm of both the maintainers of the users is going to really carry it more into the Python mainstream in the coming years.
clashmeifyoucan 6y ago
Agreed. I was in the boat of if it's working, don't change it and just using pip and setuptools over the years. Switched to poetry for one of my libraries as a test a few weeks ago, noticeably more painless! One other thing I liked about it is the community, I distinctly remember digging into the setuptools source code once to find something that was undocumented. With poetry, that was one Discord message away.
- kstrauser 6y ago
  Pip + setuptools is "it's working, no need to change" in the same way that "SVN works so why bother with Git?". It seems fine until you try the alternative, and then the thing it replaces just feels painful to use.
  - clashmeifyoucan 6y ago
    Nice, that somewhat reminds me of urllib3 versus requests. Used to use the former but then requests is/was so much simpler for most stuff.
  - jtdev 6y ago
    In hindsight... maybe git hasn’t solved that many problems.
    - kstrauser 6y ago
      It certainly has issues of its own, and I'm sympathetic to people who prefer the UI of alternatives like Mercurial. But wow, I'd never go back to svn (or god forbid, cvs) ever again.
odiroot 6y ago
> It is rapidly becoming _the_ package manager to use. As a fan of pip's simplicity, I hope this won't become the case.
- dalemyers 6y ago
  What complexity does Poetry have over pip? This is a serious question. Yes, you have to install it on top of your Python install, but other than that?
  - OJFord 6y ago
    And that's only relatively recently become not true of pip. Actually I'm not even sure it is always not true, maybe it depends on package manager (/packager)? python -m ensurepip still exists after all.
abrazensunset 6y ago
In my experience (dependency-heavy data engineering & ML), Poetry is unbearably slow[^1]. Great interface/workflow, though. [1] https://github.com/python-poetry/poetry/issues/2094 https://github.com/python-poetry/poetry/issues/2094
- makeworld 6y ago
  This is not my experience, it seems to be different for different dependencies being installed.
adontz 6y ago
Maybe it is better, but I will never ever install anything via curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - call me security paranoid, but curl | interpreter is not an installation method.
- jklehm 6y ago
  `pipx install poetry` works just as well
- iudqnolq 6y ago
  The exact same thing happens behind the scenes when you pip install something. (Download source from a trusted website, run with interpreter)
- HelloNurse 6y ago
  Particularly, it isn't an installation method someone writing a package manager should consider. Bootstrapping tends to be clumsy, but this is too much.
aden1ne 6y ago
Migrating from poetry 1.0.10 to 1.1.x has unfortunately been major pain for us. Some of our dependencies do not yet fully support PEP517, which means we're stuck on 1.0.10 for now (e.g. mypy, which had a fix merged but not yet released, see https://github.com/python-poetry/poetry/issues/3094 https://github.com/python-poetry/poetry/issues/3094). The poetry lock file also seems to get ignored for git dependencies. Say I depend on package Foo, on branch Bar, as a git dependency. At install time I get revision 1, which gets added to the lock file. Now let's say the head of branch Bar moves to revision 2. If I re-run poetry install, I now get revision 2, even though revision 1 is still mentioned in the lock file. The solution is simple: depend on revisions / tags, rather than on branches (and this sounds like good practice anyway), but it is surprising behavior.
milin 6y ago
I use https://github.com/peterdemin/pip-compile-multi https://github.com/peterdemin/pip-compile-multi. Which uses pip-tools under the hood. It even has a pre-commit-hook to make sure your packages lock files are up to date

skrtskrt 6y ago

Pasting this from another comment: Poetry is still rough around the edges but I think the core experience is great and it's well on its way to be a very popular tool. Before using it at work, we are waiting on waiting on https://github.com/python-poetry/poetry/issues/2610 https://github.com/python-poetry/poetry/issues/2610 (alternate repository not getting used for transitive dependencies) and ideally this https://github.com/python-poetry/poetry/issues/1556 https://github.com/python-poetry/poetry/issues/1556 (disable SSL verify for alternate repositories) If you don't use your own PyPi for a bunch of internal packages, it works great imo. One more wish item would be having absolute path dependencies instead of only relative path.

wp381640 6y ago
I hope the second is never changed - it’s 2020, time to start authenticating the servers you’re downloading and executing code from Internal doesn’t mean secure
- MrOxiMoron 6y ago
  Have to agree, and if it is truly internal why are you doing SSL?
  - zzzeek 6y ago
    SSL for internal services seems to be becoming common these days and that's not a bad thing. Corporate/institutional information warfare is becoming a pretty big deal now (see all the US hospitals being infiltrated) so hardening on the inside to at least slow down an internal threat is not a bad thing at all.

ghshephard 6y ago

I occasionally ask our principle / sr. Python engineers about this, and their response is always, "These things come and go, virtualenv/wrappers + pip + requirements.txt works fine - no need to look at anything else." We've got about 15 repos, with the largest repo containing about 1575 files and 34MBytes of .py source, 14 current developers (with about 40 over the last 10 years) - and they really are quite proficient, but haven't demonstrated any interest at looking at anything outside pip/virtualenv. Is there a reason to look at poetry if you've got the pip/virtualenv combination working fine? People who use poetry seem to love it - so I'm interested in whether it provides any new abilities / flexibility that pip doesn't.

mplewis 6y ago
You're losing the ability to easily update using fuzzy specs and a lockfile if you're using pip vanilla without pip-tools or poetry.
- skrtskrt 6y ago
  To clarify further - lockfile == reproducible builds without having to pip freeze every single dependency in the tree. Fuzzy specs == effortless upgrades according to your risk tolerance for a given library (major version for boto3, minor version for pandas, something like that). Poetry gets you the combination of the two: Let your dep versions float, and easily revert back to a previous deterministic build using the version-controlled lockfile if something breaks.
  - xapata 6y ago
    setup.cfg vs requirements.txt offers this.
    - skrtskrt 6y ago
      Poetry is by no means the only option for this. Lots of people like pip-tools, it would feel a lot more lightweight and closer to pip than Poetry does. Pipenv exists but... steer clear for a multitude of reasons. Personally I like that Poetry centers itself around the pyproject.toml standard. I also think that its usability and the enthusiasm of both the maintainers of the users is going to really carry it more into the Python mainstream in the coming years.

clashmeifyoucan 6y ago

Agreed. I was in the boat of if it's working, don't change it and just using pip and setuptools over the years. Switched to poetry for one of my libraries as a test a few weeks ago, noticeably more painless! One other thing I liked about it is the community, I distinctly remember digging into the setuptools source code once to find something that was undocumented. With poetry, that was one Discord message away.

kstrauser 6y ago
Pip + setuptools is "it's working, no need to change" in the same way that "SVN works so why bother with Git?". It seems fine until you try the alternative, and then the thing it replaces just feels painful to use.
- clashmeifyoucan 6y ago
  Nice, that somewhat reminds me of urllib3 versus requests. Used to use the former but then requests is/was so much simpler for most stuff.
- jtdev 6y ago
  In hindsight... maybe git hasn’t solved that many problems.
  - kstrauser 6y ago
    It certainly has issues of its own, and I'm sympathetic to people who prefer the UI of alternatives like Mercurial. But wow, I'd never go back to svn (or god forbid, cvs) ever again.

odiroot 6y ago

> It is rapidly becoming _the_ package manager to use. As a fan of pip's simplicity, I hope this won't become the case.

dalemyers 6y ago
What complexity does Poetry have over pip? This is a serious question. Yes, you have to install it on top of your Python install, but other than that?
- OJFord 6y ago
  And that's only relatively recently become not true of pip. Actually I'm not even sure it is always not true, maybe it depends on package manager (/packager)? python -m ensurepip still exists after all.

abrazensunset 6y ago

In my experience (dependency-heavy data engineering & ML), Poetry is unbearably slow[^1]. Great interface/workflow, though. [1] https://github.com/python-poetry/poetry/issues/2094 https://github.com/python-poetry/poetry/issues/2094

makeworld 6y ago
This is not my experience, it seems to be different for different dependencies being installed.

adontz 6y ago

Maybe it is better, but I will never ever install anything via curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - call me security paranoid, but curl | interpreter is not an installation method.

jklehm 6y ago
`pipx install poetry` works just as well
iudqnolq 6y ago
The exact same thing happens behind the scenes when you pip install something. (Download source from a trusted website, run with interpreter)
HelloNurse 6y ago
Particularly, it isn't an installation method someone writing a package manager should consider. Bootstrapping tends to be clumsy, but this is too much.

aden1ne 6y ago

Migrating from poetry 1.0.10 to 1.1.x has unfortunately been major pain for us. Some of our dependencies do not yet fully support PEP517, which means we're stuck on 1.0.10 for now (e.g. mypy, which had a fix merged but not yet released, see https://github.com/python-poetry/poetry/issues/3094 https://github.com/python-poetry/poetry/issues/3094). The poetry lock file also seems to get ignored for git dependencies. Say I depend on package Foo, on branch Bar, as a git dependency. At install time I get revision 1, which gets added to the lock file. Now let's say the head of branch Bar moves to revision 2. If I re-run poetry install, I now get revision 2, even though revision 1 is still mentioned in the lock file. The solution is simple: depend on revisions / tags, rather than on branches (and this sounds like good practice anyway), but it is surprising behavior.

milin 6y ago

I use https://github.com/peterdemin/pip-compile-multi https://github.com/peterdemin/pip-compile-multi. Which uses pip-tools under the hood. It even has a pre-commit-hook to make sure your packages lock files are up to date