21 ms·

Python Pip 20.3 Released with new resolver

224 points by groodt 6y ago

misnome 6y ago
Since this pull isn’t very helpful, the NEWS file with what’s changed is at https://github.com/pypa/pip/blob/master/NEWS.rst#203-2020-11-30 https://github.com/pypa/pip/blob/master/NEWS.rst#203-2020-11...
- groodt 6y ago
  Permalink to the release notes: https://github.com/pypa/pip/blob/c31c148a5b1d87591862c715adc7a7e5f3242fba/NEWS.rst https://github.com/pypa/pip/blob/c31c148a5b1d87591862c715adc...
tgb 6y ago
The release notes are here [1] and I assume the reason this is being posted is because of: > The new resolver now resolves packages in a deterministic order. (https://github.com/pypa/pip/pull/9100 https://github.com/pypa/pip/pull/9100) [1] https://pip.pypa.io/en/stable/news/#id1 https://pip.pypa.io/en/stable/news/#id1
- groodt 6y ago
  Yes, the new resolver is the highlight. Permalink to the release notes: https://github.com/pypa/pip/blob/c31c148a5b1d87591862c715adc7a7e5f3242fba/NEWS.rst https://github.com/pypa/pip/blob/c31c148a5b1d87591862c715adc...
- pydry 6y ago
  Does it still allow conflicting dependencies?
  - randlet 6y ago
    It refuses to install when there are conflicts now. $ pip install "six<1.12" "virtualenv==20.0.2" -q ERROR: Cannot install six<1.12 and virtualenv==20.0.2 because these package versions have conflicting dependencies. ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
    - deleted 6y ago
      [deleted]
    - rla3rd 6y ago
      only if you are installing the packages simultaneously. it still does the wrong thing if it conflicts with a package already installed
st1x7 6y ago
It's impressive how my pip version seems to always be out of date.
- tsjq 6y ago
  ;-)
  - asah 6y ago
    In all seriousness, the constant warnings mean that users may ignore serious upgrades.
    - mehrdadn 6y ago
      No kidding. I pretty much don't even believe in the concept of a serious upgrade anymore, as far as pip goes anyway. As long as it still works when I run it, nobody is missing out on any serious upgrades as far as I'm concerned at this point.
      - belval 6y ago
        That's a pretty bad mindset, new pip versions (20+) include support for binary distributions that saves a lot of time when installing pretty much anything that needs to be compiled. Just run `pip install --upgrade pip` every month and you will be fine.
        hobofan 6y ago
        Do you have a reference for that? IIRC binary wheels have been supported for ages and not just in 20+ (which was released this year).
        dralley 6y ago
        What the likely mean, is that newer versions of pip support newer platforms underpinning the binary packages. Python binary packages are a bit of a kludge, they take a binary built on a base platform like CentOS 5, and use tools like patchelf to take all the libraries that they dynamically link against and re-wrap them in a new package. Until about a year ago, CentOS 5 was the newest base platform available. So if your library needed a glibc feature or some other library feature from the past decade you were SoL.
muglug 6y ago
Here's a longer article about the resolver: https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-rol...
Chico75 6y ago
So if I understand correctly, pip will now install the list of packages in the same order instead of choosing randomly, so that when there are version conflicts, you always get the same result? I'm surprised I never ran into the issue, but I suppose it mainly show up if you have a large number of dependencies?
- BurningFrog 6y ago
  I'm surprised that isn't the obvious and standard thing to do.
  - tmp538394722 6y ago
    People have to write that code ya know?
- takeda 6y ago
  Is that what it does? My understanding is that it actually was analyzing dependencies before installing finding solution that satisfies all of them. Kind of like what poetry or zypper in SuSE does. If it's just the order, then that's really lame.
cosmic_quanta 6y ago
It's not super relevant, but I was poking around their CI infrastructure, and I noted the use of a temporary RAM disk to speed-up tests: https://github.com/pypa/pip/blob/master/.azure-pipelines/scripts/New-RAMDisk.ps1 https://github.com/pypa/pip/blob/master/.azure-pipelines/scr... I'm very surprised. Is this common?
- dijit 6y ago
  Yes, it has secondary benefits like not burning out SSDs and wearing out HDDs. The drawbacks are that memory is kinda expensive in high amounts, and compilers love memory.
  - acdha 6y ago
    These days, “high amounts” is a lot more than most projects need - even ultra-portables have 8GB or more. Most projects don’t have unit test suites limited on file I/O measured in gigabytes unless they also have a budget.
- detaro 6y ago
  It's a fairly straight-forward optimization if disk is a relevant part of your run time and you don't need terribly much of it. Probably not exactly common, because many people don't bother, but also not extraordinary.
  - sethhochberg 6y ago
    I'd bet its becoming increasingly popular/convenient to do so with the growth of container-based test suites - really trivial to include a "postgres-ram" image for your test stack instead of one that uses disk, and you don't even need to know what tmpfs is to do so.
    - pletnes 6y ago
      Maybe even more so on cloud instances?
    - 6y ago
wokwokwok 6y ago
new version of rust comes out. me: ooo... new shiny toys. new version of pip comes out. again. me: :( this will probably break something. again. I now just tell people to use conda.
- yjftsjthsd-h 6y ago
  What has it broken before? I'm not doubting you, I'm just curious about their failure modes.
  - wokwokwok 6y ago
    Look at the pip issue history. most recently (october?) they vendored some package that was a system dependency before, and it broke the vendored versions of pip (eg. on debian). I get, “not their fault” debian goes and modifies packages... but from a user perspective: it broke. I would say my experience is roughly on every six months something to do with pip breaks for me... but I really cant be bothered trying to keep track of it. I just try to avoid using it now. Down vote all you like, I don't care. Pip has broken my CI enough times its lost any good will it ever had with me.
    - detaro 6y ago
      > and it broke the vendored versions of pip Wait, Debian updated their pip, breaking it in the process without noticing?
  - black3r 6y ago
    it mostly breaks CIs or docker builds if you don't hardwire the pip version, because they tend to use latest pip by default..., It's mostly not directly pip's fault, but it can get annoying. I remember issues when they changed their caching mechanism, or when a couple of libraries I was using were importing internal stuff from pip which got changed in a new version. Also some packages for some reason need to be installed in the correct order and it's not immediately clear until pip tweaks their installation procedure.
zests 6y ago
I prefer this resolver to `pip freeze` type pinning for dependency pull safety. Pip freeze makes it a nightmare to remove old packages if you have hundreds of packages frozen.
- takeda 6y ago
  This doesn't replace pip freeze. There are two kinds of dependencies: - fluid ones, where you specify immediate dependencies of your application with version ranges (typically versions that are api compatible with your app) - locked versions (this is what requirements.txt supposed to be) You get can get this kind of behavior if you define packages in setup.cfg in install_requires and then use pip-compile (from pip-tools) to generate requirements.txt based on it. pip-sync can then synchronize packages to requirements.txt. Alternatively you could just use poetry which does all of this with a nicer interface.
  - zests 6y ago
    This replaces a use case of pip freeze. If you only use pip freeze for that use case then it is a replacement.
Mlller 6y ago
IME, pip and its inclusion in python installations made a great and very positive difference for using Python on Windows: before, third-party installations mostly (sic) didn’t succeed; after, they almost always succeed. I’m grateful.
colechristensen 6y ago
Your package manager should be boring, extremely backward and forward compatible, and never broken. Experience has shown this not to be true for python. Several times over the years i’ve found myself, pinning, upgrading, downgrading, or otherwise juggling versions of setuptools and pip in order to work around some bug. Historically I have had far more problems with the machinery to install python packages I have had with all of the other python packages being installed combined, and that is absurd.
- whalesalad 6y ago
  That is interesting. I haven't worried about setuptools since like 2009 and haven't cared about my pip version since like 2012. I have had FAR more issues with performing Bundler/Ruby upgrades than I've ever had with Python.
  - incanus77 6y ago
    Same here.
  - colechristensen 6y ago
    Here is one from September: https://news.ycombinator.com/item?id=24336058 https://news.ycombinator.com/item?id=24336058
    - user5994461 6y ago
      That one looks like an issue with the debian package trying to alter paths and blowing things up. There's definitely recurring issues with distributions trying to handle python dependencies, which doesn't match how python handles dependencies.
  - LevGoldstein 6y ago
    I take it you're doing a lot of greenfield projects with the latest versions of Python then. I've encountered issues with Setuptools as recently as ~3 months ago: https://github.com/pypa/setuptools/issues/2352 https://github.com/pypa/setuptools/issues/2352 I've also had to play the pip version pinning game a few times in the past.
dalemyers 6y ago
If anyone hasn't seen it, now is a good time to look at https://python-poetry.org/ https://python-poetry.org/ It is rapidly becoming _the_ package manager to use. I've used it in a bunch of personal and professional projects with zero issues. It's been rock solid so far, and I'm definitely a massive fan.
- skrtskrt 6y ago
  Pasting this from another comment: Poetry is still rough around the edges but I think the core experience is great and it's well on its way to be a very popular tool. Before using it at work, we are waiting on waiting on https://github.com/python-poetry/poetry/issues/2610 https://github.com/python-poetry/poetry/issues/2610 (alternate repository not getting used for transitive dependencies) and ideally this https://github.com/python-poetry/poetry/issues/1556 https://github.com/python-poetry/poetry/issues/1556 (disable SSL verify for alternate repositories) If you don't use your own PyPi for a bunch of internal packages, it works great imo. One more wish item would be having absolute path dependencies instead of only relative path.
  - wp381640 6y ago
    I hope the second is never changed - it’s 2020, time to start authenticating the servers you’re downloading and executing code from Internal doesn’t mean secure
    - MrOxiMoron 6y ago
      Have to agree, and if it is truly internal why are you doing SSL?
      - zzzeek 6y ago
        SSL for internal services seems to be becoming common these days and that's not a bad thing. Corporate/institutional information warfare is becoming a pretty big deal now (see all the US hospitals being infiltrated) so hardening on the inside to at least slow down an internal threat is not a bad thing at all.
mhxion 6y ago
Also maybe worth mentioning the PyPI team has 1-to-1 UX feedback/study for `pip` https://www.ei8fdb.org/thoughts/2020/03/pip-ux-study-recruitment/ https://www.ei8fdb.org/thoughts/2020/03/pip-ux-study-recruit.... I'd be more interested opting in for open web survey (question and answer field) though. Nevertheless, great to see they're open to user feedback in forms other than Git issues.
- uranusjr 6y ago
  There are a few of those as well, see https://pip.pypa.io/en/latest/ux_research_design/ https://pip.pypa.io/en/latest/ux_research_design/
thijsvandien 6y ago
Yay! I already got tired of typing --use-feature=2020-resolver. With this, it's a lot easier to upgrade everything without getting conflicts: pip freeze | cut -d= -f1 | xargs pip install --upgrade.
saiadarsh99 6y ago
Oh, wow
optimalsolver 6y ago
Get on Poetry, people. Most Python devs don't seem to realize that the packaging problem is now solved: https://python-poetry.org/ https://python-poetry.org/
- mixmastamyk 6y ago
  I haven’t had a packaging problem in ten years. Maybe because no C extensions, or maybe because I found a pattern that works, not sure.
  - takeda 6y ago
    setup.py (especially if you use declarative setup.cfg) works quite well, you can then use pip-tools to generate requirements.txt which then acts like a lock file. Frankly I would still be using it if it wasn't that PyPA really trying hard to kill it. This forced me to try poetry though and is quite decent frankly. I wish it would support building C packages though and I'm missing plugins like setuptools_scm which generates package version from SCM (e.g. git) tags.
dang 6y ago
We changed the URL from https://github.com/pypa/pip/pull/9177 https://github.com/pypa/pip/pull/9177 to the first link that the main comment there (https://github.com/pypa/pip/pull/9177#issuecomment-735830828 https://github.com/pypa/pip/pull/9177#issuecomment-735830828) is pointing HN readers to. I presume that's the most informative one.
groodt 6y ago
Seen a few mentions of poetry. Not many for pip-tools which has been around longer, is less opinionated and has many of the same benefits https://github.com/jazzband/pip-tools https://github.com/jazzband/pip-tools
- dzonga 6y ago
  poetry is slow. and ultimately uses pip underneath. yeah python packaging needs to be fixed. but using a virtualenv + pip | pip-tools goes a long way.
jdeibele 6y ago
pip-review works great for keeping packages up to date. https://pypi.org/project/pip-review/ https://pypi.org/project/pip-review/ > pip-review Faker==4.18.0 is available (you have 4.17.1) pip==20.3 is available (you have 20.2.4) > pip-review --auto --verbose Collecting Faker==4.18.0 Downloading Faker-4.18.0-py3-none-any.whl (1.1 MB) || 1.1 MB 730 kB/s Collecting pip==20.3 Downloading pip-20.3-py2.py3-none-any.whl (1.5 MB) || 1.5 MB 2.0 MB/s Requirement already satisfied: python-dateutil>=2.4 in /usr/local/lib/python3.8/site-packages (from Faker==4.18.0) (2.8.1) Requirement already satisfied: text-unidecode==1.3 in /usr/local/lib/python3.8/site-packages (from Faker==4.18.0) (1.3) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/site-packages (from python-dateutil>=2.4->Faker==4.18.0) (1.15.0) Installing collected packages: Faker, pip Attempting uninstall: Faker Found existing installation: Faker 4.17.1 Uninstalling Faker-4.17.1: Successfully uninstalled Faker-4.17.1 Attempting uninstall: pip Found existing installation: pip 20.2.4 Uninstalling pip-20.2.4: Successfully uninstalled pip-20.2.4 ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts. We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default. lektor 3.2.0 requires Werkzeug<1, but you'll have werkzeug 1.0.1 which is incompatible. Successfully installed Faker-4.18.0 pip-20.3
economusty 6y ago
I just switched to pipes, ugh
devy 6y ago
Hynek, a CPython committer, had written a blog post[1] the state of Python application dependencies in 2018, updated in 2019 (no change in 2020, I asked). It was also surfaced on HN 3 times but did not get much attention[2] [1]: https://hynek.me/articles/python-app-deps-2018/#petry https://hynek.me/articles/python-app-deps-2018/#petry [2]: https://hn.algolia.com/?q=python-app-deps-2018 https://hn.algolia.com/?q=python-app-deps-2018
iverjo 6y ago
Sadly, pip 20.3 seems to have broken docker builds in one of my projects. The symptom is that the pip install seems to hang indefinitely (>40000 seconds). I switched back to 20.2 for now.

224 points by groodt 6y ago

Since this pull isn’t very helpful, the NEWS file with what’s changed is at https://github.com/pypa/pip/blob/master/NEWS.rst#203-2020-11-30 https://github.com/pypa/pip/blob/master/NEWS.rst#203-2020-11...

groodt 6y ago
Permalink to the release notes: https://github.com/pypa/pip/blob/c31c148a5b1d87591862c715adc7a7e5f3242fba/NEWS.rst https://github.com/pypa/pip/blob/c31c148a5b1d87591862c715adc...

tgb 6y ago

The release notes are here [1] and I assume the reason this is being posted is because of: > The new resolver now resolves packages in a deterministic order. (https://github.com/pypa/pip/pull/9100 https://github.com/pypa/pip/pull/9100) [1] https://pip.pypa.io/en/stable/news/#id1 https://pip.pypa.io/en/stable/news/#id1

groodt 6y ago
Yes, the new resolver is the highlight. Permalink to the release notes: https://github.com/pypa/pip/blob/c31c148a5b1d87591862c715adc7a7e5f3242fba/NEWS.rst https://github.com/pypa/pip/blob/c31c148a5b1d87591862c715adc...
pydry 6y ago
Does it still allow conflicting dependencies?
- randlet 6y ago
  It refuses to install when there are conflicts now. $ pip install "six<1.12" "virtualenv==20.0.2" -q ERROR: Cannot install six<1.12 and virtualenv==20.0.2 because these package versions have conflicting dependencies. ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
  - deleted 6y ago
    [deleted]
  - rla3rd 6y ago
    only if you are installing the packages simultaneously. it still does the wrong thing if it conflicts with a package already installed

st1x7 6y ago

It's impressive how my pip version seems to always be out of date.

tsjq 6y ago
;-)
- asah 6y ago
  In all seriousness, the constant warnings mean that users may ignore serious upgrades.
  - mehrdadn 6y ago
    No kidding. I pretty much don't even believe in the concept of a serious upgrade anymore, as far as pip goes anyway. As long as it still works when I run it, nobody is missing out on any serious upgrades as far as I'm concerned at this point.
    - belval 6y ago
      That's a pretty bad mindset, new pip versions (20+) include support for binary distributions that saves a lot of time when installing pretty much anything that needs to be compiled. Just run `pip install --upgrade pip` every month and you will be fine.
      - hobofan 6y ago
        Do you have a reference for that? IIRC binary wheels have been supported for ages and not just in 20+ (which was released this year).
        dralley 6y ago
        What the likely mean, is that newer versions of pip support newer platforms underpinning the binary packages. Python binary packages are a bit of a kludge, they take a binary built on a base platform like CentOS 5, and use tools like patchelf to take all the libraries that they dynamically link against and re-wrap them in a new package. Until about a year ago, CentOS 5 was the newest base platform available. So if your library needed a glibc feature or some other library feature from the past decade you were SoL.

muglug 6y ago

Here's a longer article about the resolver: https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-rol...

Chico75 6y ago

So if I understand correctly, pip will now install the list of packages in the same order instead of choosing randomly, so that when there are version conflicts, you always get the same result? I'm surprised I never ran into the issue, but I suppose it mainly show up if you have a large number of dependencies?

BurningFrog 6y ago
I'm surprised that isn't the obvious and standard thing to do.
- tmp538394722 6y ago
  People have to write that code ya know?
takeda 6y ago
Is that what it does? My understanding is that it actually was analyzing dependencies before installing finding solution that satisfies all of them. Kind of like what poetry or zypper in SuSE does. If it's just the order, then that's really lame.

cosmic_quanta 6y ago

It's not super relevant, but I was poking around their CI infrastructure, and I noted the use of a temporary RAM disk to speed-up tests: https://github.com/pypa/pip/blob/master/.azure-pipelines/scripts/New-RAMDisk.ps1 https://github.com/pypa/pip/blob/master/.azure-pipelines/scr... I'm very surprised. Is this common?

dijit 6y ago
Yes, it has secondary benefits like not burning out SSDs and wearing out HDDs. The drawbacks are that memory is kinda expensive in high amounts, and compilers love memory.
- acdha 6y ago
  These days, “high amounts” is a lot more than most projects need - even ultra-portables have 8GB or more. Most projects don’t have unit test suites limited on file I/O measured in gigabytes unless they also have a budget.
detaro 6y ago
It's a fairly straight-forward optimization if disk is a relevant part of your run time and you don't need terribly much of it. Probably not exactly common, because many people don't bother, but also not extraordinary.
- sethhochberg 6y ago
  I'd bet its becoming increasingly popular/convenient to do so with the growth of container-based test suites - really trivial to include a "postgres-ram" image for your test stack instead of one that uses disk, and you don't even need to know what tmpfs is to do so.
  - pletnes 6y ago
    Maybe even more so on cloud instances?
  - 6y ago

wokwokwok 6y ago

new version of rust comes out. me: ooo... new shiny toys. new version of pip comes out. again. me: :( this will probably break something. again. I now just tell people to use conda.

yjftsjthsd-h 6y ago
What has it broken before? I'm not doubting you, I'm just curious about their failure modes.
- wokwokwok 6y ago
  Look at the pip issue history. most recently (october?) they vendored some package that was a system dependency before, and it broke the vendored versions of pip (eg. on debian). I get, “not their fault” debian goes and modifies packages... but from a user perspective: it broke. I would say my experience is roughly on every six months something to do with pip breaks for me... but I really cant be bothered trying to keep track of it. I just try to avoid using it now. Down vote all you like, I don't care. Pip has broken my CI enough times its lost any good will it ever had with me.
  - detaro 6y ago
    > and it broke the vendored versions of pip Wait, Debian updated their pip, breaking it in the process without noticing?
- black3r 6y ago
  it mostly breaks CIs or docker builds if you don't hardwire the pip version, because they tend to use latest pip by default..., It's mostly not directly pip's fault, but it can get annoying. I remember issues when they changed their caching mechanism, or when a couple of libraries I was using were importing internal stuff from pip which got changed in a new version. Also some packages for some reason need to be installed in the correct order and it's not immediately clear until pip tweaks their installation procedure.

zests 6y ago

I prefer this resolver to `pip freeze` type pinning for dependency pull safety. Pip freeze makes it a nightmare to remove old packages if you have hundreds of packages frozen.

takeda 6y ago
This doesn't replace pip freeze. There are two kinds of dependencies: - fluid ones, where you specify immediate dependencies of your application with version ranges (typically versions that are api compatible with your app) - locked versions (this is what requirements.txt supposed to be) You get can get this kind of behavior if you define packages in setup.cfg in install_requires and then use pip-compile (from pip-tools) to generate requirements.txt based on it. pip-sync can then synchronize packages to requirements.txt. Alternatively you could just use poetry which does all of this with a nicer interface.
- zests 6y ago
  This replaces a use case of pip freeze. If you only use pip freeze for that use case then it is a replacement.

Mlller 6y ago

IME, pip and its inclusion in python installations made a great and very positive difference for using Python on Windows: before, third-party installations mostly (sic) didn’t succeed; after, they almost always succeed. I’m grateful.

colechristensen 6y ago

Your package manager should be boring, extremely backward and forward compatible, and never broken. Experience has shown this not to be true for python. Several times over the years i’ve found myself, pinning, upgrading, downgrading, or otherwise juggling versions of setuptools and pip in order to work around some bug. Historically I have had far more problems with the machinery to install python packages I have had with all of the other python packages being installed combined, and that is absurd.

whalesalad 6y ago
That is interesting. I haven't worried about setuptools since like 2009 and haven't cared about my pip version since like 2012. I have had FAR more issues with performing Bundler/Ruby upgrades than I've ever had with Python.
- incanus77 6y ago
  Same here.
- colechristensen 6y ago
  Here is one from September: https://news.ycombinator.com/item?id=24336058 https://news.ycombinator.com/item?id=24336058
  - user5994461 6y ago
    That one looks like an issue with the debian package trying to alter paths and blowing things up. There's definitely recurring issues with distributions trying to handle python dependencies, which doesn't match how python handles dependencies.
- LevGoldstein 6y ago
  I take it you're doing a lot of greenfield projects with the latest versions of Python then. I've encountered issues with Setuptools as recently as ~3 months ago: https://github.com/pypa/setuptools/issues/2352 https://github.com/pypa/setuptools/issues/2352 I've also had to play the pip version pinning game a few times in the past.

dalemyers 6y ago

If anyone hasn't seen it, now is a good time to look at https://python-poetry.org/ https://python-poetry.org/ It is rapidly becoming _the_ package manager to use. I've used it in a bunch of personal and professional projects with zero issues. It's been rock solid so far, and I'm definitely a massive fan.

skrtskrt 6y ago
Pasting this from another comment: Poetry is still rough around the edges but I think the core experience is great and it's well on its way to be a very popular tool. Before using it at work, we are waiting on waiting on https://github.com/python-poetry/poetry/issues/2610 https://github.com/python-poetry/poetry/issues/2610 (alternate repository not getting used for transitive dependencies) and ideally this https://github.com/python-poetry/poetry/issues/1556 https://github.com/python-poetry/poetry/issues/1556 (disable SSL verify for alternate repositories) If you don't use your own PyPi for a bunch of internal packages, it works great imo. One more wish item would be having absolute path dependencies instead of only relative path.
- wp381640 6y ago
  I hope the second is never changed - it’s 2020, time to start authenticating the servers you’re downloading and executing code from Internal doesn’t mean secure
  - MrOxiMoron 6y ago
    Have to agree, and if it is truly internal why are you doing SSL?
    - zzzeek 6y ago
      SSL for internal services seems to be becoming common these days and that's not a bad thing. Corporate/institutional information warfare is becoming a pretty big deal now (see all the US hospitals being infiltrated) so hardening on the inside to at least slow down an internal threat is not a bad thing at all.

mhxion 6y ago

Also maybe worth mentioning the PyPI team has 1-to-1 UX feedback/study for `pip` https://www.ei8fdb.org/thoughts/2020/03/pip-ux-study-recruitment/ https://www.ei8fdb.org/thoughts/2020/03/pip-ux-study-recruit.... I'd be more interested opting in for open web survey (question and answer field) though. Nevertheless, great to see they're open to user feedback in forms other than Git issues.

uranusjr 6y ago
There are a few of those as well, see https://pip.pypa.io/en/latest/ux_research_design/ https://pip.pypa.io/en/latest/ux_research_design/

thijsvandien 6y ago

Yay! I already got tired of typing --use-feature=2020-resolver. With this, it's a lot easier to upgrade everything without getting conflicts: pip freeze | cut -d= -f1 | xargs pip install --upgrade.

saiadarsh99 6y ago

Oh, wow

optimalsolver 6y ago

Get on Poetry, people. Most Python devs don't seem to realize that the packaging problem is now solved: https://python-poetry.org/ https://python-poetry.org/

mixmastamyk 6y ago
I haven’t had a packaging problem in ten years. Maybe because no C extensions, or maybe because I found a pattern that works, not sure.
- takeda 6y ago
  setup.py (especially if you use declarative setup.cfg) works quite well, you can then use pip-tools to generate requirements.txt which then acts like a lock file. Frankly I would still be using it if it wasn't that PyPA really trying hard to kill it. This forced me to try poetry though and is quite decent frankly. I wish it would support building C packages though and I'm missing plugins like setuptools_scm which generates package version from SCM (e.g. git) tags.

dang 6y ago

We changed the URL from https://github.com/pypa/pip/pull/9177 https://github.com/pypa/pip/pull/9177 to the first link that the main comment there (https://github.com/pypa/pip/pull/9177#issuecomment-735830828 https://github.com/pypa/pip/pull/9177#issuecomment-735830828) is pointing HN readers to. I presume that's the most informative one.

groodt 6y ago

Seen a few mentions of poetry. Not many for pip-tools which has been around longer, is less opinionated and has many of the same benefits https://github.com/jazzband/pip-tools https://github.com/jazzband/pip-tools

dzonga 6y ago
poetry is slow. and ultimately uses pip underneath. yeah python packaging needs to be fixed. but using a virtualenv + pip | pip-tools goes a long way.

jdeibele 6y ago

pip-review works great for keeping packages up to date. https://pypi.org/project/pip-review/ https://pypi.org/project/pip-review/ > pip-review Faker==4.18.0 is available (you have 4.17.1) pip==20.3 is available (you have 20.2.4) > pip-review --auto --verbose Collecting Faker==4.18.0 Downloading Faker-4.18.0-py3-none-any.whl (1.1 MB) || 1.1 MB 730 kB/s Collecting pip==20.3 Downloading pip-20.3-py2.py3-none-any.whl (1.5 MB) || 1.5 MB 2.0 MB/s Requirement already satisfied: python-dateutil>=2.4 in /usr/local/lib/python3.8/site-packages (from Faker==4.18.0) (2.8.1) Requirement already satisfied: text-unidecode==1.3 in /usr/local/lib/python3.8/site-packages (from Faker==4.18.0) (1.3) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/site-packages (from python-dateutil>=2.4->Faker==4.18.0) (1.15.0) Installing collected packages: Faker, pip Attempting uninstall: Faker Found existing installation: Faker 4.17.1 Uninstalling Faker-4.17.1: Successfully uninstalled Faker-4.17.1 Attempting uninstall: pip Found existing installation: pip 20.2.4 Uninstalling pip-20.2.4: Successfully uninstalled pip-20.2.4 ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts. We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default. lektor 3.2.0 requires Werkzeug<1, but you'll have werkzeug 1.0.1 which is incompatible. Successfully installed Faker-4.18.0 pip-20.3

economusty 6y ago

I just switched to pipes, ugh

devy 6y ago

Hynek, a CPython committer, had written a blog post[1] the state of Python application dependencies in 2018, updated in 2019 (no change in 2020, I asked). It was also surfaced on HN 3 times but did not get much attention[2] [1]: https://hynek.me/articles/python-app-deps-2018/#petry https://hynek.me/articles/python-app-deps-2018/#petry [2]: https://hn.algolia.com/?q=python-app-deps-2018 https://hn.algolia.com/?q=python-app-deps-2018

iverjo 6y ago

Sadly, pip 20.3 seems to have broken docker builds in one of my projects. The symptom is that the pip install seems to hang indefinitely (>40000 seconds). I switched back to 20.2 for now.