3 ms·

What is wrong with pip that people thought we needed conda, and all these other managers?

by bagels 3y ago

duped 3y ago
If you want to learn every lesson a package manager author has learned in the last twenty years by way of counterexample, use pip. Global dependencies by default (fixed by venvs). Shipping precompiled binaries that are not portable across platforms or python versions (kinda fixed by wheels, I forget the PEP number). No lock files with checksum verification by default (kinda fixed by requirements.txt with require hashes, but not really). Also, there's a bunch of weird shit that can happen if the pip version and python version disagree due setup tools. It's got a great interface for installing stuff. But the likelihood that what it installs breaks something on your machine is pretty high the longer you use it. Edit: the real, fundamental problem with pip is that it can't be saved. Too many dockerfiles and server provisioning scripts are relying on it - it's interface, semantics, CLI output, etc. It's not 100% broken, which is why it's still useful. But any true fix has to be done by creating a separate tool.
okanat 3y ago
Imperative dependency (mis)management. Pip resolves what needs to be installed while it installs things depending on the setup script of the package. It requires a full execution and it is neither repeatable nor reproducible. So it is impossible to know what needs to be pulled from Pypi ahead of time. Pip is also strongly tied to the specific Python version. If you upgraded Python interpreter on a system, congrats all virtual environments and all PyPi packages are broken now. Hopefully you saved the minimum set of requirements and they don't have some catastrophic dependency breakage. You run pip install and pray. What? Native binaries? Building them on the fly while installing a pip package? Oh you're brave.
- mixmastamyk 3y ago
  > upgraded Python ... all virtual environments and all PyPi packages are broken now. Misleading. Continue to use the older version, or packages will need to be installed in a new folder for the new version. Recreate venvs and/or simply reinstall any --user.
lolinder 3y ago
A long list of terrible defaults with workarounds that get you halfway to a decent package manager: * Defaults to installing packages globally instead of per project. * requirements.txt feels very much like a second class citizen when you contrast it with package.json, Cargo.toml, build.gradle, pom.xml, or most other dependency systems in other package managers. * No native support for lockfiles short of pinning every version of every project in requirements.txt. This solution is inferior to what is available in ~every other package manager, because a pip freeze doesn't distinguish between explicit dependencies and transitive dependencies. I'm sure there are others, but they're all along the same lines—pip is strictly inferior than basically every other package manager out there for other languages. Things that people expect to be part of a package manager have ugly hacky workarounds instead that aren't uniformly applied across projects.
__MatrixMan__ 3y ago
Have you ever seen a pip failure that required you to go use a different package manager and then come back and try pip again? That's what.
- bagels 3y ago
  I haven't. The only packaging problem I saw deploying python applications was with one particular package that was only available through conda. I was able to build useful things with pip + virtualenv
  - __MatrixMan__ 3y ago
    Ah well I'm happy that you've managed to avoid it but there are a million stack overflow posts just like this one: https://stackoverflow.com/a/34631976 https://stackoverflow.com/a/34631976 Pip fails with some kind of "not found" error and then you have to use some other package manager to supply the missing thing.

What is wrong with pip that people thought we needed conda, and all these other managers?

by bagels 3y ago

What is wrong with pip that people thought we needed conda, and all these other managers?

If you want to learn every lesson a package manager author has learned in the last twenty years by way of counterexample, use pip. Global dependencies by default (fixed by venvs). Shipping precompiled binaries that are not portable across platforms or python versions (kinda fixed by wheels, I forget the PEP number). No lock files with checksum verification by default (kinda fixed by requirements.txt with require hashes, but not really). Also, there's a bunch of weird shit that can happen if the pip version and python version disagree due setup tools. It's got a great interface for installing stuff. But the likelihood that what it installs breaks something on your machine is pretty high the longer you use it. Edit: the real, fundamental problem with pip is that it can't be saved. Too many dockerfiles and server provisioning scripts are relying on it - it's interface, semantics, CLI output, etc. It's not 100% broken, which is why it's still useful. But any true fix has to be done by creating a separate tool.

okanat 3y ago

Imperative dependency (mis)management. Pip resolves what needs to be installed while it installs things depending on the setup script of the package. It requires a full execution and it is neither repeatable nor reproducible. So it is impossible to know what needs to be pulled from Pypi ahead of time. Pip is also strongly tied to the specific Python version. If you upgraded Python interpreter on a system, congrats all virtual environments and all PyPi packages are broken now. Hopefully you saved the minimum set of requirements and they don't have some catastrophic dependency breakage. You run pip install and pray. What? Native binaries? Building them on the fly while installing a pip package? Oh you're brave.

mixmastamyk 3y ago
> upgraded Python ... all virtual environments and all PyPi packages are broken now. Misleading. Continue to use the older version, or packages will need to be installed in a new folder for the new version. Recreate venvs and/or simply reinstall any --user.

lolinder 3y ago

A long list of terrible defaults with workarounds that get you halfway to a decent package manager: * Defaults to installing packages globally instead of per project. * requirements.txt feels very much like a second class citizen when you contrast it with package.json, Cargo.toml, build.gradle, pom.xml, or most other dependency systems in other package managers. * No native support for lockfiles short of pinning every version of every project in requirements.txt. This solution is inferior to what is available in ~every other package manager, because a pip freeze doesn't distinguish between explicit dependencies and transitive dependencies. I'm sure there are others, but they're all along the same lines—pip is strictly inferior than basically every other package manager out there for other languages. Things that people expect to be part of a package manager have ugly hacky workarounds instead that aren't uniformly applied across projects.

__MatrixMan__ 3y ago

Have you ever seen a pip failure that required you to go use a different package manager and then come back and try pip again? That's what.

bagels 3y ago
I haven't. The only packaging problem I saw deploying python applications was with one particular package that was only available through conda. I was able to build useful things with pip + virtualenv
- __MatrixMan__ 3y ago
  Ah well I'm happy that you've managed to avoid it but there are a million stack overflow posts just like this one: https://stackoverflow.com/a/34631976 https://stackoverflow.com/a/34631976 Pip fails with some kind of "not found" error and then you have to use some other package manager to supply the missing thing.