3 ms·

Use pip-compile[1] to generate a lock file and pip to install it. Update said lock file periodically to pull in package updates. Simple and effective in my ex

Use pip-compile[1] to generate a lock file and pip to install it. Update said lock file periodically to pull in package updates. Simple and effective in my experience. How is this falling short? Why such a push for different tooling? Why the continued complaints about Python packaging being substandard? Honest questions because I've been using this for web and cli applications for years without issue. I also use pipx[2] to install Python based tools onto my system so they have their own virtualenv. 1: https://github.com/jazzband/pip-tools https://github.com/jazzband/pip-tools 2: https://github.com/pypa/pipx https://github.com/pypa/pipx

Phlogistique 4y ago
pip-compile does not use the new Pip dependency resolver, and as a result I have gotten in situations where Pip could resolve dependencies, but pip-compile would need manual pinning of many dependencies to do the same. (on one project at work it became untenable and people started updating the lockfile manually)
- rsyring 4y ago
  That's helpful to know. Looks like an update to use the new resolver is close: https://github.com/jazzband/pip-tools/pull/1539 https://github.com/jazzband/pip-tools/pull/1539
epage 4y ago
pip-compile only generates it for your current platform and python version. Poetry generates a lock file for all platforms and python versions.
- groodt 4y ago
  This is true. However if the transitive-closure in the dependencies contains an sdist package anywhere in the tree, written in a way that wasn't designed to be imported on all platforms (this is rare, but does happen), then it isn't possible to resolve cross platform. The prevalence of wheels does alleviate this to some extent. I think it is one of the early design decisions that pip can only install/resolve for the current platform and this will be very difficult to unwind. There are movements slowly in a direction that may one day make it possible to statically resolve python packages and remove dynamic dependency metadata or builds happening during dependency resolution.
  - ofek 4y ago
    https://peps.python.org/pep-0643/ https://peps.python.org/pep-0643/ will help the situation eventually

Phlogistique 4y ago

pip-compile does not use the new Pip dependency resolver, and as a result I have gotten in situations where Pip could resolve dependencies, but pip-compile would need manual pinning of many dependencies to do the same. (on one project at work it became untenable and people started updating the lockfile manually)

rsyring 4y ago
That's helpful to know. Looks like an update to use the new resolver is close: https://github.com/jazzband/pip-tools/pull/1539 https://github.com/jazzband/pip-tools/pull/1539

epage 4y ago

pip-compile only generates it for your current platform and python version. Poetry generates a lock file for all platforms and python versions.

groodt 4y ago
This is true. However if the transitive-closure in the dependencies contains an sdist package anywhere in the tree, written in a way that wasn't designed to be imported on all platforms (this is rare, but does happen), then it isn't possible to resolve cross platform. The prevalence of wheels does alleviate this to some extent. I think it is one of the early design decisions that pip can only install/resolve for the current platform and this will be very difficult to unwind. There are movements slowly in a direction that may one day make it possible to statically resolve python packages and remove dynamic dependency metadata or builds happening during dependency resolution.
- ofek 4y ago
  https://peps.python.org/pep-0643/ https://peps.python.org/pep-0643/ will help the situation eventually