4 ms·

I ship a lot of Python in a CI/CD or devops context, and also deploy it to embedded targets. I've never needed anything more than pip, a venv, and pip-tools (to

I ship a lot of Python in a CI/CD or devops context, and also deploy it to embedded targets. I've never needed anything more than pip, a venv, and pip-tools (to provide pip-compile). Venvs are treated as disposable. The basic workflow I use is: One-time (or as-needed for manual upgrades): 1. Make a venv with setuptools, wheel, and pip-tools (to get pip-compile) installed. 2. Use venv's pip-compile to generate a fully-pinned piptools-requirements.txt for the venv. 3. Check piptools-requirements.txt into my repo. This is used to get a stable, versioned `pip-compile` for use on my payload requirements. During normal development: 1. Add high-level dependencies to a `requirements.in` file. Usually unversioned, unless there's a good reason to specify something more exact. 2. On changes to `requirements.in`, make a venv from `piptools-requirements.txt` and its `pip-compile` to solve `requirements.in` into a fully-pinned `requirements.txt`. 3. Check requirements.in and requirements.txt into the repo. 4. Install packages from requirements.txt when making the venv that I need for production. This approach is very easy to automate, CI/CD friendly, and completely repeatable. It doesn't require any nonstandard tools to deploy (and only needs pip-compile when recompiling requirements.txt). It also makes a clear distinction between "what packages do the developers actually want?" and "what is the fully-versioned set of all dependencies". It's worked great for me over the years, and I'd highly recommend it as a reliable way to use the standard Python package tooling.

sealedservant 2y ago
My problem with pip-tools is this: # pip-tools dependencies dependencies = [ # direct dependencies "build >= 1.0.0", "click >= 8", "pip >= 22.2", "pyproject_hooks", "tomli; python_version < '3.11'", # indirect dependencies "setuptools", # typically needed when pip-tools invokes setup.py "wheel", # pip plugin needed by pip-tools ] pip is nice since it comes out of the box with Python. setuptools used to but now it's gone. The dependency explosion is a huge problem for a lot of people trying to lockdown the security and maintainability of their codebases. I think that's why a lot of people are rallying around Astral's projects like uv and ruff... they do so many things and they do them well.
- nrclark 2y ago
  The nice thing about my approach is that pip-tools is only needed when modifying your requirements, and it also lives in a separate venv/requirements.txt from the dependencies you actually care about. So yes: you need those dependencies in a development context, if you're actively modifying your Python requirements. But they don't make their way into your production requirements.txt, and don't need to be installed anywhere other than a short-lived venv.
alexey-salmin 2y ago
This is the only way I know to get it right with (almost) raw pip. Surprisingly it's not widely known or documented (as can be seen in comments here too), so you need to train the team to use it and then to enforce it. This is easier to achieve with more integrated tools.

sealedservant 2y ago

My problem with pip-tools is this: # pip-tools dependencies dependencies = [ # direct dependencies "build >= 1.0.0", "click >= 8", "pip >= 22.2", "pyproject_hooks", "tomli; python_version < '3.11'", # indirect dependencies "setuptools", # typically needed when pip-tools invokes setup.py "wheel", # pip plugin needed by pip-tools ] pip is nice since it comes out of the box with Python. setuptools used to but now it's gone. The dependency explosion is a huge problem for a lot of people trying to lockdown the security and maintainability of their codebases. I think that's why a lot of people are rallying around Astral's projects like uv and ruff... they do so many things and they do them well.

nrclark 2y ago
The nice thing about my approach is that pip-tools is only needed when modifying your requirements, and it also lives in a separate venv/requirements.txt from the dependencies you actually care about. So yes: you need those dependencies in a development context, if you're actively modifying your Python requirements. But they don't make their way into your production requirements.txt, and don't need to be installed anywhere other than a short-lived venv.

alexey-salmin 2y ago

This is the only way I know to get it right with (almost) raw pip. Surprisingly it's not widely known or documented (as can be seen in comments here too), so you need to train the team to use it and then to enforce it. This is easier to achieve with more integrated tools.