4 ms·
An interesting aspect of this that isn't touched on in the article is about the much-maligned [1] scheme where Debian's Python is patched to look for packages i
by mikepurvis 6y ago
An interesting aspect of this that isn't touched on in the article is about the much-maligned [1] scheme where Debian's Python is patched to look for packages in `dist-packages` instead of `site-packages`. The scenario in the Docker image is I believe the exact one that this is intended to provide sanity for, since you have two Pythons in play— the Debian-packaged python and also the built-from-source Python. Things get hairy once you start using the pip from either of these pythons to start installing pypi packages, especially if those packages or their dependencies have compiled parts which may not work for the other Python. Anyone who has used a Mac with brewed Python has also likely experienced this pain— especially in the days before wheels, where you basically had to use brewed Python for scientific work unless you wanted to compile big packages like numpy from source.
So the deal is that the system-supplied Python can get deps from apt packages, or from pip (into ~/.local, or as root into /usr/local), but either way will install them into a dist-packages directory. This keeps them separate from the packages which the built-from-source Python installs into site-packages.
Upstream Python has put a bunch of pieces in place to address this natively, for example with the "magic" tags that keep compiled assets separate in a directory of packages being potentially used by multiple different interpreters (see https://www.python.org/dev/peps/pep-3147/#proposal https://www.python.org/dev/peps/pep-3147/#proposal). And obviously the ideal solution where possible is to simply use a virtualenv and be totally isolated from the system python. But there are situations where that isn't possible or desirable, such as in this docker container, and so here we are.
1: eg https://github.com/pypa/setuptools/issues/2232 https://github.com/pypa/setuptools/issues/2232
- Kwpolska 6y ago> […] simply use a virtualenv and be totally isolated from the system python. But there are situations where that isn't possible or desirable, such as in this docker container, and so here we are. Why would it not be possible or desirable to use a venv in the Docker container? The tools are available and work fine (`venv` is built into modern Python 3, and they don’t break it unlike Debian). venv will give you separation from anything unusual on the system, and let you safely install your specifically pinned requirements.txt, with zero downsides.