3 ms·

Interesting! I use miniconda extensively for local development to manage virtual environments for different python versions and love it. I hardly ever actually

Interesting! I use miniconda extensively for local development to manage virtual environments for different python versions and love it. I hardly ever actually use the conda packages though. I assume the main benefit of using these images would be if you are installing from conda repos instead of pip? Otherwise just using the official python images would be as good if not better Edit: I guess if you needed multiple python versions in a single container this would be a good solution for that as well

westurner 6y ago
Use cases for conda or conda+pip: - Already-compiled packages (where there may not be binary wheels) instead of requiring reinstallation and subsequent removal of e.g. build-essentials for every install - Support for R, Julia, NodeJS, Qt, ROS, CUDA, MKL, etc. - Here's what the Kaggle docker-python Dockerfile installs with conda and with pip: https://github.com/Kaggle/docker-python/blob/master/Dockerfile https://github.com/Kaggle/docker-python/blob/master/Dockerfi... - Build matrix in one container with conda envs Disadvantages of the official python images as compared with conda+pip: - Necessary to (re)install build dependencies and a compiler for every build (if there's not a bdist or a wheel for the given architecture) and then uninstall all unnecessary transitive dependencies. This is where a [multi-stage] build of a manylinux wheel may be the best approach. - No LSM (AppArmor, SELinux, ) for one or more processes in the container (which may have read access to /etc or environment variables and/or --privileged) - Necessary to build basically everything on non x86[-64] architectures for every container build Disadvantages of conda / conda+pip: - Different package repo infrastructure to mirror - Users complaining that they don't need conda who then proceed to re-download and re-build wheels locally multiple times a day Additional attributes for comparison: - The new pip solver (which is slower than the traditional iterative non-solver), conda, and mamba - repo2docker (and thus BinderHub) can build an up-to-date container from requirements.txt, environment.yml, install.R, postBuild and any of the other dependency specification formats supported by REES: Reproducible Environment Execution Standard; which may be helpful as Docker Hub images will soon be deleted if they're not retrieved at least once every 6 months (possibly with a GitHub Actions cron task)
Cacti 6y ago
Quite a few conda packages have patches added by the conda team to help fix problems in packages relying on native code or binaries. Particularly on Windows. If something is available on the primary conda repos it will almost assuredly work with few of any problems cross-platform, whereas pip is hit or miss. If you’re always on Linux you may never appreciate it but some pip packages are a nightmare to get working properly on Windows. If you look through the source of the conda repos, you’ll see all kinds of small patches to fix weird and breaking edge cases, particularly in libs with significant C back ends.
- westurner 6y ago
  Here's the meta.yml for the conda-forge/python-feedstock: https://github.com/conda-forge/python-feedstock/blob/master/recipe/meta.yaml https://github.com/conda-forge/python-feedstock/blob/master/... It includes patches just like distro packages often do.

westurner 6y ago

Use cases for conda or conda+pip: - Already-compiled packages (where there may not be binary wheels) instead of requiring reinstallation and subsequent removal of e.g. build-essentials for every install - Support for R, Julia, NodeJS, Qt, ROS, CUDA, MKL, etc. - Here's what the Kaggle docker-python Dockerfile installs with conda and with pip: https://github.com/Kaggle/docker-python/blob/master/Dockerfile https://github.com/Kaggle/docker-python/blob/master/Dockerfi... - Build matrix in one container with conda envs Disadvantages of the official python images as compared with conda+pip: - Necessary to (re)install build dependencies and a compiler for every build (if there's not a bdist or a wheel for the given architecture) and then uninstall all unnecessary transitive dependencies. This is where a [multi-stage] build of a manylinux wheel may be the best approach. - No LSM (AppArmor, SELinux, ) for one or more processes in the container (which may have read access to /etc or environment variables and/or --privileged) - Necessary to build basically everything on non x86[-64] architectures for every container build Disadvantages of conda / conda+pip: - Different package repo infrastructure to mirror - Users complaining that they don't need conda who then proceed to re-download and re-build wheels locally multiple times a day Additional attributes for comparison: - The new pip solver (which is slower than the traditional iterative non-solver), conda, and mamba - repo2docker (and thus BinderHub) can build an up-to-date container from requirements.txt, environment.yml, install.R, postBuild and any of the other dependency specification formats supported by REES: Reproducible Environment Execution Standard; which may be helpful as Docker Hub images will soon be deleted if they're not retrieved at least once every 6 months (possibly with a GitHub Actions cron task)

Cacti 6y ago

Quite a few conda packages have patches added by the conda team to help fix problems in packages relying on native code or binaries. Particularly on Windows. If something is available on the primary conda repos it will almost assuredly work with few of any problems cross-platform, whereas pip is hit or miss. If you’re always on Linux you may never appreciate it but some pip packages are a nightmare to get working properly on Windows. If you look through the source of the conda repos, you’ll see all kinds of small patches to fix weird and breaking edge cases, particularly in libs with significant C back ends.

westurner 6y ago
Here's the meta.yml for the conda-forge/python-feedstock: https://github.com/conda-forge/python-feedstock/blob/master/recipe/meta.yaml https://github.com/conda-forge/python-feedstock/blob/master/... It includes patches just like distro packages often do.