5 ms·
I use anaconda environments and install everything with pip in them. Am I doing it wrong?
by vimy 4y ago
I use anaconda environments and install everything with pip in them. Am I doing it wrong?
- mardifoufs 4y agoThat's basically where we are at for tons of our pipelines, but it kind of defeats the purpose since a dockerfile with a proper base image is basically equivalent at that point.
- disgruntledphd2 4y agoConda is also really annoying to get working in a Dockerfile, because (if I remember correctly) of differences between login and non-login shells.
- nerdponx 4y agoIn general you shouldn't need to "activate" a Conda environment in the shell. Things generally "just work" if you use absolute paths. Something like this: COPY conda.yaml /env/conda.yaml RUN conda env create -f /env/conda.yaml -p /env/conda COPY foo.py /env/foo.py CMD ["/env/conda/bin/python", "/env/foo.py"] What is a little funny is installing a consistent version of Conda inside a container, because the official Miniconda installers are rolling-release only. However you might be able to downgrade to your desired version of Conda after installation.
- blandcoffee 4y agoWhat's the point of this vs. creating your own venv (python -m venv .venv) and having a much smaller library of intentional tools?
- tagh 4y agoEach environment can have its own version of python which downloads with everything else (venv would require separate system installs for this).
- blandcoffee 4y agoCan you clarify what you mean by separate system installs? Today, I manage python via pyenv (local and global state) that let's me create infinite .venvs with different python versions.
- nerdponx 4y agoConda does the same job as Pyenv here, with the additional ability to install system-level libraries, compiler toolchains, etc.
- inconceivable 4y agoi use venv this way. i download and compile specific python versions and install them in a non-system dir with all the other versions. then just run the specific binary to create a venv and it seems to work as expected.
- alexchantavy 4y agoAnaconda automatically handles things like making sure the correct version of cuDNN for your graphics card is installed. When I tried doing this myself with venv it was really painful.
- nerdponx 4y agoThere's nothing wrong with this. IMO, Conda is a general-purpose "system environment" and package manager that happens to be written in Python. The fact that its package ecosystem is oriented towards machine learning with Python is almost an historical coincidence.
- vishal0123 4y agoExactly my setup. I tried to use `conda install` few times, but every time after just few globally installed packages, conda SAT solver always struggles, and I now live with assumption that if incompatible package combination does not throw any error in dev environment, it is likely fine.
- alephxyz 4y agoFYI the libmamba solver released last year is way faster than the classic one at modifying environments.