3 ms·

I respectfully disagree. I find docker to be more-or-less unavoidable in ML work, but thankfully conda is still avoidable. My recommendation is not to touch co

by Y_Y 4y ago

I respectfully disagree. I find docker to be more-or-less unavoidable in ML work, but thankfully conda is still avoidable. My recommendation is not to touch conda, unless you're on Windows and not don't know about cuda drivers or WSL. Otherwise IMHO it's far less trouble to stick with pip (and virtualenv/pipenv/poetry) and it's not difficult to translate the environment.

stared 4y ago
While your millage (clearly) varies from mine, Anaconda is in practice the standard environment for deep learning (and, generally, in most of the Python data science ecosystem). For example, when you go to the front page of PyTorch (https://pytorch.org/ https://pytorch.org/), the default way to go is with Anaconda. It precisely makes it easy to install things regardless of the system and with matching versions. For example, out of box, it gives GPU support for Apple Silicon - not extra installation instructions. Pip installers don't work with non-Python dependencies. Of course, you can manually install things any way you like (including inside Docker), but it is up to you to make sure that all dependencies are compatible. And it is a non-trivial task, given frequent updates of all things involved (including CUDA kernels, Python versions, PyTorch/TF versions, and all libraries related to them one way or the other). I know times before Anaconda matured. Docker was often necessary to make code reusable - and I am grateful I don't need to go back to these times. Of course, now you CAN use Docker. Quite a lot of decent installation scripts place conda (usually miniconda) instructions in a Dockerfile.
- Y_Y 4y ago
  Pip certainly does install plenty of non-python dependencies, but also leaves to the system package manager many of the things managed by conda. I don't know if there's a good definition for "in practice the standard" but I think conda is a heap of shit and I wish people would use it less.

by Y_Y 4y ago

stared 4y ago

While your millage (clearly) varies from mine, Anaconda is in practice the standard environment for deep learning (and, generally, in most of the Python data science ecosystem). For example, when you go to the front page of PyTorch (https://pytorch.org/ https://pytorch.org/), the default way to go is with Anaconda. It precisely makes it easy to install things regardless of the system and with matching versions. For example, out of box, it gives GPU support for Apple Silicon - not extra installation instructions. Pip installers don't work with non-Python dependencies. Of course, you can manually install things any way you like (including inside Docker), but it is up to you to make sure that all dependencies are compatible. And it is a non-trivial task, given frequent updates of all things involved (including CUDA kernels, Python versions, PyTorch/TF versions, and all libraries related to them one way or the other). I know times before Anaconda matured. Docker was often necessary to make code reusable - and I am grateful I don't need to go back to these times. Of course, now you CAN use Docker. Quite a lot of decent installation scripts place conda (usually miniconda) instructions in a Dockerfile.

Y_Y 4y ago
Pip certainly does install plenty of non-python dependencies, but also leaves to the system package manager many of the things managed by conda. I don't know if there's a good definition for "in practice the standard" but I think conda is a heap of shit and I wish people would use it less.