5 ms·

Anaconda does most of the stuff mentioned, and also makes it much easier to install packages based on C/C++ libraries (which most deep learning things are). So

Anaconda does most of the stuff mentioned, and also makes it much easier to install packages based on C/C++ libraries (which most deep learning things are). So you're better off staying with anaconda. It's widely used in commercial data science projects so the idea that noone "takes it seriously" as someone else suggests is a bit silly. I assume they're thinking about a different context to data science projects. That said anaconda does have a whole variety of extremely annoying quirks, like packages not being backwards compatible with old versions of conda, or conda going crazy and reinstalling itself, or the way the conda-forge repo has far more packages than the official conda repo. It's very far from perfect. But for data science I think it's basically the standard package manager in Python land.

Rotareti 9y ago
> ... and also makes it much easier to install packages based on C/C++ libraries I hear this often, though I cannot remember ever running into a pip package where this was an issue. Out of curiosity, could someone point me to a pip package and its conda equivalent where this is the case?
- jhayward 9y ago
  Install scipy on Windows and include the Intel linear matrix libraries.
  - akx 9y ago
    The numpy+mkl wheel is available on Gohlke's site.
- _coveredInBees 9y ago
  Try pip installing scipy or Numpy and you'll see the value of conda.
  - zeptomu 9y ago
    Although I mostly use the system's numpy package (usually shipped with the distribution), I just tried installing it via pip and had no problem (obviously just one data point).
  - wirrbel 9y ago
    As if that still was a problem, now that we have wheels, and especially manylinux wheels.
    - pwang 9y ago
      See Myth #6 here: http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/#Myth-#6:-Now-that-pip-uses-wheels,-conda-is-no-longer-necessary http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-mis...
      - wirrbel 9y ago
        Valid points. Still most of them are not issues in my daily work. And by the way Peter, thank you for the amazing work you do.
  - Rotareti 9y ago
    Just ran `pipenv install scipy numpy`. Installs like any other package..
- johnminter 9y ago
  I really like conda. It is easy to have multiple enviroments and minimizes the footprint by symbolically linking in what is needed.
- pwang 9y ago
  This is also a useful read: http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/ http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-mis...
- dagw 9y ago
  Rasterio, fiona and anything else GDAL based always causes me headaches.
wirrbel 9y ago
Anaconda was the competition pip needed to become good. When Anaconda was introduced, I (a pip person), was impressed. However, at work we had pip workflows that were working okayish, so I never made the switch. Today, pip et. al. has so dramatically improved, that I hardly see a reason to use anaconda. I am not using deep learning stuff, so I cannot comment on this, but for most scientific python stuff (scikit learn, pandas, numpy, etc.). Pip and pre-built wheels work very well. On my new job, I was given a Windows laptop that I happen to use now for my development work (because I am too stupid/lazy to properly configure and maintain a separate virtual linux machine). And I started with anaconda, assuming it was less trouble, but quickly ran into trouble. pip worked like a charm []. [] Ironically pip is installed via the anaconda base installation if I am not mistaken :)
- james_the_cat 9y ago
  Tensorflow has a pip wheel, so probably pip works just as well as conda. Maybe mkl is easier with conda, though then again I constantly install the "nomkl" meta package in conda because it adds hundreds of megabytes to Docker images and doesn't seem to have any positive effect for anything I've worked on. As you say, they're now sufficiently similar that there's no big reason to switch from one to the other...
- gonvaled 9y ago
  I could provide a vagrant based repo that lets you spawn your ubuntu VM for dev in minutes. I'll try to get it in github.
- beagle3 9y ago
  Nothing ironic. Anaconda integrates pip, it doesn't not compete with it. I find that Anaconda is the best among the virtual python envs I tried; It does as good or better job of separation and tracking installations as any, AND falls back to pip (with complete integration and tracking) when a package is not in the conda repos. pip inside conda works better than pip outside in my opinion. I don't understand the general sentiment towards (ana)conda.
kalefranz 9y ago
> like packages not being backwards compatible with old versions of conda When new capabilities are added to conda, like the ability to support noarch python packages, you're going to need to be working with an up-to-date version of conda to use those packages. Just `conda update conda`. > conda going crazy and reinstalling itself Conda will pretty aggressively auto-update itself. Going crazy and reinstalling is a new one though. File an issue with details at https://github.com/conda/conda https://github.com/conda/conda > the way the conda-forge repo has far more packages than the official conda repo Conda-forge is community driven and more "upstream" than the Anaconda, Inc.-provided 'defaults' repositories. Think of conda-forge like Fedora, and 'defaults' like CentOS/RHEL.

Rotareti 9y ago

> ... and also makes it much easier to install packages based on C/C++ libraries I hear this often, though I cannot remember ever running into a pip package where this was an issue. Out of curiosity, could someone point me to a pip package and its conda equivalent where this is the case?

jhayward 9y ago
Install scipy on Windows and include the Intel linear matrix libraries.
- akx 9y ago
  The numpy+mkl wheel is available on Gohlke's site.
_coveredInBees 9y ago
Try pip installing scipy or Numpy and you'll see the value of conda.
- zeptomu 9y ago
  Although I mostly use the system's numpy package (usually shipped with the distribution), I just tried installing it via pip and had no problem (obviously just one data point).
- wirrbel 9y ago
  As if that still was a problem, now that we have wheels, and especially manylinux wheels.
  - pwang 9y ago
    See Myth #6 here: http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/#Myth-#6:-Now-that-pip-uses-wheels,-conda-is-no-longer-necessary http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-mis...
    - wirrbel 9y ago
      Valid points. Still most of them are not issues in my daily work. And by the way Peter, thank you for the amazing work you do.
- Rotareti 9y ago
  Just ran `pipenv install scipy numpy`. Installs like any other package..
johnminter 9y ago
I really like conda. It is easy to have multiple enviroments and minimizes the footprint by symbolically linking in what is needed.
pwang 9y ago
This is also a useful read: http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/ http://jakevdp.github.io/blog/2016/08/25/conda-myths-and-mis...
dagw 9y ago
Rasterio, fiona and anything else GDAL based always causes me headaches.

wirrbel 9y ago

Anaconda was the competition pip needed to become good. When Anaconda was introduced, I (a pip person), was impressed. However, at work we had pip workflows that were working okayish, so I never made the switch. Today, pip et. al. has so dramatically improved, that I hardly see a reason to use anaconda. I am not using deep learning stuff, so I cannot comment on this, but for most scientific python stuff (scikit learn, pandas, numpy, etc.). Pip and pre-built wheels work very well. On my new job, I was given a Windows laptop that I happen to use now for my development work (because I am too stupid/lazy to properly configure and maintain a separate virtual linux machine). And I started with anaconda, assuming it was less trouble, but quickly ran into trouble. pip worked like a charm []. [] Ironically pip is installed via the anaconda base installation if I am not mistaken :)

james_the_cat 9y ago
Tensorflow has a pip wheel, so probably pip works just as well as conda. Maybe mkl is easier with conda, though then again I constantly install the "nomkl" meta package in conda because it adds hundreds of megabytes to Docker images and doesn't seem to have any positive effect for anything I've worked on. As you say, they're now sufficiently similar that there's no big reason to switch from one to the other...
gonvaled 9y ago
I could provide a vagrant based repo that lets you spawn your ubuntu VM for dev in minutes. I'll try to get it in github.
beagle3 9y ago
Nothing ironic. Anaconda integrates pip, it doesn't not compete with it. I find that Anaconda is the best among the virtual python envs I tried; It does as good or better job of separation and tracking installations as any, AND falls back to pip (with complete integration and tracking) when a package is not in the conda repos. pip inside conda works better than pip outside in my opinion. I don't understand the general sentiment towards (ana)conda.

kalefranz 9y ago

> like packages not being backwards compatible with old versions of conda When new capabilities are added to conda, like the ability to support noarch python packages, you're going to need to be working with an up-to-date version of conda to use those packages. Just `conda update conda`. > conda going crazy and reinstalling itself Conda will pretty aggressively auto-update itself. Going crazy and reinstalling is a new one though. File an issue with details at https://github.com/conda/conda https://github.com/conda/conda > the way the conda-forge repo has far more packages than the official conda repo Conda-forge is community driven and more "upstream" than the Anaconda, Inc.-provided 'defaults' repositories. Think of conda-forge like Fedora, and 'defaults' like CentOS/RHEL.