4 ms·

I've been using containers for this, and mounting the source code.

jacquesm 7y ago
I've been using Anaconda and it finally made Python work without endless rounds of 'incompatible libary bingo'. It also works well for machine learning projects.
- tudelo 7y ago
  +1 for Anaconda, worth trying for anyone who has a problem with package versions.
  - echelon 7y ago
    As a software engineer and non-data scientist, I hate Anaconda because it feels like it's a tool that tries to be the be-all, end-all package management tool for everyone in the data science field, yet it feels like a sloppily built, bloated whale. It's even managed to overwrite PATH on some of my Linux machines, which is where I drew the line. I vastly prefer creating hermetic environments with either venv or Docker. They're much cleaner and easier to work with. I wish data scientists would adopt these tools instead. Sadly, many of the ML models I investigate on Github don't even have their package requirements frozen. It's an uphill battle...
    - jacquesm 7y ago
      > I vastly prefer creating hermetic environments with either venv or Docker. They're much cleaner and easier to work with. I wish data scientists would adopt these tools instead. I suspect you have a lot of time on your hands. But for me the 'batteries included' approach really nails it, why repeat the headache over and over again when a single entity can take care of that in such a way that incompatibilities are almost impossible to create? The hardest time I've had was to re-create an environment that ran some python code from a while ago, with Anaconda it was super easy. I'm sure it has its limitations and just like every other tool there are situations where it is best to avoid it but for now it suits me very well.
    - mstokholm 7y ago
      I would suggest you try out Miniconda (https://docs.conda.io/en/latest/miniconda.html https://docs.conda.io/en/latest/miniconda.html). It comes with just the basics, and let's you install TF with GPU support by simply doing: conda install -c anaconda tensorflow-gpu
- dekhn 7y ago
  it's incredibly slow.
  - xxxtentachyon 7y ago
    What’s incredibly slow? Installing things with conda?
    - dekhn 7y ago
      conda install for our environment.yml: about 3-5 minutes solving, then 5-10 minutes installing. pip install with almost exactly the same set of packages: 3-5 minutes total.
- jwilber 7y ago
  Anaconda may work well as a virtual environment for some ml projects, but it is by no means a solution for getting a gpu-working installation of tensorflow on Windows.
  - jacquesm 7y ago
    > getting a gpu-working installation of tensorflow on Windows I think part of your problem is the last word there. Windows is a bad match for such an environment. On Ubuntu it is pretty much painless.
    - jwilber 7y ago
      I agree, and prefer Linux myself, but some clients only allow for solutions based in Windows (and no containers) :/
  - deleted 7y ago
    [deleted]

I've been using Anaconda and it finally made Python work without endless rounds of 'incompatible libary bingo'. It also works well for machine learning projects.

tudelo 7y ago
+1 for Anaconda, worth trying for anyone who has a problem with package versions.
- echelon 7y ago
  As a software engineer and non-data scientist, I hate Anaconda because it feels like it's a tool that tries to be the be-all, end-all package management tool for everyone in the data science field, yet it feels like a sloppily built, bloated whale. It's even managed to overwrite PATH on some of my Linux machines, which is where I drew the line. I vastly prefer creating hermetic environments with either venv or Docker. They're much cleaner and easier to work with. I wish data scientists would adopt these tools instead. Sadly, many of the ML models I investigate on Github don't even have their package requirements frozen. It's an uphill battle...
  - jacquesm 7y ago
    > I vastly prefer creating hermetic environments with either venv or Docker. They're much cleaner and easier to work with. I wish data scientists would adopt these tools instead. I suspect you have a lot of time on your hands. But for me the 'batteries included' approach really nails it, why repeat the headache over and over again when a single entity can take care of that in such a way that incompatibilities are almost impossible to create? The hardest time I've had was to re-create an environment that ran some python code from a while ago, with Anaconda it was super easy. I'm sure it has its limitations and just like every other tool there are situations where it is best to avoid it but for now it suits me very well.
  - mstokholm 7y ago
    I would suggest you try out Miniconda (https://docs.conda.io/en/latest/miniconda.html https://docs.conda.io/en/latest/miniconda.html). It comes with just the basics, and let's you install TF with GPU support by simply doing: conda install -c anaconda tensorflow-gpu
dekhn 7y ago
it's incredibly slow.
- xxxtentachyon 7y ago
  What’s incredibly slow? Installing things with conda?
  - dekhn 7y ago
    conda install for our environment.yml: about 3-5 minutes solving, then 5-10 minutes installing. pip install with almost exactly the same set of packages: 3-5 minutes total.
jwilber 7y ago
Anaconda may work well as a virtual environment for some ml projects, but it is by no means a solution for getting a gpu-working installation of tensorflow on Windows.
- jacquesm 7y ago
  > getting a gpu-working installation of tensorflow on Windows I think part of your problem is the last word there. Windows is a bad match for such an environment. On Ubuntu it is pretty much painless.
  - jwilber 7y ago
    I agree, and prefer Linux myself, but some clients only allow for solutions based in Windows (and no containers) :/
- deleted 7y ago
  [deleted]