3 ms·

This is why I started using Conda recently. It works quite well to create isolated python environments, having many prebuilt packages, and most importantly, sys

This is why I started using Conda recently. It works quite well to create isolated python environments, having many prebuilt packages, and most importantly, system level dependencies. So, you can install some of those, and for anything that is not in conda-forge, pip install works and builds.

dnautics 6y ago
just be careful. I don't know if things have improved in conda-land, but when I last used it a few years ago I table flipped and quit conda (and, ultimately, python) when the system-level dependency management got so broken that I was forced to reinstall linux from scratch and start COMPLETELY over (it had broken things outside of its domain). Tensorflow being hard-pinned to CUDA versions probably didn't help.
- xapata 6y ago
  I've been using conda for quite some time, maybe version 0, I forget, and this has never happened to me. I hope you submitted an issue.
- icedistilled 6y ago
  Having gone through the nightmare of setting up tensor flow with cuda, I can't help but think you're letting that color your view of python as a whole. Tensorflow, cuda, and nvidia were a nightmare that took me 5 days to resolve on ubuntu 18 since it was my first time setting it up ever. There was literally no working guides or people posting with my exact issue, but tons of posts about other people having a nightmare of a time. In the end I figured out I had to go find this one file/library nvidia had moved around between versions of drivers/cuda even though documentation suggested they hadn't and I was using the versions that were supposed to work fine. In the end, fixing it was either by setting an environment variable because the default path was wrong or editing the path in a config file. I'm not remembering which now.
  - dnautics 6y ago
    I dunno, I think the problem is not having a project-specific location for your deps. Obviously you want global deps for a console scripting language, but if you are doing any sort of bigger project, having a global deps location (which is implicitly resolved by the language, no less) is unacceptable for reproducible development for larger projects today. The env hack that python managers like conda and venv use is, IMO, extremely dangerous, because it's hiding state. These days I use languages that don't do this for my daily drivers, it's better for my sanity.
xapata 6y ago
I've had a better experience using Conda for just Python and Pip, and anything Pip doesn't handle as I wish, then using Pip for the bulk of the dependency tree. Pip's new resolver should make this even better.
- systemvoltage 6y ago
  Have you tried poetry? It's pretty good.
  - xapata 6y ago
    Not yet. I'm happy enough with my setup.
mbreese 6y ago
Some people I work with have had major issues with Conda packaging. Sometimes it works, sometimes not. This can be especially tricky when you’re working with system libraries with competing versions. I still have no idea how a particular set of tools has ever run, given their configuration. The exact same Conda packages work on a local Mac, but fail on a Linux server. It can be frustrating. Because of this, I try to avoid it as much as possible. I think Conda is trying too hard to hit the middle ground between full containers (Docker or Singularity) and virtualenv’s. If your program is too complicated to be managed with a venv, it’s better to package it up as a full container and be done with it. Also, I don’t think Conda would have helped here when the problem is lack of specific Python versions installed at the system level. On a Mac (and everywhere, really), I try my best to avoid the main system Python install and use a user-local (compiled) install when I need a specific version that isn’t available by default.

dnautics 6y ago

just be careful. I don't know if things have improved in conda-land, but when I last used it a few years ago I table flipped and quit conda (and, ultimately, python) when the system-level dependency management got so broken that I was forced to reinstall linux from scratch and start COMPLETELY over (it had broken things outside of its domain). Tensorflow being hard-pinned to CUDA versions probably didn't help.

xapata 6y ago
I've been using conda for quite some time, maybe version 0, I forget, and this has never happened to me. I hope you submitted an issue.
icedistilled 6y ago
Having gone through the nightmare of setting up tensor flow with cuda, I can't help but think you're letting that color your view of python as a whole. Tensorflow, cuda, and nvidia were a nightmare that took me 5 days to resolve on ubuntu 18 since it was my first time setting it up ever. There was literally no working guides or people posting with my exact issue, but tons of posts about other people having a nightmare of a time. In the end I figured out I had to go find this one file/library nvidia had moved around between versions of drivers/cuda even though documentation suggested they hadn't and I was using the versions that were supposed to work fine. In the end, fixing it was either by setting an environment variable because the default path was wrong or editing the path in a config file. I'm not remembering which now.
- dnautics 6y ago
  I dunno, I think the problem is not having a project-specific location for your deps. Obviously you want global deps for a console scripting language, but if you are doing any sort of bigger project, having a global deps location (which is implicitly resolved by the language, no less) is unacceptable for reproducible development for larger projects today. The env hack that python managers like conda and venv use is, IMO, extremely dangerous, because it's hiding state. These days I use languages that don't do this for my daily drivers, it's better for my sanity.

xapata 6y ago

I've had a better experience using Conda for just Python and Pip, and anything Pip doesn't handle as I wish, then using Pip for the bulk of the dependency tree. Pip's new resolver should make this even better.

systemvoltage 6y ago
Have you tried poetry? It's pretty good.
- xapata 6y ago
  Not yet. I'm happy enough with my setup.

mbreese 6y ago

Some people I work with have had major issues with Conda packaging. Sometimes it works, sometimes not. This can be especially tricky when you’re working with system libraries with competing versions. I still have no idea how a particular set of tools has ever run, given their configuration. The exact same Conda packages work on a local Mac, but fail on a Linux server. It can be frustrating. Because of this, I try to avoid it as much as possible. I think Conda is trying too hard to hit the middle ground between full containers (Docker or Singularity) and virtualenv’s. If your program is too complicated to be managed with a venv, it’s better to package it up as a full container and be done with it. Also, I don’t think Conda would have helped here when the problem is lack of specific Python versions installed at the system level. On a Mac (and everywhere, really), I try my best to avoid the main system Python install and use a user-local (compiled) install when I need a specific version that isn’t available by default.