4 ms·

Coming from a data science background, don't anaconda and environment description files [1] solve this problem with python? Not even sudo is needed to install a

Coming from a data science background, don't anaconda and environment description files [1] solve this problem with python? Not even sudo is needed to install anaconda and setup / clone environments, also it can manage python versions independently from the system's python. [1] https://conda.io/docs/user-guide/tasks/manage-environments.html#building-identical-conda-environments https://conda.io/docs/user-guide/tasks/manage-environments.h...

reacharavindh 9y ago
Thanks for your comment. I have just started using Conda locally to learn before taking it to work. I'm a new sysadmin and my predecessor maintained a restricted list of python packages on all cluster systems like a dictator. I don't want to be one. Does conda cloning the environment mean I can just copy the project directory to another server and expect my program to run without it looking for the packages on the internet?
- StavrosK 9y ago
  I'm not sure about Conda, but virtualenv already does that.
  - aidos 9y ago
    Oh really? You can't move a virtualenv folder around on a single machine. If you move it to the same location on a different machine, does it work?
    - StavrosK 9y ago
      It does work, AFAIK, but you can also move it around on a single machine if you rerun "virtualenv <dir>" on the env (it fixes the links). It's been a while since I've done that, so I'm not sure if it still works, but it should. EDIT: Yeah, just tried it again, it rewrites the shell files so everything works again. It even uses the same python version as you used.
- reacharavindh 9y ago
  UPDATE: Conda seems to be able to do just that. conda create -n myenv python=3.6.3 conda install -n myenv scipy conda install -n myenv numpy conda install -n myenv pandas source activate myenv # To make use of the environment. source deactivate conda create --name myclone --clone myenv. # Clones the environment. du -sh /Users/asampath/anaconda3/envs/myclone du -sh /Users/asampath/anaconda3/envs/myenv 813M /Users/asampath/anaconda3/envs/myenv Fat environments. But, this works out just fine in our NFS mounted storage in cluster.
- proto-n 9y ago
  No, the intended way of cloning works by exporting the exact specifications to a text file and re-creating the environment from the text file. If you use packages from conda channels (which are repositories of packages), then it should work flawlessly. Usually basically everything is available from either the official channel (anaconda) or some community channel (like conda-forge). At least in the data-science domain. I'm not really familiar with the python web-development scene, so not sure about that. Edit: actually once I had to copy an environment from one machine to another (because of connection problems), and I seem to recall that worked too, but I don't think this method is officially supported.
  - oblio 9y ago
    > No, the intended way of cloning works by exporting the exact specifications to a text file and re-creating the environment from the text file. Snatching defeat from the jaws of victory :) But I'm not sure who to believe, the other reply to the parent comment says that copying things around is actually possible.
    - proto-n 9y ago
      Well, as my edit says, I've had success with installing anaconda on an other machine and copying the environment folder to the proper place inside the installation, but AFAIK that's not an officially supported way of doing it.

reacharavindh 9y ago

Thanks for your comment. I have just started using Conda locally to learn before taking it to work. I'm a new sysadmin and my predecessor maintained a restricted list of python packages on all cluster systems like a dictator. I don't want to be one. Does conda cloning the environment mean I can just copy the project directory to another server and expect my program to run without it looking for the packages on the internet?

StavrosK 9y ago
I'm not sure about Conda, but virtualenv already does that.
- aidos 9y ago
  Oh really? You can't move a virtualenv folder around on a single machine. If you move it to the same location on a different machine, does it work?
  - StavrosK 9y ago
    It does work, AFAIK, but you can also move it around on a single machine if you rerun "virtualenv <dir>" on the env (it fixes the links). It's been a while since I've done that, so I'm not sure if it still works, but it should. EDIT: Yeah, just tried it again, it rewrites the shell files so everything works again. It even uses the same python version as you used.
reacharavindh 9y ago
UPDATE: Conda seems to be able to do just that. conda create -n myenv python=3.6.3 conda install -n myenv scipy conda install -n myenv numpy conda install -n myenv pandas source activate myenv # To make use of the environment. source deactivate conda create --name myclone --clone myenv. # Clones the environment. du -sh /Users/asampath/anaconda3/envs/myclone du -sh /Users/asampath/anaconda3/envs/myenv 813M /Users/asampath/anaconda3/envs/myenv Fat environments. But, this works out just fine in our NFS mounted storage in cluster.
proto-n 9y ago
No, the intended way of cloning works by exporting the exact specifications to a text file and re-creating the environment from the text file. If you use packages from conda channels (which are repositories of packages), then it should work flawlessly. Usually basically everything is available from either the official channel (anaconda) or some community channel (like conda-forge). At least in the data-science domain. I'm not really familiar with the python web-development scene, so not sure about that. Edit: actually once I had to copy an environment from one machine to another (because of connection problems), and I seem to recall that worked too, but I don't think this method is officially supported.
- oblio 9y ago
  > No, the intended way of cloning works by exporting the exact specifications to a text file and re-creating the environment from the text file. Snatching defeat from the jaws of victory :) But I'm not sure who to believe, the other reply to the parent comment says that copying things around is actually possible.
  - proto-n 9y ago
    Well, as my edit says, I've had success with installing anaconda on an other machine and copying the environment folder to the proper place inside the installation, but AFAIK that's not an officially supported way of doing it.