4 ms·

I'm at the same spot myself, trying to learn Go by building a simple web app. Your article was a refreshing read to me. One extra thing that is the major motiv

I'm at the same spot myself, trying to learn Go by building a simple web app. Your article was a refreshing read to me. One extra thing that is the major motivation for me to go to Go is the ability to deploy as a single God damn binary and not deal with dependancies and pip. pip and counting on OS repositories may be the right thing to do, but it is just too cumbersome. I have an air gapped setup at work, and there is no easy way for me to deploy Python applications - because all projects just say pip instal this and pip install that. It would make my life so much easier building something on my laptop, scp the binary to servers and expect them to "just work".

pletnes 9y ago
I don’t disagree, but FYI pip has a —cache flag to use packages from file, and the files could be on a usb stick of course. Same approach can be used to speed up CI setups.
proto-n 9y ago
Coming from a data science background, don't anaconda and environment description files [1] solve this problem with python? Not even sudo is needed to install anaconda and setup / clone environments, also it can manage python versions independently from the system's python. [1] https://conda.io/docs/user-guide/tasks/manage-environments.html#building-identical-conda-environments https://conda.io/docs/user-guide/tasks/manage-environments.h...
- reacharavindh 9y ago
  Thanks for your comment. I have just started using Conda locally to learn before taking it to work. I'm a new sysadmin and my predecessor maintained a restricted list of python packages on all cluster systems like a dictator. I don't want to be one. Does conda cloning the environment mean I can just copy the project directory to another server and expect my program to run without it looking for the packages on the internet?
  - StavrosK 9y ago
    I'm not sure about Conda, but virtualenv already does that.
    - aidos 9y ago
      Oh really? You can't move a virtualenv folder around on a single machine. If you move it to the same location on a different machine, does it work?
      - StavrosK 9y ago
        It does work, AFAIK, but you can also move it around on a single machine if you rerun "virtualenv <dir>" on the env (it fixes the links). It's been a while since I've done that, so I'm not sure if it still works, but it should. EDIT: Yeah, just tried it again, it rewrites the shell files so everything works again. It even uses the same python version as you used.
  - reacharavindh 9y ago
    UPDATE: Conda seems to be able to do just that. conda create -n myenv python=3.6.3 conda install -n myenv scipy conda install -n myenv numpy conda install -n myenv pandas source activate myenv # To make use of the environment. source deactivate conda create --name myclone --clone myenv. # Clones the environment. du -sh /Users/asampath/anaconda3/envs/myclone du -sh /Users/asampath/anaconda3/envs/myenv 813M /Users/asampath/anaconda3/envs/myenv Fat environments. But, this works out just fine in our NFS mounted storage in cluster.
  - proto-n 9y ago
    No, the intended way of cloning works by exporting the exact specifications to a text file and re-creating the environment from the text file. If you use packages from conda channels (which are repositories of packages), then it should work flawlessly. Usually basically everything is available from either the official channel (anaconda) or some community channel (like conda-forge). At least in the data-science domain. I'm not really familiar with the python web-development scene, so not sure about that. Edit: actually once I had to copy an environment from one machine to another (because of connection problems), and I seem to recall that worked too, but I don't think this method is officially supported.
jnwatson 9y ago
I'm in a similar situation with Python and deployment that has no internet. We have a solution that works well. The philosophy is to think of Python as a compiled language and then everything works. The CI system make the "image" and that gets pushed. In our case we make a virtualenv but you could also use pyfreeze. Works like a charm.
TheAceOfHearts 9y ago
Surely there's tools to bundle python apps? I'm not as familiarized with the python ecosystem, but with node you can use browserify or webpack to bundle an app into a single JS file, or pkg [0] to generate a single executable. Some languages have even more powerful tools for that sort of deployment. One of the best examples I've seen is distillery [0], an Elixir package which allows for hot upgrades. That means you can just copy a single a file and upgrade without having to take anything down. [0] https://github.com/zeit/pkg https://github.com/zeit/pkg [1] https://hexdocs.pm/distillery/getting-started.html https://hexdocs.pm/distillery/getting-started.html

pletnes 9y ago

I don’t disagree, but FYI pip has a —cache flag to use packages from file, and the files could be on a usb stick of course. Same approach can be used to speed up CI setups.

proto-n 9y ago

Coming from a data science background, don't anaconda and environment description files [1] solve this problem with python? Not even sudo is needed to install anaconda and setup / clone environments, also it can manage python versions independently from the system's python. [1] https://conda.io/docs/user-guide/tasks/manage-environments.html#building-identical-conda-environments https://conda.io/docs/user-guide/tasks/manage-environments.h...

reacharavindh 9y ago
Thanks for your comment. I have just started using Conda locally to learn before taking it to work. I'm a new sysadmin and my predecessor maintained a restricted list of python packages on all cluster systems like a dictator. I don't want to be one. Does conda cloning the environment mean I can just copy the project directory to another server and expect my program to run without it looking for the packages on the internet?
- StavrosK 9y ago
  I'm not sure about Conda, but virtualenv already does that.
  - aidos 9y ago
    Oh really? You can't move a virtualenv folder around on a single machine. If you move it to the same location on a different machine, does it work?
    - StavrosK 9y ago
      It does work, AFAIK, but you can also move it around on a single machine if you rerun "virtualenv <dir>" on the env (it fixes the links). It's been a while since I've done that, so I'm not sure if it still works, but it should. EDIT: Yeah, just tried it again, it rewrites the shell files so everything works again. It even uses the same python version as you used.
- reacharavindh 9y ago
  UPDATE: Conda seems to be able to do just that. conda create -n myenv python=3.6.3 conda install -n myenv scipy conda install -n myenv numpy conda install -n myenv pandas source activate myenv # To make use of the environment. source deactivate conda create --name myclone --clone myenv. # Clones the environment. du -sh /Users/asampath/anaconda3/envs/myclone du -sh /Users/asampath/anaconda3/envs/myenv 813M /Users/asampath/anaconda3/envs/myenv Fat environments. But, this works out just fine in our NFS mounted storage in cluster.
- proto-n 9y ago
  No, the intended way of cloning works by exporting the exact specifications to a text file and re-creating the environment from the text file. If you use packages from conda channels (which are repositories of packages), then it should work flawlessly. Usually basically everything is available from either the official channel (anaconda) or some community channel (like conda-forge). At least in the data-science domain. I'm not really familiar with the python web-development scene, so not sure about that. Edit: actually once I had to copy an environment from one machine to another (because of connection problems), and I seem to recall that worked too, but I don't think this method is officially supported.

jnwatson 9y ago

I'm in a similar situation with Python and deployment that has no internet. We have a solution that works well. The philosophy is to think of Python as a compiled language and then everything works. The CI system make the "image" and that gets pushed. In our case we make a virtualenv but you could also use pyfreeze. Works like a charm.

TheAceOfHearts 9y ago

Surely there's tools to bundle python apps? I'm not as familiarized with the python ecosystem, but with node you can use browserify or webpack to bundle an app into a single JS file, or pkg [0] to generate a single executable. Some languages have even more powerful tools for that sort of deployment. One of the best examples I've seen is distillery [0], an Elixir package which allows for hot upgrades. That means you can just copy a single a file and upgrade without having to take anything down. [0] https://github.com/zeit/pkg https://github.com/zeit/pkg [1] https://hexdocs.pm/distillery/getting-started.html https://hexdocs.pm/distillery/getting-started.html