4 ms·

Deploying a python application is still a horrible experience. My best attempts so far involve doing `pip wheel` during the build, then discarding anything tha

Deploying a python application is still a horrible experience. My best attempts so far involve doing `pip wheel` during the build, then discarding anything that's platform dependent and fetching their source dists instead, then bundling that together with a script that sets up a virtualenv and installs all these dependencies and the project being installed ... Compare to a jvm uberjar.

brianwawok 10y ago
Except Python you can often run from git. Git pull do some pip done.
- shadowmint 10y ago
  To be fair, you need a python runtime installed to run python, the same way you need a java runtime installed to run uberjars. ...but, the deployment is a single artifact. I don't think doing a git clone, followed by a pip install and waiting for all the dependencies to download... and compile... (or not, if you have the wrong system packages installed) really compares.
  - brianwawok 10y ago
    Why not? If you are doing a blue/green type deploy.. you are basically building up a new VM to be your new environment. It can do all kinds of download and compile type stuff. Who cares if some is pip installing?
- Eridrus 10y ago
  You can do the same things in Java land as long as you have used a build system written in the last decade such as Maven or Gradle, you can just as easily pull down a git repo and run mvn package
  - brianwawok 10y ago
    Well that is a) A 30 sec - 10 minute process to build (vs 0 in python) b) Weird because Java has a fatjar concept. Python and Java are different. The easy path to deploy them doesn't need to be the same. You may need to make them the same for ops sanity.. but if you are just doing 1 or the other, meh
    - Eridrus 10y ago
      > a) A 30 sec - 10 minute process to build (vs 0 in python) Have you ever run someone else's python code? You have the exact same issue pulling down their requirements.txt, which is often not enough because you need to apt-get other libraries, try pip again, apt-get more, try pip again, etc. > b) Weird because Java has a fatjar concept. Sure, that's what you would do if you wanted to deploy an application to proc, my point was that it supports the same git checkout workflow if you want it to, which can be useful when dealing with other people's code and you want to make changes and not just run whatever jar they made.
kiwidrew 10y ago
Yep, I'm doing the same thing to deploy Python apps on Amazon's Elastic Beanstalk... I write an "abstract" app-requirements.txt (which typically just has a single line for my application's own package), create a fresh virtualenv, `pip wheel -r app-requirements.txt` to resolve dependencies and build wheels for all the packages, and finally `pip freeze > pinned-requirements.txt` to create a list of version-pinned packages. Zip it all up and then deploy with `pip install --no-index --find-links=/path/to/wheels -r pinned-requirements.txt`. Et voila! Binary artifacts that don't suddenly break because PyPI goes down or a package maintainer uploads a new version that unexpectedly deprecates an API that you're using.
themckman 10y ago
You might want to consider checking out Pex (https://pex.readthedocs.io/en/stable/ https://pex.readthedocs.io/en/stable/). It helps you create a single, executable artifact for your Python project very similar to an UberJAR.

brianwawok 10y ago

Except Python you can often run from git. Git pull do some pip done.

shadowmint 10y ago
To be fair, you need a python runtime installed to run python, the same way you need a java runtime installed to run uberjars. ...but, the deployment is a single artifact. I don't think doing a git clone, followed by a pip install and waiting for all the dependencies to download... and compile... (or not, if you have the wrong system packages installed) really compares.
- brianwawok 10y ago
  Why not? If you are doing a blue/green type deploy.. you are basically building up a new VM to be your new environment. It can do all kinds of download and compile type stuff. Who cares if some is pip installing?
Eridrus 10y ago
You can do the same things in Java land as long as you have used a build system written in the last decade such as Maven or Gradle, you can just as easily pull down a git repo and run mvn package
- brianwawok 10y ago
  Well that is a) A 30 sec - 10 minute process to build (vs 0 in python) b) Weird because Java has a fatjar concept. Python and Java are different. The easy path to deploy them doesn't need to be the same. You may need to make them the same for ops sanity.. but if you are just doing 1 or the other, meh
  - Eridrus 10y ago
    > a) A 30 sec - 10 minute process to build (vs 0 in python) Have you ever run someone else's python code? You have the exact same issue pulling down their requirements.txt, which is often not enough because you need to apt-get other libraries, try pip again, apt-get more, try pip again, etc. > b) Weird because Java has a fatjar concept. Sure, that's what you would do if you wanted to deploy an application to proc, my point was that it supports the same git checkout workflow if you want it to, which can be useful when dealing with other people's code and you want to make changes and not just run whatever jar they made.

kiwidrew 10y ago

Yep, I'm doing the same thing to deploy Python apps on Amazon's Elastic Beanstalk... I write an "abstract" app-requirements.txt (which typically just has a single line for my application's own package), create a fresh virtualenv, `pip wheel -r app-requirements.txt` to resolve dependencies and build wheels for all the packages, and finally `pip freeze > pinned-requirements.txt` to create a list of version-pinned packages. Zip it all up and then deploy with `pip install --no-index --find-links=/path/to/wheels -r pinned-requirements.txt`. Et voila! Binary artifacts that don't suddenly break because PyPI goes down or a package maintainer uploads a new version that unexpectedly deprecates an API that you're using.

themckman 10y ago

You might want to consider checking out Pex (https://pex.readthedocs.io/en/stable/ https://pex.readthedocs.io/en/stable/). It helps you create a single, executable artifact for your Python project very similar to an UberJAR.