4 ms·

Somewhat related, I had a devil of a time a little bit ago trying to ship a small Python app as a fully standalone environment runnable on "any Linux" (but for

Somewhat related, I had a devil of a time a little bit ago trying to ship a small Python app as a fully standalone environment runnable on "any Linux" (but for practical purposes, Ubuntu 16.04, 18.04, and 20.04). It turns out that if you don't want to use pip, and you don't want to build separate bundles for different OSes and Python versions, it can be surprisingly tricky to get this right. Just bundling the whole interpreter doesn't work either because it's tied to a particular stdlib which is then linked to specific versions of a bunch of system dependencies, so if you go that route, you basically end up taking an entire rootfs/container with you. After evaluating a number of different solutions, I ended up being quite happy with pex: https://github.com/pantsbuild/pex https://github.com/pantsbuild/pex It basically bundles up the wheels for whatever your workspace needs, and then ships them in an archive with a bootstrap script that can recreate that environment on your target. But critically, it natively supports the idea of targeting multiple OS and Python versions, you just explicitly tell it which ones to include, eg: --platform=manylinux2014_x86_64-cp-38-cp38 # 16.04 --platform=manylinux2014_x86_64-cp-36-cp36m # 18.04 --platform=manylinux2014_x86_64-cp-35-cp35m # 20.04 Docs on this: https://pex.readthedocs.io/en/latest/buildingpex.html#platform https://pex.readthedocs.io/en/latest/buildingpex.html#platfo... And you can see the tags in use for any package on PyPI which ships compiled parts, eg: https://pypi.org/project/numpy/#files https://pypi.org/project/numpy/#files I don't know that this would be suitable for something like a game, but in my case for a small utility supporting a commercial product, it was perfect.

jtdev 6y ago
> “if you don't want to use pip” Why wouldn’t you want to use pip?
- mikepurvis 6y ago
  Pip is suitable for use by developers working in python, setting up python workspaces with python sources and python dependencies, but it's a UX fiasco for an end-user who just wants to run a black box application and not have to care. In my particular case the "application" was in fact interactive bootstrap/install scripts for a large, proprietary blob which wouldn't have been suitable for publishing on PyPI, anyway. Setting up a separate, possibly authenticated PyPI instance, and then training end users how to use it, vs just shipping everything together in a single package? Total non-starter.
  - jtdev 6y ago
    Interesting, sounds like a very unique use case. Is containerizing not a possible solution?
    - mikepurvis 6y ago
      That would have worked, but it would have made the whole thing a lot bigger— even a featherweight base image would have added more than what pex was able to do. It complicates the usage side too, as then you need to be root to chroot/nspawn/docker/whatever your way into the container. Definitely a complicating factor was that all of this was meant to be usable by non-nerds and in an environment with limited or possibly no internet access. It wouldn't have been acceptable to download your installer package at the hotel, and then get to site and invoke it only to discover that you were on the hook for a few GBs of transfer from docker.io.
    - mvolfik 6y ago
      This sounds a bit like a GUI application, so containers would bring their own problems. Also you again force end user install docker etc
mianos 6y ago
I recently just used pyinstaller and pip on an Ubuntu 16.04 build machine. Everything works for 16, 18, 20 and even some late Redhat versions with no work. Installed it on 3000 servers with paramiko under prefect. Aside from the odd individual server issue it all worked.
- mikepurvis 6y ago
  Pyinstaller was definitely one of the ones I evaluated— I don't have detailed notes, but it seems I wasn't able to get it working with one of my dependencies, possibly an issue with cffi.

jtdev 6y ago

> “if you don't want to use pip” Why wouldn’t you want to use pip?

mikepurvis 6y ago
Pip is suitable for use by developers working in python, setting up python workspaces with python sources and python dependencies, but it's a UX fiasco for an end-user who just wants to run a black box application and not have to care. In my particular case the "application" was in fact interactive bootstrap/install scripts for a large, proprietary blob which wouldn't have been suitable for publishing on PyPI, anyway. Setting up a separate, possibly authenticated PyPI instance, and then training end users how to use it, vs just shipping everything together in a single package? Total non-starter.
- jtdev 6y ago
  Interesting, sounds like a very unique use case. Is containerizing not a possible solution?
  - mikepurvis 6y ago
    That would have worked, but it would have made the whole thing a lot bigger— even a featherweight base image would have added more than what pex was able to do. It complicates the usage side too, as then you need to be root to chroot/nspawn/docker/whatever your way into the container. Definitely a complicating factor was that all of this was meant to be usable by non-nerds and in an environment with limited or possibly no internet access. It wouldn't have been acceptable to download your installer package at the hotel, and then get to site and invoke it only to discover that you were on the hook for a few GBs of transfer from docker.io.
  - mvolfik 6y ago
    This sounds a bit like a GUI application, so containers would bring their own problems. Also you again force end user install docker etc

mianos 6y ago

I recently just used pyinstaller and pip on an Ubuntu 16.04 build machine. Everything works for 16, 18, 20 and even some late Redhat versions with no work. Installed it on 3000 servers with paramiko under prefect. Aside from the odd individual server issue it all worked.

mikepurvis 6y ago
Pyinstaller was definitely one of the ones I evaluated— I don't have detailed notes, but it seems I wasn't able to get it working with one of my dependencies, possibly an issue with cffi.