4 ms·

Self-contained highly-portable Python distributions

167 points by jcbhmr 3mo ago

rsyring 3mo ago
> Many users of these distributions might be better served by the PyOxy sister project [1]. PyOxy takes these Python distributions and adds some Rust code for enhancing the functionality of the Python interpreter. The official PyOxy release binaries are single file executables providing a full-featured Python interpreter. 1: https://github.com/indygreg/PyOxidizer/ https://github.com/indygreg/PyOxidizer/ From that readme, it seems PyOxy has a few related uses: - It can produce a single file executable representing a Python app including the interpreter - It can ship self-contained Python interpreters and related to be embedded or used as a library in a larger application - PyOxidizer can serve as a bridge between Rust and Python - "PyOxidizer can be used to easily add a Python interpreter to any Rust project. But the opposite is also true: PyOxidizer can also be used to add Rust to Python."
- d3Xt3r 3mo ago
  That project seems to have been abandoned though. Last release was 4 years ago, and last commit was 2 years ago.
  - rsyring 3mo ago
    You are correct. I just hadn't noticed: - Project Status Update: https://github.com/indygreg/PyOxidizer/discussions/740 https://github.com/indygreg/PyOxidizer/discussions/740 - https://gregoryszorc.com/blog/2024/03/17/my-shifting-open-source-priorities/ https://gregoryszorc.com/blog/2024/03/17/my-shifting-open-so...
- julienfr112 3mo ago
  It kind of worked, but had some problems, like mostly targeted to mac os (probably the dev env) but not so much with linux. And could maybe do to much, with a flaky configuration in https://starlark-lang.org/ https://starlark-lang.org/. I've spend some hours trying to make it work for my use case, but it ended beeing wasted time...
simonw 3mo ago
These distributions are excellent. Astral took over maintenance of them a while ago so technically they sit under OpenAI now. https://github.com/astral-sh/python-build-standalone https://github.com/astral-sh/python-build-standalone If you're looking to bundle Python into another application - a macOS desktop app for example - these are exactly what you need.
charliermarsh 3mo ago
These are the Python distributions we use in uv (https://github.com/astral-sh/uv https://github.com/astral-sh/uv), i.e., when you install Python with uv, you're installing from python-build-standalone. (Same goes for many of the other tools that can install Python for you, like pipx, Hatch, Poetry, Bazel, etc.) Most of our engineering time on the project over the past ~year and a half has been split into three buckets: 1. Keeping up with upstream CPython. (We're also hoping to upstream as much of this work as we can into CPython itself.) 2. Fixing any / all of the known "quirks" vis-a-vis CPython. 3. Making these distributions as fast as (or faster than) any other CPython distribution. If you're interested, I wrote a bit about the why / how here when we took over maintenance of the project: https://x.com/charliermarsh/status/1864050698574311561 https://x.com/charliermarsh/status/1864050698574311561
- gooseyard 3mo ago
  At my employer, we'd been using a similar scheme to build and ship interpreters along with releases of various pieces of software for years. Although I was interested in p-b-s for some time, I was loathe to consider switching to it since one of the major motivations for building our own binaries was the collection of random obscure changes or customizations we'd accumulated in our build harness over the years, and the fact that we need to provide some guarantees about the provenance of the binaries. Despite those needs though, I eventually looked into the p-b-s builder and found that I couldn't make a serious argument against continuing to maintain our builder and all of the random crap that's needed to manage all the dependencies and using p-b-s. It solves all the same problems and after having switched to it a couple of years ago, my initial concerns about possibly needing to get changes upstreamed never became an issue as your folks pretty much always run into and fix stuff before I do and have been both really pragmatic and helpful in any of the discussions surrounding issues. I keep hoping for an opportunity to contribute something back since the project has really saved me a great deal of toil over the past year or two since adopting it. I really like the model of treating the interpreter as just another runtime dependency, and after working in the Erlang universe for a few years I keep thinking there are some interesting opportunities for tooling related to bundling stripped-down versions of the runtime stuff along with an application rather than some of the more exotic approaches. Stuff like rebar/erlang.mk might be a nice pattern to study. Anyhow if Indygreg is reading this, huge thanks and nice work, and I really appreciate all the work you (Charlie) and the astral folks have put in, it's been a real bright spot at work for a while.
zie 3mo ago
There is also the APE/Cosmopolitan cross platform binaries, which includes a python. Yes, cross platform binaries. The binaries run "natively on Linux + Mac + Windows + FreeBSD + OpenBSD 7.3 + NetBSD + BIOS with the best possible performance and the tiniest footprint imaginable." * Python Source: https://github.com/jart/cosmopolitan/tree/master/third_party/python https://github.com/jart/cosmopolitan/tree/master/third_party... * Python Binary: https://cosmo.zip/pub/cosmos/bin/python https://cosmo.zip/pub/cosmos/bin/python which is ~ 40MB. Apparently you can zip your .py files in with the python binary and make it run, but I haven't had a chance to try and play with that yet.
- Imustaskforhelp 3mo ago
  speaking of Cosmopolitan, is postgres supported within Cosmopolitan. I would like to know because I would love a static cross platform postgres
  - zie 3mo ago
    Not PG no. At least not yet. I wonder if they can get WASM to work, which seems to be working roughly the same problem from a totally different starting point and perspective. I'm pretty sure PG binaries for WASM do exist though.
    - Imustaskforhelp 3mo ago
      I think that you might be talking about pglite.dev when talking about wasm from my understanding, Wasm has a performance hit though from my understanding and unless one plans to run postgres on web. on the other hand a static cross platform single binary Postgres binary doesn't really have any performance hit. (anecdotally, Personally I would be interested for something like this existing and maybe even integrating it with golang embed. Some of my apps are full stack golang with sqlite backend all baked into a single static binary which I can run and be cross-compiled but I would like to have the same level of flexibility but with postgres as well hopefully)
      - 3mo ago
orliesaurus 3mo ago
The important distinction is that self-contained doesn’t mean one binary that runs everywhere...does it? A virtualenv still depends on the host interpreter and OS. These distributions pin the interpreter, standard library, and most runtime dependencies, but the remaining boundary is where production surprises happen: - glibc version, CPU feature level, and third-party native extensions. The musl builds remove libc dependencies, for example, but can’t load ordinary `.so` extensions. I understand why it's important for uv to maintain this as it provides a reproducible Python base while making the platform-specific extension layer an explicit part of the deployment decision
- zie 3mo ago
  > The important distinction is that self-contained doesn’t mean one binary that runs everywhere...does it? That's what Cosmopolitan/APE does though, and they deliver a python: * Source: https://github.com/jart/cosmopolitan/ https://github.com/jart/cosmopolitan/ * Python Binary: https://cosmo.zip/pub/cosmos/bin/python https://cosmo.zip/pub/cosmos/bin/python which is ~ 40MB.
1928416 3mo ago
What is this sudden revived Astral marketing? Did Astral's package cache proxy or another part cause the OpenAI/Huggingface incident?
djfobbz 3mo ago
Isn’t this what Sublime Text uses?
tonymet 3mo ago
Are they less than 700mb?
- xAlecto 3mo ago
  Between 15MB and 20MB depending on which Python version, when installing with uv.
  - tonymet 3mo ago
    let's hope google cloud can use it to reduce their cli install size
- zahlman 3mo ago
  What distributions are anywhere near that large?
  - tonymet 3mo ago
    it's a sort of inside joke about google cloud cli being > 1gb , because they include a python distribution and a bunch of third party deps . so i'm glad to hear that this app can ship smaller packages
0cf8612b2e1e 3mo ago
I keep meaning to explore compiling Python +libs to WASM for running in a desktop environment. It is already Python, so I am willing to accommodate enormous performance losses. I just want to package up the code into something more straightforward than PyInstaller.
- rented_mule 3mo ago
  JavaScript has module bundlers that will merge all your code modules into one module for deployment. It's a little surprising nobody (that I'm aware of) has done this for Python and PEP 723 dependency specifications. Then the single binary for uv is all that's required to run the bundle using this type of approach, at least on unix-ish machines (not sure if there's a trick to make this work on Windows): https://docs.astral.sh/uv/guides/scripts/#using-a-shebang-to-create-an-executable-file https://docs.astral.sh/uv/guides/scripts/#using-a-shebang-to...
  - 0cf8612b2e1e 3mo ago
    That’s not a bad idea. uvx feels like a temporary hack on which I can’t rely for a permanent deployment, but it is all probably fine. I would want to at least materialize a real environment so I know that it is not getting randomly deleted as cache/temp gets cleared. Unfortunately, it’s still not my ideal use case of distributing executables to Windows machines. I really want a single artifact users could freely share without any pre-configuration. I guess I could have a batch/vbs script which would bootstrap uv plus the rest of the machinery, but that is more moving parts than I would like.
    - fragmede 3mo ago
      pyexe and pyinstaller let you just ship an exe to run
      - quietbritishjim 3mo ago
        They just unzip the Python executable and your source files to a temporary directory and run them from there (although they are a convenient way of doing that). nuitka is a bit better (in the sense of not running executables from a temporary path) in that it really compiles your code into an executable in "standalone" mode.
greatsage_sh 3mo ago
[flagged]
amelius 3mo ago
Every time I look into scripting in programs like Kicad, Freecad or Inkscape, I run into Python compatibility problems. I can usually fix it, but what a mess. Package management in 2026 leaves a lot to be desired.
insumanth 3mo ago
> library dependencies are either distributed with the distribution or are statically linked. I have messed up my packages multiple times and have feared installing anything due to it. I always prefer this type of distributions for any software and now, it works like a charm.

167 points by jcbhmr 3mo ago

> Many users of these distributions might be better served by the PyOxy sister project [1]. PyOxy takes these Python distributions and adds some Rust code for enhancing the functionality of the Python interpreter. The official PyOxy release binaries are single file executables providing a full-featured Python interpreter. 1: https://github.com/indygreg/PyOxidizer/ https://github.com/indygreg/PyOxidizer/ From that readme, it seems PyOxy has a few related uses: - It can produce a single file executable representing a Python app including the interpreter - It can ship self-contained Python interpreters and related to be embedded or used as a library in a larger application - PyOxidizer can serve as a bridge between Rust and Python - "PyOxidizer can be used to easily add a Python interpreter to any Rust project. But the opposite is also true: PyOxidizer can also be used to add Rust to Python."

d3Xt3r 3mo ago
That project seems to have been abandoned though. Last release was 4 years ago, and last commit was 2 years ago.
- rsyring 3mo ago
  You are correct. I just hadn't noticed: - Project Status Update: https://github.com/indygreg/PyOxidizer/discussions/740 https://github.com/indygreg/PyOxidizer/discussions/740 - https://gregoryszorc.com/blog/2024/03/17/my-shifting-open-source-priorities/ https://gregoryszorc.com/blog/2024/03/17/my-shifting-open-so...
julienfr112 3mo ago
It kind of worked, but had some problems, like mostly targeted to mac os (probably the dev env) but not so much with linux. And could maybe do to much, with a flaky configuration in https://starlark-lang.org/ https://starlark-lang.org/. I've spend some hours trying to make it work for my use case, but it ended beeing wasted time...

simonw 3mo ago

These distributions are excellent. Astral took over maintenance of them a while ago so technically they sit under OpenAI now. https://github.com/astral-sh/python-build-standalone https://github.com/astral-sh/python-build-standalone If you're looking to bundle Python into another application - a macOS desktop app for example - these are exactly what you need.

charliermarsh 3mo ago

These are the Python distributions we use in uv (https://github.com/astral-sh/uv https://github.com/astral-sh/uv), i.e., when you install Python with uv, you're installing from python-build-standalone. (Same goes for many of the other tools that can install Python for you, like pipx, Hatch, Poetry, Bazel, etc.) Most of our engineering time on the project over the past ~year and a half has been split into three buckets: 1. Keeping up with upstream CPython. (We're also hoping to upstream as much of this work as we can into CPython itself.) 2. Fixing any / all of the known "quirks" vis-a-vis CPython. 3. Making these distributions as fast as (or faster than) any other CPython distribution. If you're interested, I wrote a bit about the why / how here when we took over maintenance of the project: https://x.com/charliermarsh/status/1864050698574311561 https://x.com/charliermarsh/status/1864050698574311561

gooseyard 3mo ago
At my employer, we'd been using a similar scheme to build and ship interpreters along with releases of various pieces of software for years. Although I was interested in p-b-s for some time, I was loathe to consider switching to it since one of the major motivations for building our own binaries was the collection of random obscure changes or customizations we'd accumulated in our build harness over the years, and the fact that we need to provide some guarantees about the provenance of the binaries. Despite those needs though, I eventually looked into the p-b-s builder and found that I couldn't make a serious argument against continuing to maintain our builder and all of the random crap that's needed to manage all the dependencies and using p-b-s. It solves all the same problems and after having switched to it a couple of years ago, my initial concerns about possibly needing to get changes upstreamed never became an issue as your folks pretty much always run into and fix stuff before I do and have been both really pragmatic and helpful in any of the discussions surrounding issues. I keep hoping for an opportunity to contribute something back since the project has really saved me a great deal of toil over the past year or two since adopting it. I really like the model of treating the interpreter as just another runtime dependency, and after working in the Erlang universe for a few years I keep thinking there are some interesting opportunities for tooling related to bundling stripped-down versions of the runtime stuff along with an application rather than some of the more exotic approaches. Stuff like rebar/erlang.mk might be a nice pattern to study. Anyhow if Indygreg is reading this, huge thanks and nice work, and I really appreciate all the work you (Charlie) and the astral folks have put in, it's been a real bright spot at work for a while.

zie 3mo ago

There is also the APE/Cosmopolitan cross platform binaries, which includes a python. Yes, cross platform binaries. The binaries run "natively on Linux + Mac + Windows + FreeBSD + OpenBSD 7.3 + NetBSD + BIOS with the best possible performance and the tiniest footprint imaginable." * Python Source: https://github.com/jart/cosmopolitan/tree/master/third_party/python https://github.com/jart/cosmopolitan/tree/master/third_party... * Python Binary: https://cosmo.zip/pub/cosmos/bin/python https://cosmo.zip/pub/cosmos/bin/python which is ~ 40MB. Apparently you can zip your .py files in with the python binary and make it run, but I haven't had a chance to try and play with that yet.

Imustaskforhelp 3mo ago
speaking of Cosmopolitan, is postgres supported within Cosmopolitan. I would like to know because I would love a static cross platform postgres
- zie 3mo ago
  Not PG no. At least not yet. I wonder if they can get WASM to work, which seems to be working roughly the same problem from a totally different starting point and perspective. I'm pretty sure PG binaries for WASM do exist though.
  - Imustaskforhelp 3mo ago
    I think that you might be talking about pglite.dev when talking about wasm from my understanding, Wasm has a performance hit though from my understanding and unless one plans to run postgres on web. on the other hand a static cross platform single binary Postgres binary doesn't really have any performance hit. (anecdotally, Personally I would be interested for something like this existing and maybe even integrating it with golang embed. Some of my apps are full stack golang with sqlite backend all baked into a single static binary which I can run and be cross-compiled but I would like to have the same level of flexibility but with postgres as well hopefully)
    - 3mo ago

orliesaurus 3mo ago

The important distinction is that self-contained doesn’t mean one binary that runs everywhere...does it? A virtualenv still depends on the host interpreter and OS. These distributions pin the interpreter, standard library, and most runtime dependencies, but the remaining boundary is where production surprises happen: - glibc version, CPU feature level, and third-party native extensions. The musl builds remove libc dependencies, for example, but can’t load ordinary `.so` extensions. I understand why it's important for uv to maintain this as it provides a reproducible Python base while making the platform-specific extension layer an explicit part of the deployment decision

zie 3mo ago
> The important distinction is that self-contained doesn’t mean one binary that runs everywhere...does it? That's what Cosmopolitan/APE does though, and they deliver a python: * Source: https://github.com/jart/cosmopolitan/ https://github.com/jart/cosmopolitan/ * Python Binary: https://cosmo.zip/pub/cosmos/bin/python https://cosmo.zip/pub/cosmos/bin/python which is ~ 40MB.

1928416 3mo ago

What is this sudden revived Astral marketing? Did Astral's package cache proxy or another part cause the OpenAI/Huggingface incident?

djfobbz 3mo ago

Isn’t this what Sublime Text uses?

tonymet 3mo ago

Are they less than 700mb?

xAlecto 3mo ago
Between 15MB and 20MB depending on which Python version, when installing with uv.
- tonymet 3mo ago
  let's hope google cloud can use it to reduce their cli install size
zahlman 3mo ago
What distributions are anywhere near that large?
- tonymet 3mo ago
  it's a sort of inside joke about google cloud cli being > 1gb , because they include a python distribution and a bunch of third party deps . so i'm glad to hear that this app can ship smaller packages

0cf8612b2e1e 3mo ago

I keep meaning to explore compiling Python +libs to WASM for running in a desktop environment. It is already Python, so I am willing to accommodate enormous performance losses. I just want to package up the code into something more straightforward than PyInstaller.

rented_mule 3mo ago
JavaScript has module bundlers that will merge all your code modules into one module for deployment. It's a little surprising nobody (that I'm aware of) has done this for Python and PEP 723 dependency specifications. Then the single binary for uv is all that's required to run the bundle using this type of approach, at least on unix-ish machines (not sure if there's a trick to make this work on Windows): https://docs.astral.sh/uv/guides/scripts/#using-a-shebang-to-create-an-executable-file https://docs.astral.sh/uv/guides/scripts/#using-a-shebang-to...
- 0cf8612b2e1e 3mo ago
  That’s not a bad idea. uvx feels like a temporary hack on which I can’t rely for a permanent deployment, but it is all probably fine. I would want to at least materialize a real environment so I know that it is not getting randomly deleted as cache/temp gets cleared. Unfortunately, it’s still not my ideal use case of distributing executables to Windows machines. I really want a single artifact users could freely share without any pre-configuration. I guess I could have a batch/vbs script which would bootstrap uv plus the rest of the machinery, but that is more moving parts than I would like.
  - fragmede 3mo ago
    pyexe and pyinstaller let you just ship an exe to run
    - quietbritishjim 3mo ago
      They just unzip the Python executable and your source files to a temporary directory and run them from there (although they are a convenient way of doing that). nuitka is a bit better (in the sense of not running executables from a temporary path) in that it really compiles your code into an executable in "standalone" mode.

greatsage_sh 3mo ago

[flagged]

amelius 3mo ago

Every time I look into scripting in programs like Kicad, Freecad or Inkscape, I run into Python compatibility problems. I can usually fix it, but what a mess. Package management in 2026 leaves a lot to be desired.

insumanth 3mo ago

> library dependencies are either distributed with the distribution or are statically linked. I have messed up my packages multiple times and have feared installing anything due to it. I always prefer this type of distributions for any software and now, it works like a charm.