4 ms·

Is Nuitka able to create self-contained binaries that can be distributed as a single file? Or do they still have external dependencies such as the Python runtim

by pansa2 5y ago

Is Nuitka able to create self-contained binaries that can be distributed as a single file? Or do they still have external dependencies such as the Python runtime? I’ve chosen Go rather than Python for a few small projects recently. Not for performance, parallelism or a particular fondness for the language, but just because Go can build truly standalone executables.

reacharavindh 5y ago
The last time I tried this was quite a while ago and your question was exactly why I tried. But, it failed with some unintuitive errors in dependencies of my Python script, and I gave up after 10 mins of Googling. I’m still wary of that experience and will avoid Python where I have such deployment needs unless the language natively comes up with such a build solution. FWIW, I think borg(backup solution written in Python and C) uses pyinstaller to get a single binary executable. It may be of interest to you.
thinker5555 5y ago
According to the readme: > The created binaries can be made executable independent of the Python installation, with --standalone and --onefile options.
vascocosta 5y ago
I use Pyinstaller to generate single file executables of my Python projects, which use Qt. It works great.
- XzAeRosho 5y ago
  The only problem I have found with Pyinstaller is that their "standalone" builds still depends on certain libraries that will change with the version of Python in the host OS, such as the glibc library, or others. The proposed solution by Pyinstaller is to build your program in the "oldest" version you support, which is silly compared to build systems like Go's, which are true standalone. Having said that, I'm curious about this solution, since it seems to claim a true standalone build...
  - physicsguy 5y ago
    Go binaries are often not truly standalone. You can certainly make static Go binaries that don't use cgo, but you can't then use standard libraries which rely on it.
    - curryst 5y ago
      I believe that if you use MUSL for libc, you can statically link libc into the binary. That should make the standard library all statically linkable, I'm not aware of any other C dependencies (although I could very well be wrong).
  - zvr 5y ago
    The other issue with PyInstaller is that, by default, it includes all dynamic libraries that the Python interpreter has on your machine. It makes sense, it needs an interpreter and collects what is needed for it to run. Unfortunately this might include libreadline.so, which is licensed under GPL, making your resulting executable unable to be under a proprietary license. There are ways to solve this issue, but one has to search and read documentation (and code, in my case -- when I was researching it the docs were not clear).
    - pansa2 5y ago
      > [PyInstaller] includes all dynamic libraries that the Python interpreter has on your machine. Yes - and last time I used it, it created either a large folder or a compressed archive containing all of those libraries. Only the latter gives a truly standalone executable - but it's very slow to start up because it has to extract the archive to disk every time it runs. It sounds like Nuitka has a solution for this problem, at least on Linux: "[the binary] will not even unpack itself, but instead loop back mount its contents as a filesystem".
      - XzAeRosho 5y ago
        Indeed it does. The only downside of this is that the resulting binary is (at least in my case) almost tripled in size compared to Pyinstaller (5MB vs 14.6MB). But I can live with that. Still doesn't statically link C libraries (or at least I didn't find the setting for it), or other libraries for that matter. Pyinstaller binary build depends only on: libdl.so.2, libz.so.1 and libc.so.6. Nuitka binary build depends on: libdl.so.2, libz.so.1 and libc.so.6 AND libpthread.so.0 (for the loopback mounts I suppose). The one that always creates problems is libc.so.6, which usually is not present 4 year old systems...
- criddell 5y ago
  Did it bundle all of Qt and its dependencies into a single executable file?
  - vascocosta 5y ago
    Yes. I do it for my weather app that uses PyQt. On the release files I have single binaries that work flawlessly: https://github.com/vascocosta/glueather https://github.com/vascocosta/glueather
    - criddell 5y ago
      I just downloaded it and it works great. Took me a minute to figure out how it wanted me to enter my location (Austin, TX USA is what worked). Would it be hard to add a button to get the location from the OS? I think all the major operating systems have a location service these days.
      - vascocosta 5y ago
        That's a very good idea. I'll try to add that feature on the next release. Thank you for using my little weather app. :)
jart 5y ago
Yeah but they only run on a single operating system, so all you've accomplished is going from O(n*2) files to O(n) which is good but it'll never be great like APE which needs 1 file.
- andai 5y ago
  In case anyone missed it, the Actually Portable Executable is well worth a read: https://justine.lol/ape.html https://justine.lol/ape.html https://news.ycombinator.com/item?id=26273960 https://news.ycombinator.com/item?id=26273960
- StavrosK 5y ago
  Yeah, but in the first case you go from 1000 files to 2, whereas in the second you go from 2 to 1, which is much less impressive. Metrics are fun!
- IshKebab 5y ago
  Are people actually using APE? It's a really clever hack but I can't imagine it's a good idea to actually distribute executables that rely on ancient COM / shell formats and a weird C library to work. It's really not that hard to build standalone binaries for Mac, Linux and Windows using GitHub Actions. Especially if you use a reasonably modern language like Go, Dart or Rust.
- throwaway894345 5y ago
  Each binary runs on a single operating system, but you can easily cross-compile in Go. This is strictly better than the compiled Python case--either you're compiling Python to native code in which case you have the same problem and none of the easy-cross-compile benefits or else you ship a zip which inevitably contains lots of C dependencies that are also specific to one platform and you still can't easily cross compile. > so all you've accomplished is going from O(n*2) files to O(n) You go from N*M*2 where `M` is the number of target platforms to M (no need to use big-O notation as far as I can tell).

by pansa2 5y ago

reacharavindh 5y ago

The last time I tried this was quite a while ago and your question was exactly why I tried. But, it failed with some unintuitive errors in dependencies of my Python script, and I gave up after 10 mins of Googling. I’m still wary of that experience and will avoid Python where I have such deployment needs unless the language natively comes up with such a build solution. FWIW, I think borg(backup solution written in Python and C) uses pyinstaller to get a single binary executable. It may be of interest to you.

thinker5555 5y ago

According to the readme: > The created binaries can be made executable independent of the Python installation, with --standalone and --onefile options.

vascocosta 5y ago

I use Pyinstaller to generate single file executables of my Python projects, which use Qt. It works great.

XzAeRosho 5y ago
The only problem I have found with Pyinstaller is that their "standalone" builds still depends on certain libraries that will change with the version of Python in the host OS, such as the glibc library, or others. The proposed solution by Pyinstaller is to build your program in the "oldest" version you support, which is silly compared to build systems like Go's, which are true standalone. Having said that, I'm curious about this solution, since it seems to claim a true standalone build...
- physicsguy 5y ago
  Go binaries are often not truly standalone. You can certainly make static Go binaries that don't use cgo, but you can't then use standard libraries which rely on it.
  - curryst 5y ago
    I believe that if you use MUSL for libc, you can statically link libc into the binary. That should make the standard library all statically linkable, I'm not aware of any other C dependencies (although I could very well be wrong).
- zvr 5y ago
  The other issue with PyInstaller is that, by default, it includes all dynamic libraries that the Python interpreter has on your machine. It makes sense, it needs an interpreter and collects what is needed for it to run. Unfortunately this might include libreadline.so, which is licensed under GPL, making your resulting executable unable to be under a proprietary license. There are ways to solve this issue, but one has to search and read documentation (and code, in my case -- when I was researching it the docs were not clear).
  - pansa2 5y ago
    > [PyInstaller] includes all dynamic libraries that the Python interpreter has on your machine. Yes - and last time I used it, it created either a large folder or a compressed archive containing all of those libraries. Only the latter gives a truly standalone executable - but it's very slow to start up because it has to extract the archive to disk every time it runs. It sounds like Nuitka has a solution for this problem, at least on Linux: "[the binary] will not even unpack itself, but instead loop back mount its contents as a filesystem".
    - XzAeRosho 5y ago
      Indeed it does. The only downside of this is that the resulting binary is (at least in my case) almost tripled in size compared to Pyinstaller (5MB vs 14.6MB). But I can live with that. Still doesn't statically link C libraries (or at least I didn't find the setting for it), or other libraries for that matter. Pyinstaller binary build depends only on: libdl.so.2, libz.so.1 and libc.so.6. Nuitka binary build depends on: libdl.so.2, libz.so.1 and libc.so.6 AND libpthread.so.0 (for the loopback mounts I suppose). The one that always creates problems is libc.so.6, which usually is not present 4 year old systems...
criddell 5y ago
Did it bundle all of Qt and its dependencies into a single executable file?
- vascocosta 5y ago
  Yes. I do it for my weather app that uses PyQt. On the release files I have single binaries that work flawlessly: https://github.com/vascocosta/glueather https://github.com/vascocosta/glueather
  - criddell 5y ago
    I just downloaded it and it works great. Took me a minute to figure out how it wanted me to enter my location (Austin, TX USA is what worked). Would it be hard to add a button to get the location from the OS? I think all the major operating systems have a location service these days.
    - vascocosta 5y ago
      That's a very good idea. I'll try to add that feature on the next release. Thank you for using my little weather app. :)

jart 5y ago

Yeah but they only run on a single operating system, so all you've accomplished is going from O(n*2) files to O(n) which is good but it'll never be great like APE which needs 1 file.

andai 5y ago
In case anyone missed it, the Actually Portable Executable is well worth a read: https://justine.lol/ape.html https://justine.lol/ape.html https://news.ycombinator.com/item?id=26273960 https://news.ycombinator.com/item?id=26273960
StavrosK 5y ago
Yeah, but in the first case you go from 1000 files to 2, whereas in the second you go from 2 to 1, which is much less impressive. Metrics are fun!
IshKebab 5y ago
Are people actually using APE? It's a really clever hack but I can't imagine it's a good idea to actually distribute executables that rely on ancient COM / shell formats and a weird C library to work. It's really not that hard to build standalone binaries for Mac, Linux and Windows using GitHub Actions. Especially if you use a reasonably modern language like Go, Dart or Rust.
throwaway894345 5y ago
Each binary runs on a single operating system, but you can easily cross-compile in Go. This is strictly better than the compiled Python case--either you're compiling Python to native code in which case you have the same problem and none of the easy-cross-compile benefits or else you ship a zip which inevitably contains lots of C dependencies that are also specific to one platform and you still can't easily cross compile. > so all you've accomplished is going from O(n*2) files to O(n) You go from N*M*2 where `M` is the number of target platforms to M (no need to use big-O notation as far as I can tell).