15 ms·

Single binary executable packages

177 points by feross 5y ago

jug 5y ago
How is even Nim (!) and Zig (!!) included there but .NET with a major developer market share and that is now cross-platform and explicitly supports self-contained executables not even mentioned in the article?? If he strongly dislikes or can't use .NET as an option he should write that and explain why. (with some limitations, but the .NET stuff with .NET dependencies including the entire .NET Runtime should at least be in there, and you can compile resources like icons or other assorted data embedded into the executable as well)
rurban 5y ago
We vastly prefer single binary executables on baremetal embedded. Not need to fight against linux or some other kernel, a libc and systemd. Updates are trivial and automatic. It's trivially secure and formally verified.
mattgreenrocks 5y ago
> Languages suitable for single binary executables > C/C++: certainly possible, but you'll need on retainer a UNIX graybeard, or at least an autotools-foo master I don't understand this. The "C/C++ is black magic!!" meme just never seems to go away. They both support static linking and always have. It mostly comes down whether you dependencies will work with it, and setting appropriate compiler flags. In other words, not terribly different than anything other supporting language.
- TillE 5y ago
  Yeah you just...link to static libraries. Which have usually already been built and installed alongside the shared libraries. Also why would you be using autotools in 2022? CMake has been the de facto standard for about a decade now.
  - bally0241 5y ago
    Came here to say this, anyone still using autohell in 2022 either is a stubborn grey beard or is seriously masochistic. Building static or shared libs/programs with CMake is treated simply as an option when calling the linker. I personally don't know anyone sane who wants to deal with the mental gymnastics needed by libtool. And don't get me started on the stupid permissions games played by 'make distcheck' where a simple git --archive would do just fine.
    - lenkite 5y ago
      Using CMake is like trying to do one-handed pushups to setup your build. Use https://mesonbuild.com/ https://mesonbuild.com/ and make your life comfortable as a couch potato.
- _3u10 5y ago
  It’s probably more realistic to have a single folder executable like .app “files” on OSX. If the app has assets it gets a little crazy quite quickly.
  - zozbot234 5y ago
    AppImage does pretty much that out of the box.
    - rathboma 5y ago
      AppImages are so good. They 'just work'. They're super easy to build and distribute too. I think more Linux distributions should embrace them as the de-facto app standard to replace snap and flatpak.
  - teakettle42 5y ago
    BeOS/Haiku appends assets to the end of the ELF file with a ToC and some easy-to-use APIs for accessing them. I always wish that’d taken off. I guess AppImage isn’t dissimilar in that it appends a squashfs image to be mounted with FUSE.
    - immibis 5y ago
      Windows also does this. Linux doesn't, preferring to KISS because with demand paging there's really no reason you can't just put them in your rodata section.
  - AnIdiotOnTheNet 5y ago
    There actually is/was two standards for this in Linux: GNUStep Application Bundles[0] and ROX-Filer AppDirs. Currently AppImage is trying to keep the dream alive, but it bundles the files in a squashfs image attached to the executable. [0] Which of course came from NeXT Application Bundles, which is also where OSX Application Bundles came from.
- Kenji 5y ago
- deleted 5y ago
  [deleted]
- doliveira 5y ago
  > The "C/C++ is black magic!!" meme just never seems to go away It's true, though. Proof of that is how many tools are made basically because no one wants to deal with Makefiles and autotools. I'd argue that even Docker fits this category of tools made to hide C/C++ builds away
  - jcelerier 5y ago
    > It's true, though. Proof of that is how many tools are made basically because no one wants to deal with Makefiles and autotools. no one wants to deal with Makefiles and autotools because as of 2022 they don't support spaces in paths and the most used desktop OS on the planet happens to have spaces in the default path for installing software
    - trinovantes 5y ago
      IIRC Microsoft specifically added the space in the default path so that developers know they need to handle file paths with spaces
- BeeOnRope 5y ago
  Creating a portable single-executable in C/C++ on Linux is not particularly easy. The dominant libc for Linux, glibc, says "static linking is highly discouraged", see [1] among many other sources for the subtle problems it causes. There are solutions like [2], [3], and [4] but they all have downsides and don't compare at all to e.g., the go (or even rust) single-executable story. --- [1] https://stackoverflow.com/questions/57476533/why-is-statically-linking-glibc-discouraged https://stackoverflow.com/questions/57476533/why-is-statical... [2] Link glibc anyway and try to avoid the pitfalls. [3] Link glibc dynamically but everything else static, and build your exe using an ancient toolchain (or better, a new toolchain built for an ancient libc) so that it works on most distros. [4] Give up on glibc and statically link an alternative libc like musl.
  - kristov 5y ago
    Or just dynamic link to libc and static link the rest as much as you can. If a binary dynamically links to libc, but is otherwise statically linked, all good no?
    - admax88qqq 5y ago
      Pretty good. Glibc does update occasionally. I've hit issues in the past trying to run old proprietary binaries that required libc.so.5 But yeah that's pretty rare tbh.
      - BeeOnRope 5y ago
        The major library version number (e.g., 5 vs 6) isn't enough: the symbols even within lib.so.6 are versions and some may update every glibc version, so in general you may have problems building against a new libc 6 then running on a system with an old libc 6 (you'll get a runtime error from the dynamic loader complaining a missing versioned symbol).
        admax88qqq 5y ago
        Yay for "semantic versioning" lol.
- mwcampbell 5y ago
  One big problem is the global symbol namespace when statically linking. If I'm not mistaken, Rust and Go are able to work around this by mangling their symbols. But it's way too easy for a C or C++ library to pollute the global namespace. Granted, on Linux, the symbol namespace is also global when dynamically linking, but that's not the case on Windows; I'm not sure about Darwin. Combine that with the multiplicity of build systems, and it's easier to just use dynamic linking. Go and Rust solve that by having a standard build system.
  - dboreham 5y ago
    Hmm. I wrote C and C++ for 20 years and while I haven't used either in some time, I don't remember this ever being a problem, except when trying to port Unix code that expected errno to be global to Windows (but that's the opposite problem of the one you're talking about).
  - rkeene2 5y ago
    Darwin also uses heirarchical namespaces by default; On Linux you can do this to a limited extent with dlmopen() but not as much as you may want (e.g., to link in a library linked to another libc -- but this is unique to Linux, most platforms only have the single libc)
dj_mc_merlin 5y ago
> Rust -- say goodbye to cross-compiling, but if you stay away from OS provided libraries, you are kind of covered; What? Rust can cross compile perfectly fine. > C / C++ -- certainly possible, but you'll need on retainer a UNIX graybeard, or at least an autotools-foo master; If you write C and don't know how to cross compile your code you shouldn't be writing it. > Python -- you're already covered, just read the zipapp documentation; Python actually can't cross build (reliably). > Java -- possible, but you'll need a startup script to just call java -jar some-tool.jar; (also not a good fit for short-lived tools, mainly due to startup times;) Use GraalVM.
- lioeters 5y ago
  > NodeJS -- technically, I know it's possible; but I can almost bet my right arm that given today's ecosystem it's not feasible Caxa - Package Node.js applications into executable binaries - https://github.com/leafac/caxa https://github.com/leafac/caxa
  - danenania 5y ago
    pkg is another one: https://github.com/vercel/pkg https://github.com/vercel/pkg It can cross-compile too.
- doliveira 5y ago
  What's your experience with GraalVM? In mine, the need to painstakingly track every single reflection use by getting the stack traces at runtime (and never be quite sure if you got all of them) pretty much killed it for me
  - deleted 5y ago
    [deleted]
  - smasher164 5y ago
    Have you tried the Tracing Agent? https://www.graalvm.org/22.0/reference-manual/native-image/Agent/ https://www.graalvm.org/22.0/reference-manual/native-image/A... It will track all uses of reflection during a program's execution. You can then use the resultant config as the starting point for the reflect-config.json.
ohCh6zos 5y ago
I think the author raises some excellent points, but it still feels somehow wasteful to me.
- closeparen 5y ago
  I think the opaque and esoteric nature of packaging, which makes is so damn hard to build packages like this, actually prompts incredible amounts of waste when companies need to deploy their own code on their own machines. You get ridiculously overpowered solutions - container schedulers and the like - when all that was really wanted was binary packaging + init scripts or service units.
  - brimble 5y ago
    I'm in the camp that uses Docker almost entirely as a cross-platform (cross-distro, at the very least), very clean (i.e. everything ends up in one place and the location of all data & config is very well documented and easily configured in a standard way for pretty much all packages) package manager. I could probably get similar benefits from learning, say, dpkg and systemd really, really well... but then that'd only work on dpkg-using Linux distros. Meanwhile, docker commands, docker-based shell scripts, and docker-compose files work just about identically ~everywhere, and can even work nearly-transparently on systems that don't actually support linux containers, like macOS and Windows, via virtualization and some command shims. I hardly even care that it uses containers. The parts of it I like don't actually require that—though they do encourage people packaging for Docker to document things well and to cut through a certain amount of various packages' config bullshit. Take Samba—it's like 100x easier to configure & manage for common use cases using Docker than most distros' default packages & config files. There's no actual reason that needs to be the case, it's just that the process of containerizing it brought that as a side-effect, but that's one of the main reasons I'm using Docker, not the containerization per se.
    - ComradePhil 5y ago
      I really wish Microsoft doesn't discontinue wsl1 and instead develops it further to make it the "Wine" equivalent (for running Linux apps on Windows without emulation or virtualization). Wsl2 relies on a hypervisor (i.e. it has to run a Linux VM underneath) so running docker apps on Windows comes with all kinds of problems from poor performance to file permissions being messed up. A mature wsl1 would be able to run almost any Linux app on Windows natively.
_3u10 5y ago
This is basically what docker does and it’s a godsend.
- chaxor 5y ago
  How do you ensure that the client has docker though? Make a cross compiled binary that downloads and installs docker and then runs your image for them? Run it through a browser on another server? The starting point there leads to similar problems. Also, docker seems to be on the way towards non free (or already in a?) /non-foss , so will probably soon be completely out of favor with everyone. Perhaps other tools like jails,bhyve, etc will rule the space soon.
  - _3u10 5y ago
    Well, the vast majority of people aren’t running operating systems that require the use of docker or even really support it. So now you’re looking at 5% of users who would need to install docker. Of those users 90% are running Debian. So you insert apt install docker into your script. The remaining .4% are using red hat. So you insert yum install docker into your bash script. Then you add a line to start your image, now put the bash script on your website, voila, single file program that works anywhere.
    - chaxor 5y ago
      I'm really confused by this statement. The vast majority of people are probably running: Windows, Mac os, Ubuntu, arch, (then others). All of those support docker - the only ones I can think of that don't are freebsd (mostly people's NAS OS?). Then there's mobile, but I usually ignore this, since most of my dev is in the biology in ML with large-ish data (few TBs?) side. Most users here using the software may not understand anything about software dev, so they need it to work on a laptop with an external hard drive - no cloud compute or cloud storage (to keep cost low for them) - but it can't have any long lists of commands to get something working. One command to install and run docker seems to be a decent solution, but the program to do that needs to be accessible from any OS without gaurantee of docker being installed.
    - 5y ago
c0l0 5y ago
I've argued this stance once before, but I feel it needs reiterating on this topic: I for one am 100% convinced that whatever the (FOSS) software future has in store for the world MUST have strong support for dynamic linking. If you look at something like `apt-cache show podman | grep ^Built-Using:` (Output: https://paste.debian.net/plain/1225449 https://paste.debian.net/plain/1225449) on Debian 11, you will see why. Imagine a few of those components shared between tens of packages, and security problems discovered im some. It's got to be any package maintainer's worst nightmare. "Traditional" distros run by volunteers and in participation of small-ish companies will not be able to cope with that kind of need for rebuild churn - and yet they will live amongst us for many years, if not decades, to come.
- infogulch 5y ago
  First, can this not be reduced to an automation and compute problem? A new version of a library with a bug fix or security fix is released, so rebuild a bunch of packages. Where's the issue? Statically linked binaries can be produced from dynamic libraries, so the "rebuild a bunch of packages" can be further reduced to "re-link a bunch of packages". Optimize bandwidth consumption by building locally and validating against a reproduceable build transparency log. Second, shouldn't the majority of the time rebuilding packages be dedicated to testing each application with the new library? Would you just skip that step if you used dynamic libraries?
- duped 5y ago
  I've argued the opposite stance numerous times, because shared dependencies are far rarer than they might appear to be while requiring custom packaging for different distros is a burden I don't want to bear as a package author. I want to have one build for all distros, not custom packaging for every one that has a slightly different definition of how software is supposed to be packaged. In my experience, non-developer users don't care about package managers (they want it to be invisible) but do care about one click install. The only way I've found to satisfy those users is to distribute applications outside the distros' package managers and statically link everything as much as possible. It saves me and my users time and money. Package management with inverted dependency trees is user and developer hostile, and it's why shit like podman and docker have to exist in the first place.
stagger87 5y ago
What about a fully self contained folder? No need to statically link everything, on Windows and on Linux you can dynamically link to local files in a provided folder. Plus the folder provides a place to store local/temp files as well. This is how I deploy software on Windows and Linux, and I find it saves a lot of headaches.
- liuliu 5y ago
  Lack of compression (or at the mercy of the file system) tends to be an issue. That is why people choose to use squashfs and then have a writable overlayfs on top of it.
pjc50 5y ago
Not mentioned yet seems to be C#, which has the dotnet core "single file" packaging option. You need to read the small print on that, though, as it (a) unpacks into a temp dir at runtime and (b) last time I tried required a couple of .so files alongside the executable, missing the point entirely.
- ripley12 5y ago
  This is a little out of date; the .NET single-file situation is pretty good as of .NET 5 (which came out in 2020). These days, if you publish a single-file app all .NET libraries live in the executable and do not need to be unpacked at runtime. Native libraries still need to get unpacked at runtime (if your application uses any; many don't), but you just set the IncludeNativeLibrariesForSelfExtract build property and everything else happens automatically for you. I'm working on getting IncludeNativeLibrariesForSelfExtract set by default, upvote here if you think that's a good idea: https://github.com/dotnet/sdk/issues/24181 https://github.com/dotnet/sdk/issues/24181
jcelerier 5y ago
> if possible, especially important for Linux with it's glibc variation across distributions, try static linking; works flawlessly with Go (including cross-compiling), somewhat with Rust / C / C++, and most-likely it's already fulfilled with interpreted languages like Python / Erlang / Deno / Java; it's not possible for GUI apps as you would have to link against the OpenGL drivers statically (and thus your software would only work on one brand of graphics card)
elitepleb 5y ago
αcτµαlly pδrταblε εxεcµταblεs are this and more https://justine.lol/ape.html https://justine.lol/ape.html
- cpach 5y ago
  It’s an extremely cool concept. I don’t really see any reason to use it in prod though.
  - dec0dedab0de 5y ago
    It would be nice as an all in one installer, if you have an app you need to run on windows mac and linux in your enterprise, you could have one task in your ansible playbook, or whatever you use. without having to make sure you copy the correct one. Unfortunately it would also would be really good for viruses.
- ihateolives 5y ago
  Oh god. I wish people woukd just stop trying to be cool. Actmally pdrtable ehecmtable? This is prime https://www.reddit.com/r/grssk/ https://www.reddit.com/r/grssk/ material.
  - jart 5y ago
    I love how the unconventional name lets me know which people think that names are what matters.
FpUser 5y ago
This is what I do. All my programs are single file exe. The worst case if something else is needed (like assets) that single exe retrieves it from the web and it's been like this forever
- buzzert 5y ago
  You can also integrate the assets into the binary itself using something like gresources.
- immibis 5y ago
  That's terrible. Your website will disappear and your executables will be useless.
  - FpUser 5y ago
    Nope. Once installed it does not need the Internet. If there is a connectivity it will use it, if not it will still work (well except the collaborative functionality). You are knocking wrong door. Try SAAS instead for your critic.
smasher164 5y ago
1. Not every library can be statically linked. Graphics drivers come to mind, but most libraries for the linux desktop expect to be dynamically linked (e.g. wlroots, libinput). So not only is there a technical limitation, but a practical one as well, considering that all of these projects would have to support it. These projects rely on dynamic linking as a way to support system-wide configuration. 2. Being able to patch dependencies is important. This isn't so much a feature to help developers, but one to help users. You could technically do this without dynamic linking, but have to make it easy to recompile a project with an updated dependency (which most package managers don't). Not being able to patch a package's dependencies puts the developer accountable for your system administration issues, which is not scalable.
- deleted 5y ago
  [deleted]
stephc_int13 5y ago
The *nix application packaging style was biggest disappointment when starting to play with Linux back in the 90s. I know there are many arguments to support this style, but I simply dislike it from an aesthetic point of view, too messy.
AdamH12113 5y ago
The convenience of binary executables is pretty clear in the DOS and Windows ecosystems, where games and native utilities have been distributed this way for decades. I have Windows binaries I've been carrying around for twenty years that still work on Windows 10, and old DOS programs run just fine in DOSBox. A key characteristic of such programs is that they include most of the libraries they use in the installation, with only a few common dependencies like the C runtime or Win32 left to the rest of the system. FFMPEG is a great example.
- nusaru 5y ago
  Meanwhile, Apple has gone through two deprecations of CPU architectures: first PowerPC and then Intel. Macs are nice, but I wish they had the same commitment to backwards compatibility.
  - tonyedgecombe 5y ago
    Microsoft does it because they cater to the enterprise far more than Apple. From that perspective I'm happy with the direction Apple takes.
  - spijdar 5y ago
    Well, 3 actually, you missed the 68000 architecture. I'm not sure it really matters, in the end. How often would people want to run PowerPC-era Mac software on modern hardware natively if they could? The HN audience will include a disproportionate number of people who do, so you guys aren't included ;-) I haven't used a Mac in several years but it seems like the whole Apple mindset is more appliance like, that if you bought old software you just keep running it on the original old hardware, and given the problems with Microsoft's eternal backwards compatibility approach (applications and the OS bundling forever ancient code and keeping ancient interfaces around forever, and ending up with a disproportionate number of API calls numbered foo2, foo3, foo4ex), I'm not sure either way is strictly superior to the other. There are trade-offs with both.
    - torginus 5y ago
mwcampbell 5y ago
> I don't think a rich GUI application like LibreOffice, Firefox, Thunderbird, Slack, Discord, Skype, GMail, etc., can be delivered as a single executable. The old Delphi-based Skype for Windows was a single executable. At least, most of it was in that big .exe.
- dheera 5y ago
  Yeah, anything can be delivered as a single executable, it just requires some tools to package it as such.
- smasher164 5y ago
  Right, but applications can treat win32.dll as essentially a system-call layer. Windows also gives you a strong compatibility guarantee with those libraries. Userspace Linux is different in that not only are these libraries typically installed separately, they all have varying degrees of compatibility between versions.
chazeon 5y ago
I really hope I can create executable packages, but when some bulky math libs in Python are involved, it really becomes very hard.
- dheera 5y ago
  You can include all the bulky math libs in the executable. That way if you want some version of Tensorflow and some other executable wants some other version of Tensorflow, there isn't a conflict. IMO it is no problem if your software is several hundred megabytes in size, I have 8 TB of hard drive. Storage is cheap as long as you aren't using Apple.
  - immibis 5y ago
    Until your bundled version of Tensorflow isn't compatible with your installed version of CUDA.
    - dheera 5y ago
      So have both CUDA and non-CUDA versions bundled, along with the specific CUDA and cudnn version it needs. Still not that large. Maybe a couple gigs at most for everything, and a LOT of headache saved in return which I think is a fine deal. I mean, very often the Tensorflow version a piece of software wants doesn't actually have a version available for my currently-installed CUDA version. I'll have CUDA 10.1, 11.0, and 12.6 already on my system for other pieces of software, and your software is hellbent dependent on Tensorflow 2.3 and CUDA 11.1 and I'll end up having to install 11.1 in addition anyway just to run your software.
      - immibis 5y ago
        Until your bundled version of CUDA isn't compatible with your installed graphics card.
        dheera 5y ago
        If so you wouldn't be able to run it anyway and you'd have to go to CPU only, whether or not you bundle stuff.
goodpoint 5y ago
1) Bundling dependencies into a blob is a disaster for security and maintainability. The blob might be a statically-linked executable, a flatpak, a docker container and so on. All this methods go against OS-based packages and prevent end users to update vulnerable libraries. It is well proven that things like containers on docker hub age like fish, leaving many unpatched images around. This is why dynamic linking and Linux distributions exist! Stop reinventing the wheel in worse way. 2) Network bandwidth for updates is far from abundant for less wealthy 4 billion people on the planet. 3) Same for drive space. A lot of phones/SBC/IoT devices have limited storage. 4) Same for I/O bandwidth. (E.g. even on a new PinePhone bulky applications take many seconds to start.)
- goodpoint 5y ago
  -3 downvotes? Amazing what kind of crap the HN community has become.
bob1029 5y ago
We don't have single binary releases, but we do have single folder releases that are guaranteed to work out of the box on any given windows server. For me, the number of actual files is not of consequence. Just being able to run something and know its going to run is the key bit. The new Self-Contained Deployments feature in .NET made our lives infinitely easier as a B2B software vendor. We no longer have to have sweaty-hands conversations with the customer about which .net frameworks are installed or even allowed (?!) in their environments. I literally click my mouse... 4 times in visual studio and have a zip file containing everything we need to send to the customer. We also moved to SQLite recently, so that one zip file is literally the entire kingdom. You could grab a blank VM from Azure and our product would startup and run on it without any extra bullshit.
- blub 5y ago
  I'm surprised that this is now possible with .NET. It's amazing what Microsoft can achieve in 20 years if they put their hearts and souls into it.
  - tonyedgecombe 5y ago
    I'm reminded of Joel Spolsky's request for this: https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-have-a-linker/ https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-h...
- stronglikedan 5y ago
  Is this the same approach as portable apps?
- ripley12 5y ago
  There's something nice about the ergonomics of a single file compared to a single folder though. A single file is trivial to distribute as-is; you don't even need to put it in an archive. You can put multiple single-file apps in a directory on your PATH, no futzing with symlinks etc. There are certainly tradeoffs involved and it's not always the right decision, but these days I usually try to publish software as a single file until I can't.
mjg59 5y ago
Simultaneously arguing that Flatpak isn't the future because it relies on runtimes that may duplicate resources, and then suggesting that you should statically link in all your dependencies (creating far more duplication than Flatpak runtimes) seems… odd?
- stitched2gethr 5y ago
  In practice I'm not sure there would be much duplication. Even for tools using the same libraries they are very often using different versions.
  - teddyfrozevelt 5y ago
    In practice, the deduplication is pretty good. I have 78 Flatpaks installed (apps and runtimes), and here's my dedup stats (calculated from https://gist.github.com/powpingdone/001a46aa7db190b9c935f71c6091eb71 https://gist.github.com/powpingdone/001a46aa7db190b9c935f71c...): =========================================== no dedupe: 16.0 GB (17228518418 B) dedupe: 11.7 GB (12594871766 B) singlelet: 8.1 GB (8677649129 B) orphan: 1.8 GB (1889889290 B) =========================================== deduplicated size ratio: 73.10 singlelet space usage: 68.90 singlelet file ratio: 63.56 orphan space usage: 15.01 orphan file ratio: 2.63
adamddev1 5y ago
I found myself wanting this for pandoc while trying to install it to a docker container. It would be nice if there was just a single binary I could download, but it seemed like all the installation methods used package managers or massively complex install steps?
at_a_remove 5y ago
I wholly agree, although single directory releases are also acceptable. Last shop I worked at was ... well, they were of an age and came from an era when installation meant "drop some files in a directory, set up an icon on a desktop, what do you mean 'dependencies' and 'framework' and 'installer'?" I did my best to accommodate, because that seems more reasonable to me, but it certainly wasn't simple. I do not dislike the concept of the Registry, it is an interesting tradeoff against the skillion dialects of .ini files and other configuration, but much further than that, it feels like installation of software is akin to a drop of ink in the water -- you can't really get it back out again.
ciprian_craciun 5y ago
[I'm the author of the article in question.] I want to highlight a few ideas that perhaps I didn't cover very well in the article: * static linking -- the concept of static linking appears in total 7 times in the article, once in the summary, once qualified with "if possible", three times as comments on a few example projects that do happen to employ it, and once discussing Go's advantage; the article is not about static linking! * dynamic liking -- it is completely fine, just don't depend on non-mainstream libraries, and don't require the version released yesterday as most stable distributions won't have it (for the next couple of years); * packaging and dependency minimalism -- as I've noted on Lobsters (<https://lobste.rs/s/adr60v/single_binary_executable_packages#c_lrxbou https://lobste.rs/s/adr60v/single_binary_executable_packages...>), the article is mainly about the simplicity of building, packaging, publishing, deploying; try to read it both from the point of view of the developer and the user;
- ripley12 5y ago
  I enjoyed the article, thanks. I think you could add an entry to the "Languages suitable for single binary executables" section; these days, .NET (C#, F#) single-file applications work very well. This is a fairly recent development (single-file support has been good for ~1.5 years now) but I've been very satisfied. https://docs.microsoft.com/en-us/dotnet/core/deploying/single-file/overview https://docs.microsoft.com/en-us/dotnet/core/deploying/singl...
- tvaughan 5y ago
  Have you seen https://systemd.io/PORTABLE_SERVICES/ https://systemd.io/PORTABLE_SERVICES/?
cfstras 5y ago
What are the golang- packages about in OpenSUSE? Go doesn‘t support dynamic linking (except for non-go-pieces like glibc or via cgo). Do they contain source-code? Or pre-built .a‘s? Both would have a hard time not conflicting with go‘s own build tooling, right?
- ciprian_craciun 5y ago
  I've just installed `golang-github-burntsushi-toml` that should be the well known Go TOML library; asking `rpm -q -l` shows mainly `*.go` files, thus I assume they are mainly used as build dependencies?
  - cfstras 5y ago
    Hmm… That makes sense. Though if you are building from source, the only gain you have over regular go tooling is downloading from your favorite mirror rather than github&co. I hope it doesn’t install these source-packages for prebuilt binaries…
    - mook 5y ago
      I believe they don't allow internet access during the build (so everything needs to be vendored, or things stuck into GOPATH); those packages are probably for the second option. My impression is that OpenSUSE go packaging is a bit behind though. But I'm not an active packager so this may have been fixed.
ComradePhil 5y ago
The author doesn't seem to understand linux containers. > docker run just-another-2g-vm-image-running-a-fully-fledged-os-with-all-the-systemd-glory-masked-as-a-container; Docker doesn't run a "vm" image and it definitely doesn't run another kernel (the kernel isn't included in docker images because it's designed to run on the host kernel). It's more closer to chrooted process (if you just consider the process) than anything... combined with some resource isolation. > that certainly won't lose any data when you purge it by mistake If you don't understand what you are doing, you can "purge data by mistake" doing other things as well. Docker images come with ALL the pros listed in the article with fewer cons. For example, they are isolated from the get go by design, so fewer security concerns than running a binary on your system a your user without namespace isolation (namespace isolation is what's happening with docker). You can choose what files and devices process running in docker has access to. The download size may be bigger than a statically linked single executable... but if you are running a lot of different processes in docker, they may share the underlying layers which may mean that you end up using less disk space overall.
- ciprian_craciun 5y ago
  > The author doesn't seem to understand linux containers. > Docker doesn't run a "vm" image and it definitely doesn't run another kernel [...] Yes I do understand them! In fact, in a former life (i.e. during the long forgotten time when Linux containers were just a novelty and LXC was seen as the "future") I was a big proponent of containers. What I was not a big supporter of is containers, that instead of being as you well say "more closer to chrooted processes", instead they are almost stand-alone VM images lacking a kernel. In fact Fly.io does exactly this -- they take a Docker container, add a Linux kernel, and run everything in a Firecracker "micro" VM. ---- > that certainly won't lose any data when you purge it by mistake > If you don't understand what you are doing, you can "purge data by mistake" doing other things as well. Yes, butter-fingers are a thing... But with Docker it's so easy to delete a container by mistake (that although shouldn't have state it does); on the other side, if you are about to issue `rm -R -f .` and you are inside `/var/lib/mysql` you do have quite a few opportunities that perhaps you are doing something stupid (you have to explicitly write `rm`, add `-R` and manually type `/var/lib/mysql`)... ---- > The download size may be bigger than a statically linked single executable... but if you are running a lot of different processes in docker, they may share the underlying layers which may mean that you end up using less disk space overall. OK, say that for "server applications" (that happen to use tens of micro-services) this makes sense. But how about a "tool application", say a static-site builder; does it now make sense to use Docker to run this tool?
mywacaday 5y ago
Not quiet single binaries but https://portableapps.com/ https://portableapps.com/ has been a great resource over the years
gompertz 5y ago
May I suggest the utility Eee (http://www.erikveen.dds.nl/eee/index.html http://www.erikveen.dds.nl/eee/index.html).... Combines multiple files into one executable. Works great!
fxtentacle 5y ago
I agree 99% with this article, but the part about packaging C++ is wrong. I'd say use "vcpkg" (yes, it's Microsoft but also works on Linux) and then CMake. CLion is a nice IDE for that kind of stuff and it's reasonably easy to setup cross-compiling or GitHub pipelines with it. The result: C++ source pushed to git => GitHub pipelines => freshly compiled binaries for all platforms That said, I have also re-written some things in Go just for its ease of packaging and cross-compilation. The fact that libraries "just work" is like a Zen experience to me. Also, "go generate" really makes it easy to merge small resource files into your one-file binary app. BTW, I come from python and Ruby, where "WTF did mkmf.rb crash?" is a daily thing...
thayne 5y ago
What about things like man pages or other offline documentation, shell completions, init service files (ex. Systemd units), .desktop files, mime type files, etc.?
- ciprian_craciun 5y ago
  Good point! I'll add to that READMEs, LICENSEs, SBOMs (Software Bill of Materials), example configuration files, etc. How to supply all those files when all one gets is a single binary executable? Simple! Bundle everything in the executable. As a bonus, because the tool outputs these files, it can now generated them dynamically. For example instead of a bland configuration file, with all the possible integrations commented out, it could either try to auto-detect where it's running and what's available, or present the user with a question-answer session to fill in the details. ---- For example, a pet project of mine <https://github.com/volution/z-run https://github.com/volution/z-run>: z-run --readme # shows the README with `less` (if on TTY) or to `stdout` z-run --readme-html # for the HTML version to be opened in `lynx` z-run --manual # or --manual-man or --manual-html z-run --sbom # or --sbom-json or --sbom-html It even gives you the source code: z-run --sources-cpio | cpio -t ---- So, does your tool need a `.desktop` file? Just create a flag for that. Or, if there are too many such extra files needed to be placed wherever, just provide an `--extras-cpio` and dump them as an archive. Or if placing them requires some work, provide an `--extras-install`, but before `sudo`, kindly ask the user for permission. Granted all this requires some extra work, and increases the bulkiness of the executable, but: * all that extra code can be extracted into a common library; (I intend to do that for my software;) * if all these are compressed, especially being text-only, they are a fraction of the final executable; ---- I am especially proud of the `--sources-cpio` option. Is something broken with a particular version of the tool that you rely on? Great, instead of bumbling around GitHub to find the particular commit that was used to build this particular version, I can just get the sources from my tool and use those. All I need is the build tools, which in case of Go is another `.tar.gz`.
foxhop 5y ago
I need an exec cheat sheet for this one.
mmphosis 5y ago
Auditable and build-able from source ■ curl -O http://harmful.tool/all-source-code.gz ■ Audit ALL of the source code ■ Build the "reproducible build" in a sandbox ■ Test / Run in a sandbox ■ Delete!
jupp0r 5y ago
glibc, which prevents being linked against statically spoils this for much of the Linux world. I know about alpine, thank you.
arunc 5y ago
> have it stowed in your .git repository so that everyone in your team gets the same experience without onboarding scripts This is ironic. Do people really version their binaries in git repos?
thangalin 5y ago
"Java -- possible, but you'll need a startup script to just call java -jar some-tool.jar; (also not a good fit for short-lived tools, mainly due to startup times;)" Technologies to look at: * Warp Packer -- https://github.com/dgiagio/warp/ https://github.com/dgiagio/warp/ * Liberica Native Image Kit -- https://bell-sw.com/pages/liberica-native-image-kit/ https://bell-sw.com/pages/liberica-native-image-kit/ Warp Packer bundles my JavaFX desktop application, KeenWrite, into single binary executable files: * https://github.com/DaveJarvis/keenwrite/releases/download/2.5.1/keenwrite.bin https://github.com/DaveJarvis/keenwrite/releases/download/2.... (Linux) * https://github.com/DaveJarvis/keenwrite/releases/download/2.5.1/keenwrite.exe https://github.com/DaveJarvis/keenwrite/releases/download/2.... (Windows) * https://github.com/DaveJarvis/keenwrite/releases/download/2.5.1/keenwrite.jar https://github.com/DaveJarvis/keenwrite/releases/download/2.... (JVM) The start-up time for the first launch of the .bin or .exe is slow because it unpacks to a user directory. Subsequent starts are fast enough, even when running from the command-line as a short-lived task. Here's the script that creates the self-contained executable files: https://github.com/DaveJarvis/keenwrite/blob/master/installer.sh https://github.com/DaveJarvis/keenwrite/blob/master/installe... To create a release for all three files, I run a single shell script from a build machine: https://github.com/DaveJarvis/keenwrite/blob/master/release.sh https://github.com/DaveJarvis/keenwrite/blob/master/release.... I could probably generate a binary for MacOS, but not enough people have asked.

177 points by feross 5y ago

jug 5y ago

How is even Nim (!) and Zig (!!) included there but .NET with a major developer market share and that is now cross-platform and explicitly supports self-contained executables not even mentioned in the article?? If he strongly dislikes or can't use .NET as an option he should write that and explain why. (with some limitations, but the .NET stuff with .NET dependencies including the entire .NET Runtime should at least be in there, and you can compile resources like icons or other assorted data embedded into the executable as well)

rurban 5y ago

We vastly prefer single binary executables on baremetal embedded. Not need to fight against linux or some other kernel, a libc and systemd. Updates are trivial and automatic. It's trivially secure and formally verified.

mattgreenrocks 5y ago

> Languages suitable for single binary executables > C/C++: certainly possible, but you'll need on retainer a UNIX graybeard, or at least an autotools-foo master I don't understand this. The "C/C++ is black magic!!" meme just never seems to go away. They both support static linking and always have. It mostly comes down whether you dependencies will work with it, and setting appropriate compiler flags. In other words, not terribly different than anything other supporting language.

TillE 5y ago
Yeah you just...link to static libraries. Which have usually already been built and installed alongside the shared libraries. Also why would you be using autotools in 2022? CMake has been the de facto standard for about a decade now.
- bally0241 5y ago
  Came here to say this, anyone still using autohell in 2022 either is a stubborn grey beard or is seriously masochistic. Building static or shared libs/programs with CMake is treated simply as an option when calling the linker. I personally don't know anyone sane who wants to deal with the mental gymnastics needed by libtool. And don't get me started on the stupid permissions games played by 'make distcheck' where a simple git --archive would do just fine.
  - lenkite 5y ago
    Using CMake is like trying to do one-handed pushups to setup your build. Use https://mesonbuild.com/ https://mesonbuild.com/ and make your life comfortable as a couch potato.
_3u10 5y ago
It’s probably more realistic to have a single folder executable like .app “files” on OSX. If the app has assets it gets a little crazy quite quickly.
- zozbot234 5y ago
  AppImage does pretty much that out of the box.
  - rathboma 5y ago
    AppImages are so good. They 'just work'. They're super easy to build and distribute too. I think more Linux distributions should embrace them as the de-facto app standard to replace snap and flatpak.
- teakettle42 5y ago
  BeOS/Haiku appends assets to the end of the ELF file with a ToC and some easy-to-use APIs for accessing them. I always wish that’d taken off. I guess AppImage isn’t dissimilar in that it appends a squashfs image to be mounted with FUSE.
  - immibis 5y ago
    Windows also does this. Linux doesn't, preferring to KISS because with demand paging there's really no reason you can't just put them in your rodata section.
- AnIdiotOnTheNet 5y ago
  There actually is/was two standards for this in Linux: GNUStep Application Bundles[0] and ROX-Filer AppDirs. Currently AppImage is trying to keep the dream alive, but it bundles the files in a squashfs image attached to the executable. [0] Which of course came from NeXT Application Bundles, which is also where OSX Application Bundles came from.
Kenji 5y ago
deleted 5y ago
[deleted]
doliveira 5y ago
> The "C/C++ is black magic!!" meme just never seems to go away It's true, though. Proof of that is how many tools are made basically because no one wants to deal with Makefiles and autotools. I'd argue that even Docker fits this category of tools made to hide C/C++ builds away
- jcelerier 5y ago
  > It's true, though. Proof of that is how many tools are made basically because no one wants to deal with Makefiles and autotools. no one wants to deal with Makefiles and autotools because as of 2022 they don't support spaces in paths and the most used desktop OS on the planet happens to have spaces in the default path for installing software
  - trinovantes 5y ago
    IIRC Microsoft specifically added the space in the default path so that developers know they need to handle file paths with spaces
BeeOnRope 5y ago
Creating a portable single-executable in C/C++ on Linux is not particularly easy. The dominant libc for Linux, glibc, says "static linking is highly discouraged", see [1] among many other sources for the subtle problems it causes. There are solutions like [2], [3], and [4] but they all have downsides and don't compare at all to e.g., the go (or even rust) single-executable story. --- [1] https://stackoverflow.com/questions/57476533/why-is-statically-linking-glibc-discouraged https://stackoverflow.com/questions/57476533/why-is-statical... [2] Link glibc anyway and try to avoid the pitfalls. [3] Link glibc dynamically but everything else static, and build your exe using an ancient toolchain (or better, a new toolchain built for an ancient libc) so that it works on most distros. [4] Give up on glibc and statically link an alternative libc like musl.
- kristov 5y ago
  Or just dynamic link to libc and static link the rest as much as you can. If a binary dynamically links to libc, but is otherwise statically linked, all good no?
  - admax88qqq 5y ago
    Pretty good. Glibc does update occasionally. I've hit issues in the past trying to run old proprietary binaries that required libc.so.5 But yeah that's pretty rare tbh.
    - BeeOnRope 5y ago
      The major library version number (e.g., 5 vs 6) isn't enough: the symbols even within lib.so.6 are versions and some may update every glibc version, so in general you may have problems building against a new libc 6 then running on a system with an old libc 6 (you'll get a runtime error from the dynamic loader complaining a missing versioned symbol).
      - admax88qqq 5y ago
        Yay for "semantic versioning" lol.
mwcampbell 5y ago
One big problem is the global symbol namespace when statically linking. If I'm not mistaken, Rust and Go are able to work around this by mangling their symbols. But it's way too easy for a C or C++ library to pollute the global namespace. Granted, on Linux, the symbol namespace is also global when dynamically linking, but that's not the case on Windows; I'm not sure about Darwin. Combine that with the multiplicity of build systems, and it's easier to just use dynamic linking. Go and Rust solve that by having a standard build system.
- dboreham 5y ago
  Hmm. I wrote C and C++ for 20 years and while I haven't used either in some time, I don't remember this ever being a problem, except when trying to port Unix code that expected errno to be global to Windows (but that's the opposite problem of the one you're talking about).
- rkeene2 5y ago
  Darwin also uses heirarchical namespaces by default; On Linux you can do this to a limited extent with dlmopen() but not as much as you may want (e.g., to link in a library linked to another libc -- but this is unique to Linux, most platforms only have the single libc)

dj_mc_merlin 5y ago

> Rust -- say goodbye to cross-compiling, but if you stay away from OS provided libraries, you are kind of covered; What? Rust can cross compile perfectly fine. > C / C++ -- certainly possible, but you'll need on retainer a UNIX graybeard, or at least an autotools-foo master; If you write C and don't know how to cross compile your code you shouldn't be writing it. > Python -- you're already covered, just read the zipapp documentation; Python actually can't cross build (reliably). > Java -- possible, but you'll need a startup script to just call java -jar some-tool.jar; (also not a good fit for short-lived tools, mainly due to startup times;) Use GraalVM.

lioeters 5y ago
> NodeJS -- technically, I know it's possible; but I can almost bet my right arm that given today's ecosystem it's not feasible Caxa - Package Node.js applications into executable binaries - https://github.com/leafac/caxa https://github.com/leafac/caxa
- danenania 5y ago
  pkg is another one: https://github.com/vercel/pkg https://github.com/vercel/pkg It can cross-compile too.
doliveira 5y ago
What's your experience with GraalVM? In mine, the need to painstakingly track every single reflection use by getting the stack traces at runtime (and never be quite sure if you got all of them) pretty much killed it for me
- deleted 5y ago
  [deleted]
- smasher164 5y ago
  Have you tried the Tracing Agent? https://www.graalvm.org/22.0/reference-manual/native-image/Agent/ https://www.graalvm.org/22.0/reference-manual/native-image/A... It will track all uses of reflection during a program's execution. You can then use the resultant config as the starting point for the reflect-config.json.

ohCh6zos 5y ago

I think the author raises some excellent points, but it still feels somehow wasteful to me.

closeparen 5y ago
I think the opaque and esoteric nature of packaging, which makes is so damn hard to build packages like this, actually prompts incredible amounts of waste when companies need to deploy their own code on their own machines. You get ridiculously overpowered solutions - container schedulers and the like - when all that was really wanted was binary packaging + init scripts or service units.
- brimble 5y ago
  I'm in the camp that uses Docker almost entirely as a cross-platform (cross-distro, at the very least), very clean (i.e. everything ends up in one place and the location of all data & config is very well documented and easily configured in a standard way for pretty much all packages) package manager. I could probably get similar benefits from learning, say, dpkg and systemd really, really well... but then that'd only work on dpkg-using Linux distros. Meanwhile, docker commands, docker-based shell scripts, and docker-compose files work just about identically ~everywhere, and can even work nearly-transparently on systems that don't actually support linux containers, like macOS and Windows, via virtualization and some command shims. I hardly even care that it uses containers. The parts of it I like don't actually require that—though they do encourage people packaging for Docker to document things well and to cut through a certain amount of various packages' config bullshit. Take Samba—it's like 100x easier to configure & manage for common use cases using Docker than most distros' default packages & config files. There's no actual reason that needs to be the case, it's just that the process of containerizing it brought that as a side-effect, but that's one of the main reasons I'm using Docker, not the containerization per se.
  - ComradePhil 5y ago
    I really wish Microsoft doesn't discontinue wsl1 and instead develops it further to make it the "Wine" equivalent (for running Linux apps on Windows without emulation or virtualization). Wsl2 relies on a hypervisor (i.e. it has to run a Linux VM underneath) so running docker apps on Windows comes with all kinds of problems from poor performance to file permissions being messed up. A mature wsl1 would be able to run almost any Linux app on Windows natively.

_3u10 5y ago

This is basically what docker does and it’s a godsend.

chaxor 5y ago
How do you ensure that the client has docker though? Make a cross compiled binary that downloads and installs docker and then runs your image for them? Run it through a browser on another server? The starting point there leads to similar problems. Also, docker seems to be on the way towards non free (or already in a?) /non-foss , so will probably soon be completely out of favor with everyone. Perhaps other tools like jails,bhyve, etc will rule the space soon.
- _3u10 5y ago
  Well, the vast majority of people aren’t running operating systems that require the use of docker or even really support it. So now you’re looking at 5% of users who would need to install docker. Of those users 90% are running Debian. So you insert apt install docker into your script. The remaining .4% are using red hat. So you insert yum install docker into your bash script. Then you add a line to start your image, now put the bash script on your website, voila, single file program that works anywhere.
  - chaxor 5y ago
    I'm really confused by this statement. The vast majority of people are probably running: Windows, Mac os, Ubuntu, arch, (then others). All of those support docker - the only ones I can think of that don't are freebsd (mostly people's NAS OS?). Then there's mobile, but I usually ignore this, since most of my dev is in the biology in ML with large-ish data (few TBs?) side. Most users here using the software may not understand anything about software dev, so they need it to work on a laptop with an external hard drive - no cloud compute or cloud storage (to keep cost low for them) - but it can't have any long lists of commands to get something working. One command to install and run docker seems to be a decent solution, but the program to do that needs to be accessible from any OS without gaurantee of docker being installed.
  - 5y ago

c0l0 5y ago

I've argued this stance once before, but I feel it needs reiterating on this topic: I for one am 100% convinced that whatever the (FOSS) software future has in store for the world MUST have strong support for dynamic linking. If you look at something like `apt-cache show podman | grep ^Built-Using:` (Output: https://paste.debian.net/plain/1225449 https://paste.debian.net/plain/1225449) on Debian 11, you will see why. Imagine a few of those components shared between tens of packages, and security problems discovered im some. It's got to be any package maintainer's worst nightmare. "Traditional" distros run by volunteers and in participation of small-ish companies will not be able to cope with that kind of need for rebuild churn - and yet they will live amongst us for many years, if not decades, to come.

infogulch 5y ago
First, can this not be reduced to an automation and compute problem? A new version of a library with a bug fix or security fix is released, so rebuild a bunch of packages. Where's the issue? Statically linked binaries can be produced from dynamic libraries, so the "rebuild a bunch of packages" can be further reduced to "re-link a bunch of packages". Optimize bandwidth consumption by building locally and validating against a reproduceable build transparency log. Second, shouldn't the majority of the time rebuilding packages be dedicated to testing each application with the new library? Would you just skip that step if you used dynamic libraries?
duped 5y ago
I've argued the opposite stance numerous times, because shared dependencies are far rarer than they might appear to be while requiring custom packaging for different distros is a burden I don't want to bear as a package author. I want to have one build for all distros, not custom packaging for every one that has a slightly different definition of how software is supposed to be packaged. In my experience, non-developer users don't care about package managers (they want it to be invisible) but do care about one click install. The only way I've found to satisfy those users is to distribute applications outside the distros' package managers and statically link everything as much as possible. It saves me and my users time and money. Package management with inverted dependency trees is user and developer hostile, and it's why shit like podman and docker have to exist in the first place.

stagger87 5y ago

What about a fully self contained folder? No need to statically link everything, on Windows and on Linux you can dynamically link to local files in a provided folder. Plus the folder provides a place to store local/temp files as well. This is how I deploy software on Windows and Linux, and I find it saves a lot of headaches.

liuliu 5y ago
Lack of compression (or at the mercy of the file system) tends to be an issue. That is why people choose to use squashfs and then have a writable overlayfs on top of it.

pjc50 5y ago

Not mentioned yet seems to be C#, which has the dotnet core "single file" packaging option. You need to read the small print on that, though, as it (a) unpacks into a temp dir at runtime and (b) last time I tried required a couple of .so files alongside the executable, missing the point entirely.

ripley12 5y ago
This is a little out of date; the .NET single-file situation is pretty good as of .NET 5 (which came out in 2020). These days, if you publish a single-file app all .NET libraries live in the executable and do not need to be unpacked at runtime. Native libraries still need to get unpacked at runtime (if your application uses any; many don't), but you just set the IncludeNativeLibrariesForSelfExtract build property and everything else happens automatically for you. I'm working on getting IncludeNativeLibrariesForSelfExtract set by default, upvote here if you think that's a good idea: https://github.com/dotnet/sdk/issues/24181 https://github.com/dotnet/sdk/issues/24181

jcelerier 5y ago

> if possible, especially important for Linux with it's glibc variation across distributions, try static linking; works flawlessly with Go (including cross-compiling), somewhat with Rust / C / C++, and most-likely it's already fulfilled with interpreted languages like Python / Erlang / Deno / Java; it's not possible for GUI apps as you would have to link against the OpenGL drivers statically (and thus your software would only work on one brand of graphics card)

elitepleb 5y ago

αcτµαlly pδrταblε εxεcµταblεs are this and more https://justine.lol/ape.html https://justine.lol/ape.html

cpach 5y ago
It’s an extremely cool concept. I don’t really see any reason to use it in prod though.
- dec0dedab0de 5y ago
  It would be nice as an all in one installer, if you have an app you need to run on windows mac and linux in your enterprise, you could have one task in your ansible playbook, or whatever you use. without having to make sure you copy the correct one. Unfortunately it would also would be really good for viruses.
ihateolives 5y ago
Oh god. I wish people woukd just stop trying to be cool. Actmally pdrtable ehecmtable? This is prime https://www.reddit.com/r/grssk/ https://www.reddit.com/r/grssk/ material.
- jart 5y ago
  I love how the unconventional name lets me know which people think that names are what matters.

FpUser 5y ago

This is what I do. All my programs are single file exe. The worst case if something else is needed (like assets) that single exe retrieves it from the web and it's been like this forever

buzzert 5y ago
You can also integrate the assets into the binary itself using something like gresources.
immibis 5y ago
That's terrible. Your website will disappear and your executables will be useless.
- FpUser 5y ago
  Nope. Once installed it does not need the Internet. If there is a connectivity it will use it, if not it will still work (well except the collaborative functionality). You are knocking wrong door. Try SAAS instead for your critic.

smasher164 5y ago

1. Not every library can be statically linked. Graphics drivers come to mind, but most libraries for the linux desktop expect to be dynamically linked (e.g. wlroots, libinput). So not only is there a technical limitation, but a practical one as well, considering that all of these projects would have to support it. These projects rely on dynamic linking as a way to support system-wide configuration. 2. Being able to patch dependencies is important. This isn't so much a feature to help developers, but one to help users. You could technically do this without dynamic linking, but have to make it easy to recompile a project with an updated dependency (which most package managers don't). Not being able to patch a package's dependencies puts the developer accountable for your system administration issues, which is not scalable.

deleted 5y ago
[deleted]

stephc_int13 5y ago

The *nix application packaging style was biggest disappointment when starting to play with Linux back in the 90s. I know there are many arguments to support this style, but I simply dislike it from an aesthetic point of view, too messy.

AdamH12113 5y ago

The convenience of binary executables is pretty clear in the DOS and Windows ecosystems, where games and native utilities have been distributed this way for decades. I have Windows binaries I've been carrying around for twenty years that still work on Windows 10, and old DOS programs run just fine in DOSBox. A key characteristic of such programs is that they include most of the libraries they use in the installation, with only a few common dependencies like the C runtime or Win32 left to the rest of the system. FFMPEG is a great example.

nusaru 5y ago
Meanwhile, Apple has gone through two deprecations of CPU architectures: first PowerPC and then Intel. Macs are nice, but I wish they had the same commitment to backwards compatibility.
- tonyedgecombe 5y ago
  Microsoft does it because they cater to the enterprise far more than Apple. From that perspective I'm happy with the direction Apple takes.
- spijdar 5y ago
  Well, 3 actually, you missed the 68000 architecture. I'm not sure it really matters, in the end. How often would people want to run PowerPC-era Mac software on modern hardware natively if they could? The HN audience will include a disproportionate number of people who do, so you guys aren't included ;-) I haven't used a Mac in several years but it seems like the whole Apple mindset is more appliance like, that if you bought old software you just keep running it on the original old hardware, and given the problems with Microsoft's eternal backwards compatibility approach (applications and the OS bundling forever ancient code and keeping ancient interfaces around forever, and ending up with a disproportionate number of API calls numbered foo2, foo3, foo4ex), I'm not sure either way is strictly superior to the other. There are trade-offs with both.
  - torginus 5y ago

mwcampbell 5y ago

> I don't think a rich GUI application like LibreOffice, Firefox, Thunderbird, Slack, Discord, Skype, GMail, etc., can be delivered as a single executable. The old Delphi-based Skype for Windows was a single executable. At least, most of it was in that big .exe.

dheera 5y ago
Yeah, anything can be delivered as a single executable, it just requires some tools to package it as such.
smasher164 5y ago
Right, but applications can treat win32.dll as essentially a system-call layer. Windows also gives you a strong compatibility guarantee with those libraries. Userspace Linux is different in that not only are these libraries typically installed separately, they all have varying degrees of compatibility between versions.

chazeon 5y ago

I really hope I can create executable packages, but when some bulky math libs in Python are involved, it really becomes very hard.

dheera 5y ago
You can include all the bulky math libs in the executable. That way if you want some version of Tensorflow and some other executable wants some other version of Tensorflow, there isn't a conflict. IMO it is no problem if your software is several hundred megabytes in size, I have 8 TB of hard drive. Storage is cheap as long as you aren't using Apple.
- immibis 5y ago
  Until your bundled version of Tensorflow isn't compatible with your installed version of CUDA.
  - dheera 5y ago
    So have both CUDA and non-CUDA versions bundled, along with the specific CUDA and cudnn version it needs. Still not that large. Maybe a couple gigs at most for everything, and a LOT of headache saved in return which I think is a fine deal. I mean, very often the Tensorflow version a piece of software wants doesn't actually have a version available for my currently-installed CUDA version. I'll have CUDA 10.1, 11.0, and 12.6 already on my system for other pieces of software, and your software is hellbent dependent on Tensorflow 2.3 and CUDA 11.1 and I'll end up having to install 11.1 in addition anyway just to run your software.
    - immibis 5y ago
      Until your bundled version of CUDA isn't compatible with your installed graphics card.
      - dheera 5y ago
        If so you wouldn't be able to run it anyway and you'd have to go to CPU only, whether or not you bundle stuff.

goodpoint 5y ago

1) Bundling dependencies into a blob is a disaster for security and maintainability. The blob might be a statically-linked executable, a flatpak, a docker container and so on. All this methods go against OS-based packages and prevent end users to update vulnerable libraries. It is well proven that things like containers on docker hub age like fish, leaving many unpatched images around. This is why dynamic linking and Linux distributions exist! Stop reinventing the wheel in worse way. 2) Network bandwidth for updates is far from abundant for less wealthy 4 billion people on the planet. 3) Same for drive space. A lot of phones/SBC/IoT devices have limited storage. 4) Same for I/O bandwidth. (E.g. even on a new PinePhone bulky applications take many seconds to start.)

goodpoint 5y ago
-3 downvotes? Amazing what kind of crap the HN community has become.

bob1029 5y ago

We don't have single binary releases, but we do have single folder releases that are guaranteed to work out of the box on any given windows server. For me, the number of actual files is not of consequence. Just being able to run something and know its going to run is the key bit. The new Self-Contained Deployments feature in .NET made our lives infinitely easier as a B2B software vendor. We no longer have to have sweaty-hands conversations with the customer about which .net frameworks are installed or even allowed (?!) in their environments. I literally click my mouse... 4 times in visual studio and have a zip file containing everything we need to send to the customer. We also moved to SQLite recently, so that one zip file is literally the entire kingdom. You could grab a blank VM from Azure and our product would startup and run on it without any extra bullshit.

blub 5y ago
I'm surprised that this is now possible with .NET. It's amazing what Microsoft can achieve in 20 years if they put their hearts and souls into it.
- tonyedgecombe 5y ago
  I'm reminded of Joel Spolsky's request for this: https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-have-a-linker/ https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-h...
stronglikedan 5y ago
Is this the same approach as portable apps?
ripley12 5y ago
There's something nice about the ergonomics of a single file compared to a single folder though. A single file is trivial to distribute as-is; you don't even need to put it in an archive. You can put multiple single-file apps in a directory on your PATH, no futzing with symlinks etc. There are certainly tradeoffs involved and it's not always the right decision, but these days I usually try to publish software as a single file until I can't.

mjg59 5y ago

Simultaneously arguing that Flatpak isn't the future because it relies on runtimes that may duplicate resources, and then suggesting that you should statically link in all your dependencies (creating far more duplication than Flatpak runtimes) seems… odd?

stitched2gethr 5y ago
In practice I'm not sure there would be much duplication. Even for tools using the same libraries they are very often using different versions.
- teddyfrozevelt 5y ago
  In practice, the deduplication is pretty good. I have 78 Flatpaks installed (apps and runtimes), and here's my dedup stats (calculated from https://gist.github.com/powpingdone/001a46aa7db190b9c935f71c6091eb71 https://gist.github.com/powpingdone/001a46aa7db190b9c935f71c...): =========================================== no dedupe: 16.0 GB (17228518418 B) dedupe: 11.7 GB (12594871766 B) singlelet: 8.1 GB (8677649129 B) orphan: 1.8 GB (1889889290 B) =========================================== deduplicated size ratio: 73.10 singlelet space usage: 68.90 singlelet file ratio: 63.56 orphan space usage: 15.01 orphan file ratio: 2.63

adamddev1 5y ago

I found myself wanting this for pandoc while trying to install it to a docker container. It would be nice if there was just a single binary I could download, but it seemed like all the installation methods used package managers or massively complex install steps?

at_a_remove 5y ago

I wholly agree, although single directory releases are also acceptable. Last shop I worked at was ... well, they were of an age and came from an era when installation meant "drop some files in a directory, set up an icon on a desktop, what do you mean 'dependencies' and 'framework' and 'installer'?" I did my best to accommodate, because that seems more reasonable to me, but it certainly wasn't simple. I do not dislike the concept of the Registry, it is an interesting tradeoff against the skillion dialects of .ini files and other configuration, but much further than that, it feels like installation of software is akin to a drop of ink in the water -- you can't really get it back out again.

ciprian_craciun 5y ago

[I'm the author of the article in question.] I want to highlight a few ideas that perhaps I didn't cover very well in the article: * static linking -- the concept of static linking appears in total 7 times in the article, once in the summary, once qualified with "if possible", three times as comments on a few example projects that do happen to employ it, and once discussing Go's advantage; the article is not about static linking! * dynamic liking -- it is completely fine, just don't depend on non-mainstream libraries, and don't require the version released yesterday as most stable distributions won't have it (for the next couple of years); * packaging and dependency minimalism -- as I've noted on Lobsters (<https://lobste.rs/s/adr60v/single_binary_executable_packages#c_lrxbou https://lobste.rs/s/adr60v/single_binary_executable_packages...>), the article is mainly about the simplicity of building, packaging, publishing, deploying; try to read it both from the point of view of the developer and the user;

ripley12 5y ago
I enjoyed the article, thanks. I think you could add an entry to the "Languages suitable for single binary executables" section; these days, .NET (C#, F#) single-file applications work very well. This is a fairly recent development (single-file support has been good for ~1.5 years now) but I've been very satisfied. https://docs.microsoft.com/en-us/dotnet/core/deploying/single-file/overview https://docs.microsoft.com/en-us/dotnet/core/deploying/singl...
tvaughan 5y ago
Have you seen https://systemd.io/PORTABLE_SERVICES/ https://systemd.io/PORTABLE_SERVICES/?

cfstras 5y ago

What are the golang- packages about in OpenSUSE? Go doesn‘t support dynamic linking (except for non-go-pieces like glibc or via cgo). Do they contain source-code? Or pre-built .a‘s? Both would have a hard time not conflicting with go‘s own build tooling, right?

ciprian_craciun 5y ago
I've just installed `golang-github-burntsushi-toml` that should be the well known Go TOML library; asking `rpm -q -l` shows mainly `*.go` files, thus I assume they are mainly used as build dependencies?
- cfstras 5y ago
  Hmm… That makes sense. Though if you are building from source, the only gain you have over regular go tooling is downloading from your favorite mirror rather than github&co. I hope it doesn’t install these source-packages for prebuilt binaries…
  - mook 5y ago
    I believe they don't allow internet access during the build (so everything needs to be vendored, or things stuck into GOPATH); those packages are probably for the second option. My impression is that OpenSUSE go packaging is a bit behind though. But I'm not an active packager so this may have been fixed.

ComradePhil 5y ago

The author doesn't seem to understand linux containers. > docker run just-another-2g-vm-image-running-a-fully-fledged-os-with-all-the-systemd-glory-masked-as-a-container; Docker doesn't run a "vm" image and it definitely doesn't run another kernel (the kernel isn't included in docker images because it's designed to run on the host kernel). It's more closer to chrooted process (if you just consider the process) than anything... combined with some resource isolation. > that certainly won't lose any data when you purge it by mistake If you don't understand what you are doing, you can "purge data by mistake" doing other things as well. Docker images come with ALL the pros listed in the article with fewer cons. For example, they are isolated from the get go by design, so fewer security concerns than running a binary on your system a your user without namespace isolation (namespace isolation is what's happening with docker). You can choose what files and devices process running in docker has access to. The download size may be bigger than a statically linked single executable... but if you are running a lot of different processes in docker, they may share the underlying layers which may mean that you end up using less disk space overall.

ciprian_craciun 5y ago
> The author doesn't seem to understand linux containers. > Docker doesn't run a "vm" image and it definitely doesn't run another kernel [...] Yes I do understand them! In fact, in a former life (i.e. during the long forgotten time when Linux containers were just a novelty and LXC was seen as the "future") I was a big proponent of containers. What I was not a big supporter of is containers, that instead of being as you well say "more closer to chrooted processes", instead they are almost stand-alone VM images lacking a kernel. In fact Fly.io does exactly this -- they take a Docker container, add a Linux kernel, and run everything in a Firecracker "micro" VM. ---- > that certainly won't lose any data when you purge it by mistake > If you don't understand what you are doing, you can "purge data by mistake" doing other things as well. Yes, butter-fingers are a thing... But with Docker it's so easy to delete a container by mistake (that although shouldn't have state it does); on the other side, if you are about to issue `rm -R -f .` and you are inside `/var/lib/mysql` you do have quite a few opportunities that perhaps you are doing something stupid (you have to explicitly write `rm`, add `-R` and manually type `/var/lib/mysql`)... ---- > The download size may be bigger than a statically linked single executable... but if you are running a lot of different processes in docker, they may share the underlying layers which may mean that you end up using less disk space overall. OK, say that for "server applications" (that happen to use tens of micro-services) this makes sense. But how about a "tool application", say a static-site builder; does it now make sense to use Docker to run this tool?

mywacaday 5y ago

Not quiet single binaries but https://portableapps.com/ https://portableapps.com/ has been a great resource over the years

gompertz 5y ago

May I suggest the utility Eee (http://www.erikveen.dds.nl/eee/index.html http://www.erikveen.dds.nl/eee/index.html).... Combines multiple files into one executable. Works great!

fxtentacle 5y ago

I agree 99% with this article, but the part about packaging C++ is wrong. I'd say use "vcpkg" (yes, it's Microsoft but also works on Linux) and then CMake. CLion is a nice IDE for that kind of stuff and it's reasonably easy to setup cross-compiling or GitHub pipelines with it. The result: C++ source pushed to git => GitHub pipelines => freshly compiled binaries for all platforms That said, I have also re-written some things in Go just for its ease of packaging and cross-compilation. The fact that libraries "just work" is like a Zen experience to me. Also, "go generate" really makes it easy to merge small resource files into your one-file binary app. BTW, I come from python and Ruby, where "WTF did mkmf.rb crash?" is a daily thing...

thayne 5y ago

What about things like man pages or other offline documentation, shell completions, init service files (ex. Systemd units), .desktop files, mime type files, etc.?

ciprian_craciun 5y ago
Good point! I'll add to that READMEs, LICENSEs, SBOMs (Software Bill of Materials), example configuration files, etc. How to supply all those files when all one gets is a single binary executable? Simple! Bundle everything in the executable. As a bonus, because the tool outputs these files, it can now generated them dynamically. For example instead of a bland configuration file, with all the possible integrations commented out, it could either try to auto-detect where it's running and what's available, or present the user with a question-answer session to fill in the details. ---- For example, a pet project of mine <https://github.com/volution/z-run https://github.com/volution/z-run>: z-run --readme # shows the README with `less` (if on TTY) or to `stdout` z-run --readme-html # for the HTML version to be opened in `lynx` z-run --manual # or --manual-man or --manual-html z-run --sbom # or --sbom-json or --sbom-html It even gives you the source code: z-run --sources-cpio | cpio -t ---- So, does your tool need a `.desktop` file? Just create a flag for that. Or, if there are too many such extra files needed to be placed wherever, just provide an `--extras-cpio` and dump them as an archive. Or if placing them requires some work, provide an `--extras-install`, but before `sudo`, kindly ask the user for permission. Granted all this requires some extra work, and increases the bulkiness of the executable, but: * all that extra code can be extracted into a common library; (I intend to do that for my software;) * if all these are compressed, especially being text-only, they are a fraction of the final executable; ---- I am especially proud of the `--sources-cpio` option. Is something broken with a particular version of the tool that you rely on? Great, instead of bumbling around GitHub to find the particular commit that was used to build this particular version, I can just get the sources from my tool and use those. All I need is the build tools, which in case of Go is another `.tar.gz`.

foxhop 5y ago

I need an exec cheat sheet for this one.

mmphosis 5y ago

Auditable and build-able from source ■ curl -O http://harmful.tool/all-source-code.gz ■ Audit ALL of the source code ■ Build the "reproducible build" in a sandbox ■ Test / Run in a sandbox ■ Delete!

jupp0r 5y ago

glibc, which prevents being linked against statically spoils this for much of the Linux world. I know about alpine, thank you.

arunc 5y ago

> have it stowed in your .git repository so that everyone in your team gets the same experience without onboarding scripts This is ironic. Do people really version their binaries in git repos?

thangalin 5y ago

"Java -- possible, but you'll need a startup script to just call java -jar some-tool.jar; (also not a good fit for short-lived tools, mainly due to startup times;)" Technologies to look at: * Warp Packer -- https://github.com/dgiagio/warp/ https://github.com/dgiagio/warp/ * Liberica Native Image Kit -- https://bell-sw.com/pages/liberica-native-image-kit/ https://bell-sw.com/pages/liberica-native-image-kit/ Warp Packer bundles my JavaFX desktop application, KeenWrite, into single binary executable files: * https://github.com/DaveJarvis/keenwrite/releases/download/2.5.1/keenwrite.bin https://github.com/DaveJarvis/keenwrite/releases/download/2.... (Linux) * https://github.com/DaveJarvis/keenwrite/releases/download/2.5.1/keenwrite.exe https://github.com/DaveJarvis/keenwrite/releases/download/2.... (Windows) * https://github.com/DaveJarvis/keenwrite/releases/download/2.5.1/keenwrite.jar https://github.com/DaveJarvis/keenwrite/releases/download/2.... (JVM) The start-up time for the first launch of the .bin or .exe is slow because it unpacks to a user directory. Subsequent starts are fast enough, even when running from the command-line as a short-lived task. Here's the script that creates the self-contained executable files: https://github.com/DaveJarvis/keenwrite/blob/master/installer.sh https://github.com/DaveJarvis/keenwrite/blob/master/installe... To create a release for all three files, I run a single shell script from a build machine: https://github.com/DaveJarvis/keenwrite/blob/master/release.sh https://github.com/DaveJarvis/keenwrite/blob/master/release.... I could probably generate a binary for MacOS, but not enough people have asked.