4 ms·

> Knowing that a language is installed by default on the most popular Linux distribution can simplify certain concerns considerably The author laments the decl

> Knowing that a language is installed by default on the most popular Linux distribution can simplify certain concerns considerably The author laments the decline of such knowledge, but the proliferation of containers, combined with the marketing efforts of the large cloud providers has reduced the number of people who work with servers directly. I should say that this knowledge can be useful in certain niche situations, though. For example, I recently learned that GitHub Actions pulls in a basic set of utilities by using complicated distro detection[1], and Gitlab runners similarly pulls in a Docker image with git just to clone a repository! Instead, the proper way to address this situation would be to compile static binaries, that, by their very nature, become distribution agnostic. The only reason I assume this wasn't pursued, I assume, would be that the developers working on this weren't even aware this was an option. It's the reason I've been maintaining a small set of packages[2] that I mount into every CI container for my personal projects. I'm also quite hopeful about the cosmopolitan project[3] that builds fat binaries which can bridge this gap, although as I discovered, some containers even lack tools like gzip (which cosmopolitan depends on), so maybe static binaries it is. [1] https://www.youtube.com/watch?v=9qljpi5jiMQ https://www.youtube.com/watch?v=9qljpi5jiMQ [2] https://github.com/supriyo-biswas/static-builds https://github.com/supriyo-biswas/static-builds [3] https://github.com/jart/cosmopolitan/ https://github.com/jart/cosmopolitan/

hnlmorg 2y ago
> Instead, the proper way to address this situation would be to compile static binaries, that, by their very nature, become distribution agnostic. Thats only true if you do not depend on glibc, which some statically compiled binaries still do. And even then, you’d have to rule out anything that requires newer sysalls or platforms that don’t offer stable ABI guarantees (Linux is actually the edge case here in that most kernels offer no guarantees). It’s certainly doable though.
- arccy 2y ago
  it's not really static if you depend on glibc... things can still break, e.g. when compiled on too new glibc it fails to run on older versions.
  - GoblinSlayer 2y ago
    I compile with glibc 2.17 and my programs run on all gnu systems.
    - ptsneves 2y ago
      Setting up a machine or build system for older glibc is not trivial . If you know of any DISTRO which allows you to downgrade glibc out of the box please let me know. Keep in mind that if you go back enough you will also have trouble with kernel headers and compiler interactions. Even in yocto which is a build system that builds custom distributions from scratch does not have a great experience for different glibc versions than what is shipped. Have a look at the nightmare recipes for libc. On the other hand they do provide the basics of how to ship a custom libc loader across heterogeneous machines but it gets complicated fast and not something any non specialist can do in an hour. In my experience anybody saying to compile statically does not know about the libc limitations. When they find them they hand wave, I smile and say be my guest. By the way 2.17 is an appropriate version to target. RHEL6 much? :D
      - dgrunwald 2y ago
        You don't need a distro with ancient glibc; you can instead create a cross-compilation toolchain. We used https://crosstool-ng.github.io/docs/ https://crosstool-ng.github.io/docs/ to create a cross-compilation toolchain that runs on modern debian, but produces executables that only need glibc 2.17. The only downside is that you no longer can simply apt-get a library you need -- all dependencies shipped along with your application also need to be built with this cross-compiler.
      - forrestthewoods 2y ago
        > Setting up a machine or build system for older glibc is not trivial . God, I hate glibc so much. It’s an absolutely terrible design from the 80s. It can and should be trivial to target any arbitrary version of glibc. The way Linux performs linking is so bad and so much worse than Windows. (Headers + thin import lib. Linking expecting a full and complete so is stupid and bad.) Zig solved this problem. It can trivially target any version of glibc and cross-compile. It’d be nice if the Linux community demanded this level of functionality from their toolchains. Alas. https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html https://andrewkelley.me/post/zig-cc-powerful-drop-in-replace...
- supriyo-biswas 2y ago
  The glibc dependency can usually be eliminated through the same process of static compilation; the binaries I linked are also built this way with no dependencies on glibc, for example. > platforms that don’t offer stable ABI guarantees If we're talking about Windows, the Win32 ABI is considered stable, which would also suffice for what I'm looking for, which are portable binaries that can be run on almost any version of a system.
  - rcxdude 2y ago
    statically linking against glibc does break some things (it's not an officially supported option because of that), though musl is an option and for most apps a reasonable replacement.
    - lloeki 2y ago
      At work we have this C++ that needs to be distributed as, in essence, a shared lib, and that needs to work with a lot of variability. So it has C++ and so needs a C++ stdlib. Dynamically linking is annoying so what if we statically link it? Oh but GNU's C++ stdlib wants glibc and has versioned BS and so even statically linking the C++ bits isn't ideal. And as mentioned statically linking glibc is a big no no. So: a) build a musl and install it in a prefix b) add a few bits (some headers and C runtime objects) and turn it into a sysroot c) build llvm's libc++ libunwind and a couple others, linking against that sysroot, and install it in the sysroot d) build our shared lib, statically linking against that sysroot The result is a shared lib that is self-contained except for the C stdlib, and any reference to the C stdlib has no GNUism nor glibc version. The trick is that musl is essentially a subset of glibc + there is no such thing as a musl-ism per musl's design tenets + the C ABI is stable enough. I don't recall us statically linking to musl - IIRC the trick is just to not have any GNUism at any level, in a way it also working in musl is kind of a happy side effect - but maybe I'm wrong. As a consequence it works on all glibc versions and musl as well, and is entirely independent of the system's preferred C++ stdlib. A requirement though is to load the shared library with RTLD_LOCAL (to not accidentally make internal symbols available to the outside) and RTLD_DEEPBIND (to not accidentally pick external symbols instead of internal ones)
  - 2y ago
- tripflag 2y ago
  Replacing glibc with musl is usually unproblematic. That's what I did for my own portable toolbox which builds to a single self-extracting binary; https://github.com/9001/lxc https://github.com/9001/lxc
pjmlp 2y ago
> Instead, the proper way to address this situation would be to compile static binaries, that, by their very nature, become distribution agnostic. The only reason I assume this wasn't pursued, I assume, would be that the developers working on this weren't even aware this was an option. Nowadays you even have younger developers praising static linking, as if this wasn't what we had around since the invention of the first linker, and dynamic linking only became mainstream in the 1990's. Naturally only knowing Linux, GCC and glibc doesn't help.
- almostgotcaught 2y ago
  > Nowadays you even have younger developers praising static linking, as if this wasn't what we had around since the invention of the first linker, and dynamic linking only became mainstream in the 1990's. > Naturally only knowing Linux, GCC and glibc doesn't help. It's amazing how low-brow condescension passes for wisdom. Let me give you a hint: ontogeny does not actually recapitulate phylogeny. So just because you have some vague memories of previous iterations of a technology does not mean that it's the same thing being discussed today
  - pjmlp 2y ago
    Here is another hint, don't waste time playing word games, assuming that I even get bothered with comments like yours. Modern Internet is much more friendly than Usenet forums, if you get my point.
- alexey-salmin 2y ago
  I'm not too young and I saw various methods of deploying code in production: deb packages, tarballs, docker images, static binaires. I have to say that in my experience for a specific use case of a highload c++ daemon, nothing beats the simplicity and reliability of the static binary with musl. You only want to deploy one thing, you have nothing to share, furthermore you wouldn't want to share anything for the sake of control. So many things to worry about and to waste time on just disappear. When things are on fire having less variables to check is gold.
otabdeveloper4 2y ago
Containers fix nothing, because inside your container is still a Linux distribution. (Often it's just Ubuntu, because why not.) > the proper way to address this situation would be to compile static binaries Nix is the proper way to address this. We'll get there eventually anyways, might as well bite the bullet now.
- supriyo-biswas 2y ago
  I don't think Nix solves this use case, the linked binaries would still have dependencies stored at /nix/store/... and is hardly a solution for obtaining binaries that work anywhere.
  - rkangel 2y ago
    It's not about obtaining binaries that work anywhere, it's about being able to ask for a predictable environment to run in. You don't have to care about the target environment if you can control the derivation that is built to run your program. I think we also need to separate "Nix" from "The General Approach that is implemented by Nix (and Guix etc.)". I think the "The Nix Approach" (as I'm now going to call it) is fantastic and really is the solution to a lot of problems that we have. I think that the actual implementation of Nix isn't great and painful to use in real life.
    - supriyo-biswas 2y ago
      > You don't have to care about the target environment if you can control the derivation that is built to run your program. This is not always possible, since people may have workflows based on different containers or OSes. I (and the author of the article) want to have tools that work across different environments. > The General Approach that is implemented by Nix (and Guix etc.) "Reproducible builds" or "hermetic environments" are commonly used adjectives for this, although they are not limited to these two tools either; Bazel from Google does it too, for example.
      - otabdeveloper4 2y ago
zvr 2y ago
This is perfectly OK for your own work, but if you want to distribute stuff to others, statically linking increases your workload a lot. Because you are essentially distributing an increased number of components and your obligations increase multi-fold.

hnlmorg 2y ago

> Instead, the proper way to address this situation would be to compile static binaries, that, by their very nature, become distribution agnostic. Thats only true if you do not depend on glibc, which some statically compiled binaries still do. And even then, you’d have to rule out anything that requires newer sysalls or platforms that don’t offer stable ABI guarantees (Linux is actually the edge case here in that most kernels offer no guarantees). It’s certainly doable though.

arccy 2y ago
it's not really static if you depend on glibc... things can still break, e.g. when compiled on too new glibc it fails to run on older versions.
- GoblinSlayer 2y ago
  I compile with glibc 2.17 and my programs run on all gnu systems.
  - ptsneves 2y ago
    Setting up a machine or build system for older glibc is not trivial . If you know of any DISTRO which allows you to downgrade glibc out of the box please let me know. Keep in mind that if you go back enough you will also have trouble with kernel headers and compiler interactions. Even in yocto which is a build system that builds custom distributions from scratch does not have a great experience for different glibc versions than what is shipped. Have a look at the nightmare recipes for libc. On the other hand they do provide the basics of how to ship a custom libc loader across heterogeneous machines but it gets complicated fast and not something any non specialist can do in an hour. In my experience anybody saying to compile statically does not know about the libc limitations. When they find them they hand wave, I smile and say be my guest. By the way 2.17 is an appropriate version to target. RHEL6 much? :D
    - dgrunwald 2y ago
      You don't need a distro with ancient glibc; you can instead create a cross-compilation toolchain. We used https://crosstool-ng.github.io/docs/ https://crosstool-ng.github.io/docs/ to create a cross-compilation toolchain that runs on modern debian, but produces executables that only need glibc 2.17. The only downside is that you no longer can simply apt-get a library you need -- all dependencies shipped along with your application also need to be built with this cross-compiler.
    - forrestthewoods 2y ago
      > Setting up a machine or build system for older glibc is not trivial . God, I hate glibc so much. It’s an absolutely terrible design from the 80s. It can and should be trivial to target any arbitrary version of glibc. The way Linux performs linking is so bad and so much worse than Windows. (Headers + thin import lib. Linking expecting a full and complete so is stupid and bad.) Zig solved this problem. It can trivially target any version of glibc and cross-compile. It’d be nice if the Linux community demanded this level of functionality from their toolchains. Alas. https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html https://andrewkelley.me/post/zig-cc-powerful-drop-in-replace...
supriyo-biswas 2y ago
The glibc dependency can usually be eliminated through the same process of static compilation; the binaries I linked are also built this way with no dependencies on glibc, for example. > platforms that don’t offer stable ABI guarantees If we're talking about Windows, the Win32 ABI is considered stable, which would also suffice for what I'm looking for, which are portable binaries that can be run on almost any version of a system.
- rcxdude 2y ago
  statically linking against glibc does break some things (it's not an officially supported option because of that), though musl is an option and for most apps a reasonable replacement.
  - lloeki 2y ago
    At work we have this C++ that needs to be distributed as, in essence, a shared lib, and that needs to work with a lot of variability. So it has C++ and so needs a C++ stdlib. Dynamically linking is annoying so what if we statically link it? Oh but GNU's C++ stdlib wants glibc and has versioned BS and so even statically linking the C++ bits isn't ideal. And as mentioned statically linking glibc is a big no no. So: a) build a musl and install it in a prefix b) add a few bits (some headers and C runtime objects) and turn it into a sysroot c) build llvm's libc++ libunwind and a couple others, linking against that sysroot, and install it in the sysroot d) build our shared lib, statically linking against that sysroot The result is a shared lib that is self-contained except for the C stdlib, and any reference to the C stdlib has no GNUism nor glibc version. The trick is that musl is essentially a subset of glibc + there is no such thing as a musl-ism per musl's design tenets + the C ABI is stable enough. I don't recall us statically linking to musl - IIRC the trick is just to not have any GNUism at any level, in a way it also working in musl is kind of a happy side effect - but maybe I'm wrong. As a consequence it works on all glibc versions and musl as well, and is entirely independent of the system's preferred C++ stdlib. A requirement though is to load the shared library with RTLD_LOCAL (to not accidentally make internal symbols available to the outside) and RTLD_DEEPBIND (to not accidentally pick external symbols instead of internal ones)
- 2y ago
tripflag 2y ago
Replacing glibc with musl is usually unproblematic. That's what I did for my own portable toolbox which builds to a single self-extracting binary; https://github.com/9001/lxc https://github.com/9001/lxc

pjmlp 2y ago

> Instead, the proper way to address this situation would be to compile static binaries, that, by their very nature, become distribution agnostic. The only reason I assume this wasn't pursued, I assume, would be that the developers working on this weren't even aware this was an option. Nowadays you even have younger developers praising static linking, as if this wasn't what we had around since the invention of the first linker, and dynamic linking only became mainstream in the 1990's. Naturally only knowing Linux, GCC and glibc doesn't help.

almostgotcaught 2y ago
> Nowadays you even have younger developers praising static linking, as if this wasn't what we had around since the invention of the first linker, and dynamic linking only became mainstream in the 1990's. > Naturally only knowing Linux, GCC and glibc doesn't help. It's amazing how low-brow condescension passes for wisdom. Let me give you a hint: ontogeny does not actually recapitulate phylogeny. So just because you have some vague memories of previous iterations of a technology does not mean that it's the same thing being discussed today
- pjmlp 2y ago
  Here is another hint, don't waste time playing word games, assuming that I even get bothered with comments like yours. Modern Internet is much more friendly than Usenet forums, if you get my point.
alexey-salmin 2y ago
I'm not too young and I saw various methods of deploying code in production: deb packages, tarballs, docker images, static binaires. I have to say that in my experience for a specific use case of a highload c++ daemon, nothing beats the simplicity and reliability of the static binary with musl. You only want to deploy one thing, you have nothing to share, furthermore you wouldn't want to share anything for the sake of control. So many things to worry about and to waste time on just disappear. When things are on fire having less variables to check is gold.

otabdeveloper4 2y ago

Containers fix nothing, because inside your container is still a Linux distribution. (Often it's just Ubuntu, because why not.) > the proper way to address this situation would be to compile static binaries Nix is the proper way to address this. We'll get there eventually anyways, might as well bite the bullet now.

supriyo-biswas 2y ago
I don't think Nix solves this use case, the linked binaries would still have dependencies stored at /nix/store/... and is hardly a solution for obtaining binaries that work anywhere.
- rkangel 2y ago
  It's not about obtaining binaries that work anywhere, it's about being able to ask for a predictable environment to run in. You don't have to care about the target environment if you can control the derivation that is built to run your program. I think we also need to separate "Nix" from "The General Approach that is implemented by Nix (and Guix etc.)". I think the "The Nix Approach" (as I'm now going to call it) is fantastic and really is the solution to a lot of problems that we have. I think that the actual implementation of Nix isn't great and painful to use in real life.
  - supriyo-biswas 2y ago
    > You don't have to care about the target environment if you can control the derivation that is built to run your program. This is not always possible, since people may have workflows based on different containers or OSes. I (and the author of the article) want to have tools that work across different environments. > The General Approach that is implemented by Nix (and Guix etc.) "Reproducible builds" or "hermetic environments" are commonly used adjectives for this, although they are not limited to these two tools either; Bazel from Google does it too, for example.
    - otabdeveloper4 2y ago

zvr 2y ago

This is perfectly OK for your own work, but if you want to distribute stuff to others, statically linking increases your workload a lot. Because you are essentially distributing an increased number of components and your obligations increase multi-fold.