7 ms·

The biggest impact suckless had on me was via. Their Stali Linux FAQ: https://sta.li/faq/ https://sta.li/faq/ . They've built an entirely statically linked use

The biggest impact suckless had on me was via. Their Stali Linux FAQ: https://sta.li/faq/ https://sta.li/faq/ . They've built an entirely statically linked user space for Linux . Until then i never questioned the default Linux "shared libraries for everything" approach and assumed that was the best way to deliver software. Every little cli tool i wrote at work - i used to create distro packages for them or a tarball with a shell script that set LD_LIBRARY_PATH to find the correct version of the xml libraries etc i used. It didn't have to be this way. Dealing with distro versioning headaches or the finnicky custom packaging of the libraries into that tar ball just to let the users run by 150 kb binary. Since then I've mostly used static linking where i can. AppImages otherwise. I'm not developing core distro libraries. I'm just developing a tiny "app" my users need to use. I'm glad with newer languages like Go etc... static linking is the default. Don't get me wrong. Dynamic linking definitely has it's place. But by default our software deployment doesn't need to be this complicated.

hyperbrainer 2y ago
https://github.com/oasislinux/oasis https://github.com/oasislinux/oasis is also a a small statically-linked linux system using musl and the like. Curious about your thoughts on that?
Levitating 2y ago
Also the stali approaches to filesystem hierarchy is aspiring https://sta.li/filesystem/ https://sta.li/filesystem/ > AppImages AppImages require a large amount of (obsolete) dependencies to run, making their portabiltiy practically worthless. Newer immutable distros like Aeon don't ship the necessary packages to run an AppImage.
kazinator 2y ago
Two things: 1. You can bake the LD_LIBRARY_PATH into your executable with the -rpath mechanism. 2. Both rpath and LD_LIBRARY_PATH expand certain tokens such as $ORIGIN. $ORIGIN expands to the location of the program. So you can (for instance) set a rpath of $ORIGIN in your executable, and then it will look for its libs in its own directory. Just like is the intelligent default on on you-know-what Redmond operating system everyone rags on.
jimmaswell 2y ago
There's definitely value in the static approach in some cases, but there are some downsides e.g. your utility will need to be recompiled and updated if a security vulnerability is discovered in one of those libraries. You also miss out on free bugfixes without recompiling. If you require a library, you can specify it as a dependency in your dpkg/pacman/portage/whatever manifest and the system should take care of making it available. You shouldn't need to write custom scripts that trawl around for the library. Another approach could be to give your users a "make install" that sticks the libraries somewhere in /opt and adds it as the lowest priority ld_library_path as a last resort, maybe?
- butterisgood 2y ago
  "Free bug fixes without compiling". I think YMMV. It depends a lot on ABI/API stability and actual modularity of ... components. There's not always a guarantee of that. Shared libraries add a lot of complexity to a system for the assumption that people can actually build modular code well in any language that can create a shared library. Sometimes you have to recompile because, while a #define might still exist, its value may have changed between versions, and chaos can ensue - including new, unexpected bugs - for free!
  - dgfitz 2y ago
    My current day job has probably 60 apps the depend on one shared library. Static linking has its place pace, no doubt, but it should not be the norm.
    - zamadatix 2y ago
      60 programs y'all maintain as part of an overall larger solution or 60 randomly selected apps maintained fully by 3rd parties that just happen to share the same major library version? The former often makes a lot of sense in terms of deployment or maintenance - it's all really 1 big system so why rebuild and deploy 60 parts to change 1 when it's all done by you anyways. I'm not sure that's really what every build environment should default to assuming is the use case though but going shared libraries makes a ton of sense for that regardless. 60 programs maintained and built by 3rd parties which just happen to share major version of a library (other than something like stdlib obviously) would seem nuts to manage in a single runtime environment (regardless of static or shared) though!
realo 2y ago
You might want to take a good look at how Nix and Guix do things..
- saidinesh5 2y ago
  I know. But Nix doesn't make any of this complexity go away. It just helps you to tame this complexity and give you a reproducible system. Even Gobo Linux for that matter. At the end of the day the apps were a simple end user applications. They used a handful of library functions from different libraries. My users cared about just using my apps to do whatever the apps did. I just care that my app should work on their machines easily - no matter what version of what distro they're using.
  - Imustaskforhelp 2y ago
    Gobo linux seems really interesting.
delta_p_delta_x 2y ago
Fun fact... Many Windows programs generally do some sort of 'hybrid static linking' (this is my own terminology) where programs are distributed with the `.dll` libraries just next to the binary. There is no concept of RPATH on Windows—the loader looks for dynamically-linked libraries in a fixed set of locations which includes the binary's directory. Windows programs generally do link dynamically to core Windows libraries—which users are never expected to mess with anyway—and the C and C++ runtimes, but even these can be statically linked against with `cl.exe /MT`. Some programs even distribute newer versions of the C/C++ runtimes; that's where the famous Visual C++ Redistributables come from. I agree, though—static linkage should be the default for end-user programs. I long for a time when Linux gets 'libc' redistributables and I can compile for an old-hat CentOS 6 distribution on a modern, updated, rolling-release Arch Linux without faffing with a Docker container.
- GoblinSlayer 2y ago
  Compiling for CentOS 6 is a linking problem, and any linker lets you link with whatever you want, it's a matter of running the linker with the right arguments.
- int_19h 2y ago
  > There is no concept of RPATH on Windows—the loader looks for dynamically-linked libraries in a fixed set of locations which includes the binary's directory. This is not true - you can control the DLL path via manifests. There's also a "known DLLs" list in registry which can globally redirect basically any DLL system-wide.
- kazinator 2y ago
  Many programs link various libraries of theirs statically, but then dynamically link to the system C library. For instance, GNU Make uses some of GNULib. https://git.savannah.gnu.org/cgit/make.git https://git.savannah.gnu.org/cgit/make.git (gl subdirectory)
- sunshowers 2y ago
  For Rust, cargo zigbuild lets you compile against arbitrary old glibc. Probably can get that working with C as well.
jclulow 2y ago
The thing is, dynamic linking doesn't mean using LD_LIBRARY_PATH or building full blown OS packages as the only way to find the correct libraries. There's a first class facility for locating shared libraries, using the -R flag to provide a RUNPATH/RPATH in the binary. The runtime link editor will use that path to locate shared libraries. You can make your binaries relocatable as well, by using $ORIGIN in the RPATH: this gets expanded at runtime to the path of the executable, so, e.g., $ORIGIN/../lib would go up one from bin/ where the executable is and down alongside into the lib directory for your software. LD_LIBRARY_PATH is a debugging and software engineering tool, and shouldn't ever be part of shipped software.
- fanf2 2y ago
  Build systems often make it a huge pain to get the right rpath. The chrpath tool makes it easy to fix the rpath after libtool got it wrong.
  - kazinator 2y ago
    If you know that the executables will be in a certain directory, such that ../lib relative to that is where the libraries are found, then you just make the rpath '$ORIGIN/../lib'. That's it; no guesswork. Build systems that make it a huge paint to calculate elaborate rpath, e.g. knowing what the sysroot will translate to on the actual target system or whatever, are just doing counterproductive nonsense. By the way, I seem to recall that Yocto uses chrpath in order to get its own tools to reference its own copy of glibc that it provides (for itself, in the build environment).
  - ttyprintk 2y ago
    The suckless tools are recompile-to-reconfigure, so they don’t hit the length limitation of chrpath or fatal bugs in patchelf.
    - kazinator 2y ago
      You wouldn't have a length limitation in using rpath just to express "find the libs near the executables, wherever those are".
o11c 2y ago
The problem is that static libraries are actually more likely to break across time in practice, since "the system" is more than just "syscalls". For example, in places where the a filesystem-related "standard" has changed, I have old static binaries that fail to start entirely, whereas the same-dated dynamic libraries just need to bother to actually install dependencies. I am convinced that every argument in favor of static linking is because they don't know how to package-manager.
- int_19h 2y ago
  The sane thing here is to maintain a clear notion of what the "OS" is versus the "app", and use dynamic linking on that boundary, but not elsewhere. Which is more or less how Windows and macOS do things.
- deleted 2y ago
  [deleted]
- 7bit 2y ago
  > I am convinced that every argument in favor of static linking is because they don't know how to package-manager. Which would be a fair reason. People who like to build things might just not want to also learn how to package stuff.
  - kelnos 2y ago
    It's not all-or-nothing, though. If you need a dependency that isn't widely available on distros, then statically link it. It's fine. No big deal. But if you actually care about being a responsible maintainer, make sure you follow new releases of that dependency, and release new versions of your app (with the new version of the dependency) if there's an important bug fix to it. If you're linking to libX11 or libgtk or something else that's common, rely on the distro providing it. I really don't get all the anti-shared-library sentiment. I've been using and developing software for Linux for a good 25 years now, and yes, I've certainly had issues with library version mismatches, but a) they were never all that difficult to solve, and b) I think the last time I had an issue was more than a decade ago. While I think my experience is probably fairly typical, I won't deny that others can have worse experiences than I do. But I'm still not convinced statically linking everything (or even a hybrid approach where static linking is more common) would be an overall improvement.
the-lazy-guy 2y ago
Unrelated to suckless, there's a project (confusingly) named stal/IX: https://stal-ix.github.io/ https://stal-ix.github.io/ It is also a statically linked Linux distribution. But it's core idea is reproducible nix-style builds (including installing as many different versions/build configurations of any package), but with less pl fuff (no fancy funcional language - just some ugly jinja2/shell style build descriptions; which in practice work amazingly well, because underlying package/dependency model is very solid - https://stal-ix.github.io/IX.html https://stal-ix.github.io/IX.html). It is very opionated (just see this - https://stal-ix.github.io/STALIX.html https://stal-ix.github.io/STALIX.html), and a bit rough, but I was able to run it in VMs sucessfully. It would be amazing if it stabilizes one day.
zahlman 2y ago
...But the point of dynamic linking is that it allows your tiny apps to actually be tiny, yes?
- freeone3000 2y ago
  The value of that isn’t what it used to be — an app being 0.1MB vs 3MB doesn’t even register in the world of 300MB apps
snarfy 2y ago
It's a trade-off. Consider how dynamic linking libc works when a critical security bug is found and fixed. To update your system you update libc.so. If it were statically linked, you need to update your whole distribution.
- aqueueaqueue 2y ago
  The article touches on that. If "your whole distribution" comparable to a Chrome in size? That gets updated.
- fc417fc802 2y ago
  I would be more accepting of the trade-off if it wasn't so brittle in practice. Nix is much closer to a "good" dynamic linking solution IMO except that it makes it overly difficult to swap things out at runtime. I appreciate that the default is reproducible and guaranteed to correspond to the hash but sometimes I want to override that at runtime for various reasons. (It's possible this has changed since I last played with that tooling. It's been awhile.)
  - beefsack 2y ago
    I generally find with Nix and NixOS that I'm able to just use a dev shell to create little custom environments at runtime as needed. Another option is `mkOutOfStoreSymlink` if you want some dynamic config for some GUI you are running. Depends on what you are trying to achieve though.
    - fc417fc802 2y ago
      (Apologies for off topic.) Suppose your dev shell contains app Foo v1 which uses lib Bar v2. It links directly against the full nix store path. So loading a different version of lib Bar (say v3) into your dev shell, which puts it on your PATH, doesn't affect Foo - the Foo executable will still link against Bar v2. That's by design and a very good thing! It assures reproducibility and fixes the version incompatibility issue that dynamic linking typically suffers from. However, what if I want to swap it out for whatever reason? Right now I have to modify the package definition and rebuild it. That's needlessly slow and resource intensive for some one off monkey patch. It also might not be feasible at all on my hardware - most people's rigs aren't cut out to build chromium or llvm, for example. The UX is also absolutely terrible. Have you ever tried to patch a nixpkgs provided definition? I actually gave up on using nix as my primary build tool over this. My expectation going in was a neat and tidy collection of independent, standalone package definitions that imported one another to satisfy dependencies as necessary. The reality was a massive monolith containing all packages at once making extracting just one to patch it rather complicated in some cases. I believe flakes were supposed to address this exact dependency modification UX issue somewhat, but they caused my code to break in other ways and I decided I was done sinking time into tinkering with tooling rather than building software.
treeshateorcs 2y ago
statically built helloworld in zig is like 8KB
Imustaskforhelp 2y ago
I also like the static approach , there is staticx which I am loving which can take a dynamic binary and make it static. I am wondering if I could create something like .deb /.rpm / appimage and flatpak all into static binaries which can work on any device
1vuio0pswjnm7 2y ago
I have a non-LFS, Stali-like project for myself, 100% statically-linked Linux. Probably the biggest PITA for me is compiling a statically-linked cmake. Even just compiling a dynamically-linked cmake takes an inordinate amount of time.
skywal_l 2y ago
Linus Torvalds agrees with you: https://youtu.be/Pzl1B7nB9Kc?feature=shared&t=65 https://youtu.be/Pzl1B7nB9Kc?feature=shared&t=65

hyperbrainer 2y ago

https://github.com/oasislinux/oasis https://github.com/oasislinux/oasis is also a a small statically-linked linux system using musl and the like. Curious about your thoughts on that?

Levitating 2y ago

Also the stali approaches to filesystem hierarchy is aspiring https://sta.li/filesystem/ https://sta.li/filesystem/ > AppImages AppImages require a large amount of (obsolete) dependencies to run, making their portabiltiy practically worthless. Newer immutable distros like Aeon don't ship the necessary packages to run an AppImage.

kazinator 2y ago

Two things: 1. You can bake the LD_LIBRARY_PATH into your executable with the -rpath mechanism. 2. Both rpath and LD_LIBRARY_PATH expand certain tokens such as $ORIGIN. $ORIGIN expands to the location of the program. So you can (for instance) set a rpath of $ORIGIN in your executable, and then it will look for its libs in its own directory. Just like is the intelligent default on on you-know-what Redmond operating system everyone rags on.

jimmaswell 2y ago

There's definitely value in the static approach in some cases, but there are some downsides e.g. your utility will need to be recompiled and updated if a security vulnerability is discovered in one of those libraries. You also miss out on free bugfixes without recompiling. If you require a library, you can specify it as a dependency in your dpkg/pacman/portage/whatever manifest and the system should take care of making it available. You shouldn't need to write custom scripts that trawl around for the library. Another approach could be to give your users a "make install" that sticks the libraries somewhere in /opt and adds it as the lowest priority ld_library_path as a last resort, maybe?

butterisgood 2y ago
"Free bug fixes without compiling". I think YMMV. It depends a lot on ABI/API stability and actual modularity of ... components. There's not always a guarantee of that. Shared libraries add a lot of complexity to a system for the assumption that people can actually build modular code well in any language that can create a shared library. Sometimes you have to recompile because, while a #define might still exist, its value may have changed between versions, and chaos can ensue - including new, unexpected bugs - for free!
- dgfitz 2y ago
  My current day job has probably 60 apps the depend on one shared library. Static linking has its place pace, no doubt, but it should not be the norm.
  - zamadatix 2y ago
    60 programs y'all maintain as part of an overall larger solution or 60 randomly selected apps maintained fully by 3rd parties that just happen to share the same major library version? The former often makes a lot of sense in terms of deployment or maintenance - it's all really 1 big system so why rebuild and deploy 60 parts to change 1 when it's all done by you anyways. I'm not sure that's really what every build environment should default to assuming is the use case though but going shared libraries makes a ton of sense for that regardless. 60 programs maintained and built by 3rd parties which just happen to share major version of a library (other than something like stdlib obviously) would seem nuts to manage in a single runtime environment (regardless of static or shared) though!

realo 2y ago

You might want to take a good look at how Nix and Guix do things..

saidinesh5 2y ago
I know. But Nix doesn't make any of this complexity go away. It just helps you to tame this complexity and give you a reproducible system. Even Gobo Linux for that matter. At the end of the day the apps were a simple end user applications. They used a handful of library functions from different libraries. My users cared about just using my apps to do whatever the apps did. I just care that my app should work on their machines easily - no matter what version of what distro they're using.
- Imustaskforhelp 2y ago
  Gobo linux seems really interesting.

delta_p_delta_x 2y ago

Fun fact... Many Windows programs generally do some sort of 'hybrid static linking' (this is my own terminology) where programs are distributed with the `.dll` libraries just next to the binary. There is no concept of RPATH on Windows—the loader looks for dynamically-linked libraries in a fixed set of locations which includes the binary's directory. Windows programs generally do link dynamically to core Windows libraries—which users are never expected to mess with anyway—and the C and C++ runtimes, but even these can be statically linked against with `cl.exe /MT`. Some programs even distribute newer versions of the C/C++ runtimes; that's where the famous Visual C++ Redistributables come from. I agree, though—static linkage should be the default for end-user programs. I long for a time when Linux gets 'libc' redistributables and I can compile for an old-hat CentOS 6 distribution on a modern, updated, rolling-release Arch Linux without faffing with a Docker container.

GoblinSlayer 2y ago
Compiling for CentOS 6 is a linking problem, and any linker lets you link with whatever you want, it's a matter of running the linker with the right arguments.
int_19h 2y ago
> There is no concept of RPATH on Windows—the loader looks for dynamically-linked libraries in a fixed set of locations which includes the binary's directory. This is not true - you can control the DLL path via manifests. There's also a "known DLLs" list in registry which can globally redirect basically any DLL system-wide.
kazinator 2y ago
Many programs link various libraries of theirs statically, but then dynamically link to the system C library. For instance, GNU Make uses some of GNULib. https://git.savannah.gnu.org/cgit/make.git https://git.savannah.gnu.org/cgit/make.git (gl subdirectory)
sunshowers 2y ago
For Rust, cargo zigbuild lets you compile against arbitrary old glibc. Probably can get that working with C as well.

jclulow 2y ago

The thing is, dynamic linking doesn't mean using LD_LIBRARY_PATH or building full blown OS packages as the only way to find the correct libraries. There's a first class facility for locating shared libraries, using the -R flag to provide a RUNPATH/RPATH in the binary. The runtime link editor will use that path to locate shared libraries. You can make your binaries relocatable as well, by using $ORIGIN in the RPATH: this gets expanded at runtime to the path of the executable, so, e.g., $ORIGIN/../lib would go up one from bin/ where the executable is and down alongside into the lib directory for your software. LD_LIBRARY_PATH is a debugging and software engineering tool, and shouldn't ever be part of shipped software.

fanf2 2y ago
Build systems often make it a huge pain to get the right rpath. The chrpath tool makes it easy to fix the rpath after libtool got it wrong.
- kazinator 2y ago
  If you know that the executables will be in a certain directory, such that ../lib relative to that is where the libraries are found, then you just make the rpath '$ORIGIN/../lib'. That's it; no guesswork. Build systems that make it a huge paint to calculate elaborate rpath, e.g. knowing what the sysroot will translate to on the actual target system or whatever, are just doing counterproductive nonsense. By the way, I seem to recall that Yocto uses chrpath in order to get its own tools to reference its own copy of glibc that it provides (for itself, in the build environment).
- ttyprintk 2y ago
  The suckless tools are recompile-to-reconfigure, so they don’t hit the length limitation of chrpath or fatal bugs in patchelf.
  - kazinator 2y ago
    You wouldn't have a length limitation in using rpath just to express "find the libs near the executables, wherever those are".

o11c 2y ago

The problem is that static libraries are actually more likely to break across time in practice, since "the system" is more than just "syscalls". For example, in places where the a filesystem-related "standard" has changed, I have old static binaries that fail to start entirely, whereas the same-dated dynamic libraries just need to bother to actually install dependencies. I am convinced that every argument in favor of static linking is because they don't know how to package-manager.

int_19h 2y ago
The sane thing here is to maintain a clear notion of what the "OS" is versus the "app", and use dynamic linking on that boundary, but not elsewhere. Which is more or less how Windows and macOS do things.
deleted 2y ago
[deleted]
7bit 2y ago
> I am convinced that every argument in favor of static linking is because they don't know how to package-manager. Which would be a fair reason. People who like to build things might just not want to also learn how to package stuff.
- kelnos 2y ago
  It's not all-or-nothing, though. If you need a dependency that isn't widely available on distros, then statically link it. It's fine. No big deal. But if you actually care about being a responsible maintainer, make sure you follow new releases of that dependency, and release new versions of your app (with the new version of the dependency) if there's an important bug fix to it. If you're linking to libX11 or libgtk or something else that's common, rely on the distro providing it. I really don't get all the anti-shared-library sentiment. I've been using and developing software for Linux for a good 25 years now, and yes, I've certainly had issues with library version mismatches, but a) they were never all that difficult to solve, and b) I think the last time I had an issue was more than a decade ago. While I think my experience is probably fairly typical, I won't deny that others can have worse experiences than I do. But I'm still not convinced statically linking everything (or even a hybrid approach where static linking is more common) would be an overall improvement.

the-lazy-guy 2y ago

Unrelated to suckless, there's a project (confusingly) named stal/IX: https://stal-ix.github.io/ https://stal-ix.github.io/ It is also a statically linked Linux distribution. But it's core idea is reproducible nix-style builds (including installing as many different versions/build configurations of any package), but with less pl fuff (no fancy funcional language - just some ugly jinja2/shell style build descriptions; which in practice work amazingly well, because underlying package/dependency model is very solid - https://stal-ix.github.io/IX.html https://stal-ix.github.io/IX.html). It is very opionated (just see this - https://stal-ix.github.io/STALIX.html https://stal-ix.github.io/STALIX.html), and a bit rough, but I was able to run it in VMs sucessfully. It would be amazing if it stabilizes one day.

zahlman 2y ago

...But the point of dynamic linking is that it allows your tiny apps to actually be tiny, yes?

freeone3000 2y ago
The value of that isn’t what it used to be — an app being 0.1MB vs 3MB doesn’t even register in the world of 300MB apps

snarfy 2y ago

It's a trade-off. Consider how dynamic linking libc works when a critical security bug is found and fixed. To update your system you update libc.so. If it were statically linked, you need to update your whole distribution.

aqueueaqueue 2y ago
The article touches on that. If "your whole distribution" comparable to a Chrome in size? That gets updated.
fc417fc802 2y ago
I would be more accepting of the trade-off if it wasn't so brittle in practice. Nix is much closer to a "good" dynamic linking solution IMO except that it makes it overly difficult to swap things out at runtime. I appreciate that the default is reproducible and guaranteed to correspond to the hash but sometimes I want to override that at runtime for various reasons. (It's possible this has changed since I last played with that tooling. It's been awhile.)
- beefsack 2y ago
  I generally find with Nix and NixOS that I'm able to just use a dev shell to create little custom environments at runtime as needed. Another option is `mkOutOfStoreSymlink` if you want some dynamic config for some GUI you are running. Depends on what you are trying to achieve though.
  - fc417fc802 2y ago
    (Apologies for off topic.) Suppose your dev shell contains app Foo v1 which uses lib Bar v2. It links directly against the full nix store path. So loading a different version of lib Bar (say v3) into your dev shell, which puts it on your PATH, doesn't affect Foo - the Foo executable will still link against Bar v2. That's by design and a very good thing! It assures reproducibility and fixes the version incompatibility issue that dynamic linking typically suffers from. However, what if I want to swap it out for whatever reason? Right now I have to modify the package definition and rebuild it. That's needlessly slow and resource intensive for some one off monkey patch. It also might not be feasible at all on my hardware - most people's rigs aren't cut out to build chromium or llvm, for example. The UX is also absolutely terrible. Have you ever tried to patch a nixpkgs provided definition? I actually gave up on using nix as my primary build tool over this. My expectation going in was a neat and tidy collection of independent, standalone package definitions that imported one another to satisfy dependencies as necessary. The reality was a massive monolith containing all packages at once making extracting just one to patch it rather complicated in some cases. I believe flakes were supposed to address this exact dependency modification UX issue somewhat, but they caused my code to break in other ways and I decided I was done sinking time into tinkering with tooling rather than building software.

treeshateorcs 2y ago

statically built helloworld in zig is like 8KB

Imustaskforhelp 2y ago

I also like the static approach , there is staticx which I am loving which can take a dynamic binary and make it static. I am wondering if I could create something like .deb /.rpm / appimage and flatpak all into static binaries which can work on any device

1vuio0pswjnm7 2y ago

I have a non-LFS, Stali-like project for myself, 100% statically-linked Linux. Probably the biggest PITA for me is compiling a statically-linked cmake. Even just compiling a dynamically-linked cmake takes an inordinate amount of time.

skywal_l 2y ago

Linus Torvalds agrees with you: https://youtu.be/Pzl1B7nB9Kc?feature=shared&t=65 https://youtu.be/Pzl1B7nB9Kc?feature=shared&t=65