5 ms·

It's funny how people insist on wanting to link everything statically when shared libraries were specifically designed to have a better alternative. Even worse

It's funny how people insist on wanting to link everything statically when shared libraries were specifically designed to have a better alternative. Even worse is containers, which has the disadvantage of both.

vv_ 9mo ago
It's easier to distribute software fully self-contained, if you ignore the pain of statically linking everything together :)
- diego_sandoval 9mo ago
  What's the pain?
  - throwaway2046 9mo ago
    I'm guessing the pain of fighting the various build systems that insist on dynamic linking, sometimes against the user's explicit wishes.
  - vv_ 9mo ago
    Most open source software tooling Were designed to be dynamically linked. It is non-standard to statically link things together, which causes various random issues.
fc417fc802 9mo ago
Dynamic linking exists to make a specific set of tradeoffs. Neither better nor worse than static linking in the general sense.
arghwhat 9mo ago
Dynamic libraries have been frowned upon since their inception as being a terrible solution to a non-existent problem, generally amplifying binary sizes and harming performance. Some fun quotes of quite notable characters on the matter here: https://harmful.cat-v.org/software/dynamic-linking/ https://harmful.cat-v.org/software/dynamic-linking/ In practice, a statically linked system is often smaller than a meticulously dynamically linked one - while there are many copies of common routines, programs only contain tightly packed, specifically optimized and sometimes inlined versions of the symbols they use. The space and performance gain per program is quite significant. Modern apps and containers are another issue entirely - linking doesn't help if your issue is gigabytes of graphical assets or using a container base image that includes the entire world.
- holowoodman 9mo ago
  Statically linked binaries are a huge security problem, as are containers, for the same reason. Vendors are too slow to patch. When dynamically linking against shared OS libraries, Updates are far quicker and easier. And as for the size advantage, just look at a typical Golang or Haskell program. Statically linked, two-digit megabytes, larger than my libc...
  - zbentley 9mo ago
    I've heard this many times, and while there might be data out there in support of it, I've never seen that, and my anecdotal experience is more complicated. In the most security-forward roles I've worked in, the vast, vast majority of vulnerabilities identified in static binaries, Docker images, Flatpaks, Snaps, and VM appliance images fell into these categories: 1. The vendor of a given piece of software based their container image on an outdated version of e.g. Debian, and the vulnerabilities were coming from that, not the software I cared about. This seems like it supports your point, but consider: the overwhelming majority of these required a distro upgrade, rather than a point dependency upgrade of e.g. libcurl or whatnot, to patch the vulnerabilities. Countless times, I took a normal long-lived Debian test VM and tried to upgrade it to the patched version and then install whatever piece of software I was running in a docker image, and had the upgrade fail in some way (everything from the less-common "doesn't boot" to the very-common "software I wanted didn't have a distribution on its website for the very latest Debian yet, so I was back to hand-building it with all of the dependencies and accumulated cruft that entails"). 2. Vulnerabilities that were unpatched or barely patched upstream (as in: a patch had merged but hadn't been baked into released artifacts yet--this applied equally to vulns in things I used directly, and vulns in their underlying OSes). 3. Massive quantities of vulnerabilities reported in "static" languages' standard libraries. Golang is particularly bad here, both because they habitually over-weight the severity of their CVEs and because most of the stdlib is packaged with each Golang binary (at least as far as SBOM scanners are concerned). That puts me somewhat between a rock and a hard place. A dynamic-link-everything world with e.g. a "libgolang" versioned separately from apps would address the 3rd item in that list, but would make the 1st item worse. "Updates are far quicker and easier" is something of a fantasy in the realm of mainstream Linux distros (or copies of the userlands of those distros packaged into container images); it's certainly easier to mechanically perform an update of dependency components of a distro, but whether or not it actually works is another question. And I'm not coming at this from a pro-container-all-the-things background. I was a Linux sysadmin long before all this stuff got popular, and it used to be a little easier to do patch cycles and point updates before container/immutable-image-of-userland systems established the convention of depending on extremely specific characteristics of a specific revision of a distro. But it was never truly easy, and isn't easy today.
RicoElectrico 9mo ago
That would be a good point if said shared libraries did not break binary backwards compatibility and behaved more like winapi.
flohofwoe 9mo ago
Dynamic libraries make a lot of sense as operating system interface when they guarantee a stable API and ABI (see Windows for how to do that) - the other scenarios where DLLs make sense is for plugin systems. But that's pretty much it, for anything else static linking is superior because it doesn't present an optimization barrier (especially for dead code elimination). No idea why the glibc can't provide API+ABI stability, but on Linux it always comes down to glibc related "DLL hell" problems (e.g. not being able to run an executable that was created on a more recent Linux system on an older Linux system even when the program doesn't access any new glibc entry points - the usually adviced solution is to link with an older glibc version, but that's also not trivial, unless you use the Zig toolchain). TL;DR: It's not static vs dynamic linking, just glibc being a an exceptionally shitty solution as operating system interface.
- uecker 9mo ago
  I do not think it is difficult compiling against versions by using a container.
- AshamedCaptain 9mo ago
  > (e.g. not being able to run an executable that was created on a more recent Linux system on an older Linux system even when the program doesn't access any new glibc entry points - the usually adviced solution is to link with an older glibc version, but that's also not trivial, unless you use the Zig toolchain). In the era of containers, I do not understand why this is "Not trivial". I could do it with even a chroot.
  - throwaway2046 9mo ago
    Linking against an older glibc means setting up an older distribution and accepting all the outdated toolchains and libraries that come with it. Need to upgrade? Get ready to compile everything from source and possibly bootstrap a toolchain. I wouldn't call this trivial. The fact that you need to use a container/chroot on Linux in the first place makes the process non trivial, when all you have to do on Windows is click a button or two.
abigail95 9mo ago
Why would I want to be constantly calling into code I have no control over, that may or may not exist, that may or may not be tampered with. I lose control of the execution state. I have to follow the calling conventions which let my flags get clobbered. To forego all of the above including link time optimization for the benefit of what exactly? Imagine developing a C program where every object file produced during compilation was dynamically linked. It's obvious why that is a stupid idea - why does it become less stupid when dealing with a separate library?
- uecker 9mo ago
  You call into dynamic libraries so that you do not need to recompile and distribute new binaries to all your users whenever there is a security issue or other critical fix in any of the dependencies.
  - sebastos 9mo ago
    But if I get to Bring My Own Dependencies, then I know the exact versions of all my dependencies. That makes testing and development faster because I don’t have to expend effort testing across many different possible platforms. And if development is just generally easier, then maybe it’s easier to react expediently to security notices and release updates as necessary.. .
    - uecker 9mo ago
      You would need to monitor all your dependencies (and their dependencies), compile new binaries for all supported platform each time their is an issue (which you likely learn about later), notify all your user, and distribute improved binaries. I think this is far more effort than using dynamic libraries and compiling for a couple of Linux distributions. And I would be surprised if entities distributing statically linked binaries actually do this (properly).
gethly 9mo ago
Isn't the sole reason why linux sucks(sucked?) for games and other software exactly that there is a gazillion of different libraries with different versions, so you have zero assumptions about the state of the OS, which makes making sw for it such a pain?
- paddim8 9mo ago
  Yes. Premature optimisation when it comes to dynamic linking is the reason for why the year of the Linux desktop is far away in my opinion
  - gethly 9mo ago
    I just remember Jonathan Blow mentioning this in one of his streams.

vv_ 9mo ago

It's easier to distribute software fully self-contained, if you ignore the pain of statically linking everything together :)

diego_sandoval 9mo ago
What's the pain?
- throwaway2046 9mo ago
  I'm guessing the pain of fighting the various build systems that insist on dynamic linking, sometimes against the user's explicit wishes.
- vv_ 9mo ago
  Most open source software tooling Were designed to be dynamically linked. It is non-standard to statically link things together, which causes various random issues.

fc417fc802 9mo ago

Dynamic linking exists to make a specific set of tradeoffs. Neither better nor worse than static linking in the general sense.

arghwhat 9mo ago

Dynamic libraries have been frowned upon since their inception as being a terrible solution to a non-existent problem, generally amplifying binary sizes and harming performance. Some fun quotes of quite notable characters on the matter here: https://harmful.cat-v.org/software/dynamic-linking/ https://harmful.cat-v.org/software/dynamic-linking/ In practice, a statically linked system is often smaller than a meticulously dynamically linked one - while there are many copies of common routines, programs only contain tightly packed, specifically optimized and sometimes inlined versions of the symbols they use. The space and performance gain per program is quite significant. Modern apps and containers are another issue entirely - linking doesn't help if your issue is gigabytes of graphical assets or using a container base image that includes the entire world.

holowoodman 9mo ago
Statically linked binaries are a huge security problem, as are containers, for the same reason. Vendors are too slow to patch. When dynamically linking against shared OS libraries, Updates are far quicker and easier. And as for the size advantage, just look at a typical Golang or Haskell program. Statically linked, two-digit megabytes, larger than my libc...
- zbentley 9mo ago
  I've heard this many times, and while there might be data out there in support of it, I've never seen that, and my anecdotal experience is more complicated. In the most security-forward roles I've worked in, the vast, vast majority of vulnerabilities identified in static binaries, Docker images, Flatpaks, Snaps, and VM appliance images fell into these categories: 1. The vendor of a given piece of software based their container image on an outdated version of e.g. Debian, and the vulnerabilities were coming from that, not the software I cared about. This seems like it supports your point, but consider: the overwhelming majority of these required a distro upgrade, rather than a point dependency upgrade of e.g. libcurl or whatnot, to patch the vulnerabilities. Countless times, I took a normal long-lived Debian test VM and tried to upgrade it to the patched version and then install whatever piece of software I was running in a docker image, and had the upgrade fail in some way (everything from the less-common "doesn't boot" to the very-common "software I wanted didn't have a distribution on its website for the very latest Debian yet, so I was back to hand-building it with all of the dependencies and accumulated cruft that entails"). 2. Vulnerabilities that were unpatched or barely patched upstream (as in: a patch had merged but hadn't been baked into released artifacts yet--this applied equally to vulns in things I used directly, and vulns in their underlying OSes). 3. Massive quantities of vulnerabilities reported in "static" languages' standard libraries. Golang is particularly bad here, both because they habitually over-weight the severity of their CVEs and because most of the stdlib is packaged with each Golang binary (at least as far as SBOM scanners are concerned). That puts me somewhat between a rock and a hard place. A dynamic-link-everything world with e.g. a "libgolang" versioned separately from apps would address the 3rd item in that list, but would make the 1st item worse. "Updates are far quicker and easier" is something of a fantasy in the realm of mainstream Linux distros (or copies of the userlands of those distros packaged into container images); it's certainly easier to mechanically perform an update of dependency components of a distro, but whether or not it actually works is another question. And I'm not coming at this from a pro-container-all-the-things background. I was a Linux sysadmin long before all this stuff got popular, and it used to be a little easier to do patch cycles and point updates before container/immutable-image-of-userland systems established the convention of depending on extremely specific characteristics of a specific revision of a distro. But it was never truly easy, and isn't easy today.

RicoElectrico 9mo ago

That would be a good point if said shared libraries did not break binary backwards compatibility and behaved more like winapi.

flohofwoe 9mo ago

Dynamic libraries make a lot of sense as operating system interface when they guarantee a stable API and ABI (see Windows for how to do that) - the other scenarios where DLLs make sense is for plugin systems. But that's pretty much it, for anything else static linking is superior because it doesn't present an optimization barrier (especially for dead code elimination). No idea why the glibc can't provide API+ABI stability, but on Linux it always comes down to glibc related "DLL hell" problems (e.g. not being able to run an executable that was created on a more recent Linux system on an older Linux system even when the program doesn't access any new glibc entry points - the usually adviced solution is to link with an older glibc version, but that's also not trivial, unless you use the Zig toolchain). TL;DR: It's not static vs dynamic linking, just glibc being a an exceptionally shitty solution as operating system interface.

uecker 9mo ago
I do not think it is difficult compiling against versions by using a container.
AshamedCaptain 9mo ago
> (e.g. not being able to run an executable that was created on a more recent Linux system on an older Linux system even when the program doesn't access any new glibc entry points - the usually adviced solution is to link with an older glibc version, but that's also not trivial, unless you use the Zig toolchain). In the era of containers, I do not understand why this is "Not trivial". I could do it with even a chroot.
- throwaway2046 9mo ago
  Linking against an older glibc means setting up an older distribution and accepting all the outdated toolchains and libraries that come with it. Need to upgrade? Get ready to compile everything from source and possibly bootstrap a toolchain. I wouldn't call this trivial. The fact that you need to use a container/chroot on Linux in the first place makes the process non trivial, when all you have to do on Windows is click a button or two.

abigail95 9mo ago

Why would I want to be constantly calling into code I have no control over, that may or may not exist, that may or may not be tampered with. I lose control of the execution state. I have to follow the calling conventions which let my flags get clobbered. To forego all of the above including link time optimization for the benefit of what exactly? Imagine developing a C program where every object file produced during compilation was dynamically linked. It's obvious why that is a stupid idea - why does it become less stupid when dealing with a separate library?

uecker 9mo ago
You call into dynamic libraries so that you do not need to recompile and distribute new binaries to all your users whenever there is a security issue or other critical fix in any of the dependencies.
- sebastos 9mo ago
  But if I get to Bring My Own Dependencies, then I know the exact versions of all my dependencies. That makes testing and development faster because I don’t have to expend effort testing across many different possible platforms. And if development is just generally easier, then maybe it’s easier to react expediently to security notices and release updates as necessary.. .
  - uecker 9mo ago
    You would need to monitor all your dependencies (and their dependencies), compile new binaries for all supported platform each time their is an issue (which you likely learn about later), notify all your user, and distribute improved binaries. I think this is far more effort than using dynamic libraries and compiling for a couple of Linux distributions. And I would be surprised if entities distributing statically linked binaries actually do this (properly).

gethly 9mo ago

Isn't the sole reason why linux sucks(sucked?) for games and other software exactly that there is a gazillion of different libraries with different versions, so you have zero assumptions about the state of the OS, which makes making sw for it such a pain?

paddim8 9mo ago
Yes. Premature optimisation when it comes to dynamic linking is the reason for why the year of the Linux desktop is far away in my opinion
- gethly 9mo ago
  I just remember Jonathan Blow mentioning this in one of his streams.