4 ms·

The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not toda

by hctaw 6y ago

The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. Until then, I'll keep doing it, because it saves me time and money. I don't really buy that dynamic linking all the things is such a boon to security. But I'll link to this: https://drewdevault.com/dynlib https://drewdevault.com/dynlib > Not including libc, the only libraries which had "critical" or "high" severity vulnerabilities in 2019 which affected over 100 binaries on my system were dbus, gnutls, cairo, libssh2, and curl. 265 binaries were affected by the rest. > The total download cost to upgrade all binaries on my system which were affected by CVEs in 2019 is 3.8 GiB. This is reduced to 1.0 GiB if you eliminate glibc. > It is also unknown if any of these vulnerabilities would have been introduced after the last build date for a given statically linked binary; if so that binary would not need to be updated. Many vulnerabilities are also limited to a specific code path or use-case, and binaries which do not invoke that code path in their dependencies will not be affected. A process to ascertain this information in the wake of a vulnerability could be automated.

AshamedCaptain 6y ago
> But I'll link to this: https://drewdevault.com/dynlib https://drewdevault.com/dynlib We already discussed this in HN https://news.ycombinator.com/item?id=23654353 https://news.ycombinator.com/item?id=23654353 . Both data and conclusions are utterly suspect.
sigjuice 6y ago
The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. Maybe do the trendy thing and link your whole distro together with your app (a.k.a containers)?
- forrestthewoods 6y ago
  The fact that containers exist and are prevalent is a damning indictment on the Linux model imho. Producing software that can simply launch is so wildly complex that the solution is... to snapshot an entire operating system install to run the application. That’s INSANE. But it’s unfortunately required. https://xkcd.com/1987/ https://xkcd.com/1987/ > Now, for the worst of all — one that combines all the aforementioned issues, and adds even more. Bundling (often called vendoring in newspeak) means including the dependencies of your program along with it. The exact consequences of bundling vary depending on the method used. Let’s consider this statement at face value. The WORST thing a program can do is... include the things the program needs to run? So programs should... NOT include the things they REQUIRE to run? I get it. I understand the argument. But if you take just a half step back it should be clear how utterly broken the whole situation is.
  - ccday 6y ago
    I'm not sure it's fair to characterize a container as a snapshot of an entire OS install - you can have containers that are very lightweight, e.g. just a static Go binary copied into a "scratch" container - but often the required effort is not put in to reduce size.
  - cycloptic 6y ago
    You are comparing having containers versus having no containers and no kind of bundling at all, which is not a reasonable comparison. If you need an OS snapshot, the only option on other operating systems is often to use a VM. That's what containers are used as an alternative to. (The other alternative there is to never install patches or upgrades on the production machines, which probably creates as many problems as using containers)
    - forrestthewoods 6y ago
      I’m not sure I made that comparison. I’m saying the popularity of containers is an existence proof that the distro-oriented shared library system is failing to meet the needs of common use cases. Programs that bundle dependencies have a radically reduced need for snapshots, VMs, or containers. Such tools do provide a variety of value. But they don’t become virtually requirements to merely launch a program without error. Ship your damn dependencies says I. Either statically or dynamically, but ship them.
      - cycloptic 6y ago
        You can't get rid of the distro-oriented shared library system, it exists as long as you are building on top of an operating system. To that end, when you say "we depend on operating system minimum version X.Y.Z" that now becomes another dependency you have to ship at some point if you're running the production machines. Containers are popular because they actually solve that exact problem. If there was some other comparison you were making I'd be interested to hear it, but AFAIK containers can only help here so it's not clear what else you were comparing to.
orra 6y ago
> I don't really buy that dynamic linking all the things is such a boon to security. Agreed. It's really not a panacea. When you upgrade a library, you probably want to restart the running applications that depend upon this. Dynamic linking won't save you. The solution is having a graph of your dependencies! IIRC, NixOS gets this right. I don't think Debian's apt did?
- 1MachineElf 6y ago
  I would really like to know how NixOS solves this problem for things like yarn, webpack, parcel, esbuild, npm, pip, conda, cargo, stack, go modules, ruby gems, etc.
  - tadfisher 6y ago
    Usually there are tools to extract the necessary dependency information (url and hash) from a package manager's lockfile and produce a fixed-output derivation so a Nix expression can reproduce the build environment. I know of yarn2nix, cargo2nix, and node2nix, and I wrote and maintain gradle2nix for JVM projects.
  - mands 6y ago
    It's doesn't really, and that's the problem. There are automated tools and scripts to help, but there are always edge cases so that you end up going back to using pip for example. Nix and language package managers just don't really play well together.
- ece 6y ago
  There are scripts and daemons that help you determine what needs restarting[1,2]. NixOS installs can go in separate directory prefixes when there are conflicts. For Gentoo and other Linux distributions, maintainers usually won't mark something stable without resolving conflicts, and this usually means sticking to older stable version of libraries until newer versions of libraries are fully supported by all installed packages. This can definitely be more work for maintainers, and as the blog posts says, it's a sisyphean task. [1] https://manpages.debian.org/buster/debian-goodies/checkrestart.1.en.html https://manpages.debian.org/buster/debian-goodies/checkresta... [2] https://github.com/liske/needrestart https://github.com/liske/needrestart
drran 6y ago
It's why we like to use LGPL for libraries: if you ship statically linked software, ship your source too, so we will be able to patch and recompile.
kasabali 6y ago
> The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. That's completely missing point of the article. You're free to ship your program binary however you want as the upstream. It only becomes a problem if dynamic linking is a second class citizen in the language or build tool you use, you bundle dependencies and don't support un-bundling, or pin specific dependency versions.

by hctaw 6y ago

AshamedCaptain 6y ago

> But I'll link to this: https://drewdevault.com/dynlib https://drewdevault.com/dynlib We already discussed this in HN https://news.ycombinator.com/item?id=23654353 https://news.ycombinator.com/item?id=23654353 . Both data and conclusions are utterly suspect.

sigjuice 6y ago

The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. Maybe do the trendy thing and link your whole distro together with your app (a.k.a containers)?

forrestthewoods 6y ago
The fact that containers exist and are prevalent is a damning indictment on the Linux model imho. Producing software that can simply launch is so wildly complex that the solution is... to snapshot an entire operating system install to run the application. That’s INSANE. But it’s unfortunately required. https://xkcd.com/1987/ https://xkcd.com/1987/ > Now, for the worst of all — one that combines all the aforementioned issues, and adds even more. Bundling (often called vendoring in newspeak) means including the dependencies of your program along with it. The exact consequences of bundling vary depending on the method used. Let’s consider this statement at face value. The WORST thing a program can do is... include the things the program needs to run? So programs should... NOT include the things they REQUIRE to run? I get it. I understand the argument. But if you take just a half step back it should be clear how utterly broken the whole situation is.
- ccday 6y ago
  I'm not sure it's fair to characterize a container as a snapshot of an entire OS install - you can have containers that are very lightweight, e.g. just a static Go binary copied into a "scratch" container - but often the required effort is not put in to reduce size.
- cycloptic 6y ago
  You are comparing having containers versus having no containers and no kind of bundling at all, which is not a reasonable comparison. If you need an OS snapshot, the only option on other operating systems is often to use a VM. That's what containers are used as an alternative to. (The other alternative there is to never install patches or upgrades on the production machines, which probably creates as many problems as using containers)
  - forrestthewoods 6y ago
    I’m not sure I made that comparison. I’m saying the popularity of containers is an existence proof that the distro-oriented shared library system is failing to meet the needs of common use cases. Programs that bundle dependencies have a radically reduced need for snapshots, VMs, or containers. Such tools do provide a variety of value. But they don’t become virtually requirements to merely launch a program without error. Ship your damn dependencies says I. Either statically or dynamically, but ship them.
    - cycloptic 6y ago
      You can't get rid of the distro-oriented shared library system, it exists as long as you are building on top of an operating system. To that end, when you say "we depend on operating system minimum version X.Y.Z" that now becomes another dependency you have to ship at some point if you're running the production machines. Containers are popular because they actually solve that exact problem. If there was some other comparison you were making I'd be interested to hear it, but AFAIK containers can only help here so it's not clear what else you were comparing to.

orra 6y ago

> I don't really buy that dynamic linking all the things is such a boon to security. Agreed. It's really not a panacea. When you upgrade a library, you probably want to restart the running applications that depend upon this. Dynamic linking won't save you. The solution is having a graph of your dependencies! IIRC, NixOS gets this right. I don't think Debian's apt did?

1MachineElf 6y ago
I would really like to know how NixOS solves this problem for things like yarn, webpack, parcel, esbuild, npm, pip, conda, cargo, stack, go modules, ruby gems, etc.
- tadfisher 6y ago
  Usually there are tools to extract the necessary dependency information (url and hash) from a package manager's lockfile and produce a fixed-output derivation so a Nix expression can reproduce the build environment. I know of yarn2nix, cargo2nix, and node2nix, and I wrote and maintain gradle2nix for JVM projects.
- mands 6y ago
  It's doesn't really, and that's the problem. There are automated tools and scripts to help, but there are always edge cases so that you end up going back to using pip for example. Nix and language package managers just don't really play well together.
ece 6y ago
There are scripts and daemons that help you determine what needs restarting[1,2]. NixOS installs can go in separate directory prefixes when there are conflicts. For Gentoo and other Linux distributions, maintainers usually won't mark something stable without resolving conflicts, and this usually means sticking to older stable version of libraries until newer versions of libraries are fully supported by all installed packages. This can definitely be more work for maintainers, and as the blog posts says, it's a sisyphean task. [1] https://manpages.debian.org/buster/debian-goodies/checkrestart.1.en.html https://manpages.debian.org/buster/debian-goodies/checkresta... [2] https://github.com/liske/needrestart https://github.com/liske/needrestart

drran 6y ago

It's why we like to use LGPL for libraries: if you ship statically linked software, ship your source too, so we will be able to patch and recompile.

kasabali 6y ago

> The day I stop static linking is the day I can compile on one distro and ship to many without worrying about users reporting loader errors. That day is not today. That's completely missing point of the article. You're free to ship your program binary however you want as the upstream. It only becomes a problem if dynamic linking is a second class citizen in the language or build tool you use, you bundle dependencies and don't support un-bundling, or pin specific dependency versions.