4 ms·

Then once one of the static libraries has a major bug, a major catastrophe happens, and hilarity ensues. Use static linking with parsimony and intent. It is a

Then once one of the static libraries has a major bug, a major catastrophe happens, and hilarity ensues. Use static linking with parsimony and intent. It is a double edge sword. Don't make that your default. Learn from past mistakes. That said, the concept is interesting. I would just use dynamic libraries inside. Space and speed issues are premature optimizations.

AnIdiotOnTheNet 8y ago
> Learn from past mistakes You mean like DLL hell, and the Linux distro fragmentation mess? History has shown that the trade-offs of dynamic linking are often worse. Case in point: the reason containers are so popular in the first place.
- devereaux 8y ago
  The only reason dynamic libraries are worse (beside speed) is the dependency hell. Put the dynamic libraries INSIDE the container and you can upgrade them when/if you want, while nothing will be broken during a systemic upgrade where you don't chose to update them.
  - tedivm 8y ago
    In an environment like this the only difference between static and dynamically linked libraries is that the dynamically linked ones can be upgraded without needing to recompile. Other benefits- such as not needing to ship as much data- disappear, and the negatives (such as speed) remain. In this context then there doesn't seem to be much of a benefit towards dynamic linking. Most people are going to rebuild the container completely rather than enter it to upgrade a library, or if it's a distributed program they're just going to grab the latest version- being able to upgrade components individual is not a use case that I see being all that important.
    - devereaux 8y ago
      Then I guess we have different opinions, because I would only update the parts of the container that need updating, and I wouldn't touch what isn't broken - just like how I do a apt-get upgrade more often than a apt-get dist-upgrade: because even if it should all work fine in theory, in practice it doesn't. I am mostly interested by gains in security, gains in maintainability, and gains in simplicity. Dynamic libraries reduce the number of parts that need to be managed, reducing overall complexity. Also, if I remember correctly, UWP apps in windows are managed a bit like that, with the equivalent of a xdelta (binary diff) Being able to update just one or a few libraries would be handy in other cases - not just critical exploits. On top of my head, the first use case I see is old apps, those which have deployed say years ago, where the compilation scripts/recipe/call it whatever you want have been lost and the employees left, and you need to change one piece of the whole for whatever reason. There's totally a place for static binaries, but it is a very specific one, where you have special constraints (ex: when I update a remote server, I want to have a static busybox)
      - mitchty 8y ago
        > I am mostly interested by gains in security, gains in maintainability, and gains in simplicity. Dynamic libraries reduce the number of parts that need to be managed, reducing overall complexity. I'd argue they do no such thing, the complexity is still there, only now your users can upgrade libraries and potentially break your app in ways you never expected. Lets take openssl as an example, i've seen upgrades of that break applications easily. Your app should take any dependency change as a new version or patch upgrade. And if you're releasing a new version of the application, what gain did a library provide? About none. I've seen it from both static and dynamic sides, I vastly prefer static binaries cons to the cons I've seen from dynamic libraries.
        webmaven 8y ago
        How would this impact the sort of incremental app updates that Chrome (for example) does using Omaha? eg. would dynamic linking result in smaller or simpler deltas compared to static linking?
- wahern 8y ago
  DLL hell is mostly an issue of incompatible CRTs (aka libc in Unix parlance) on Windows. Historically (until circa 2015) Windows programs (particularly those compiled with Visual Studio) often loaded a multiplicity of incompatible CRTs. It was often the case that malloc'd memory from one DLL could not be free'd from another DLL because each linked to a different CRT. Compounding matters, applications would often globally install popular DLLs which, even if compiled from the exact same source code, would break apps if linked against a different CRT version at build time. Which would invariably be the case as Visual Studio by default links in its own version-specific CRT rather than the system CRT, the Linux equivalent of GCC shipping and statically linking in a new version of musl libc, making mixing-and-matching of shared libraries compiled in different environments a risky endeavor indeed. On Unix there's almost always a single, system-wide CRT. Moreover, commercial environments like Solaris as well as Linux/glibc have maintained strong backwards compatibility (mostly a matter of ABI stability) so that compiled programs continue to work correctly with future libc versions. Believe it or not, third-party dependencies aside, compiling a C program on Red Hat and successfully (and correctly) running on Debian is the norm, not the exception, assuming you never on execute on a system with an older version of glibc than during compilation.[1] (Even though musl libc doesn't do symbol versioning, I assume the story is similar as it rigorously sticks to exposing only POSIX interfaces and opaque structures as much as it can.) Therefore, DLL hell is a much less pronounced issue in Linux land. A typical example involves the loading of incompatible versions of third-party libraries. But most widely used third-party libraries like libz or libxml2 have maintained strong backward compatibility. The only culprit I've repeatedly encountered in this regard is OpenSSL, which until OpenSSL 1.1 never committed to a stable ABI, largely because it relied so heavily on exposed, preprocessor generated structures as opposed to idiomatic opaque structure pointers. But IME this was more an issue on macOS as the norm in Linux is to simply use the system OpenSSL version. (Who here bothers to download, build, and install OpenSSL in their containers rather than using the system version of their Debian- or RedHat-derived container base image? It's common enough at large companies but hardly so common that it was a substantial factor in the adoption of containers.) The ELF toolchain on modern Linux systems is actually sufficiently advanced that you could theoretically support the loading of multiple versions of a library like OpenSSL (as long as you didn't pass objects between them), such as the SONAME linker tag and the far more powerful but little known ELF symbol versioning. See, e.g., https://www.berrange.com/posts/2011/01/13/versioning-in-the-libvirt-library/ https://www.berrange.com/posts/2011/01/13/versioning-in-the-... Alas, SONAME is used inconsistently and ELF symbol versioning almost not at all--only symbol versioning users of note AFAIK are glibc and libvirt. It's at this point that executing in containers becomes the path of least resistance. It's true that installing scripting language packages or building complex C++ programs which have evolved nightmarish dependency trees as impressive as Node.js/npm is a huge motivator for containerization, but that's not the kind of thing that is meant by DLL hell. And historically this issue was sufficiently addressed by packaging systems. It's no coincidence Ubuntu is the most popular container base image--the Ubuntu package repository is multiple times the size of Red Hat's repository, and much more up-to-date. (It's larger even than CentOS' or Fedora's repositories, even when including third-party repositories like EPEL.) In other words, even in the era of containers people still largely rely on the older packaging infrastructure. [1] Maintaining this "never older than" invariant is an important caveat, but IME I've run into more issues with kernel versions (e.g. unsupported syscalls) than with libc versions. Containers don't solve the kernel version issue.
jasonhansel 8y ago
A question: do tools like this count as "static linking" for the purposes of LGPL licensing? If so, there may be legal issues at stake.
elsonrodriguez 8y ago
An application consists of its code and dependencies. If you change a dynamic dependency, then you are changing the application, and this merits a new release of the application and a version bump. Many shops arrived at this pattern (with bash scripts or config management) after trying a simple `apt-get upgrade` for some security patches, only to realize the application behavior changed or broke entirely. Static linking simply codifies this practice in a tighter construct, as do containers in general.

AnIdiotOnTheNet 8y ago

> Learn from past mistakes You mean like DLL hell, and the Linux distro fragmentation mess? History has shown that the trade-offs of dynamic linking are often worse. Case in point: the reason containers are so popular in the first place.

devereaux 8y ago
The only reason dynamic libraries are worse (beside speed) is the dependency hell. Put the dynamic libraries INSIDE the container and you can upgrade them when/if you want, while nothing will be broken during a systemic upgrade where you don't chose to update them.
- tedivm 8y ago
  In an environment like this the only difference between static and dynamically linked libraries is that the dynamically linked ones can be upgraded without needing to recompile. Other benefits- such as not needing to ship as much data- disappear, and the negatives (such as speed) remain. In this context then there doesn't seem to be much of a benefit towards dynamic linking. Most people are going to rebuild the container completely rather than enter it to upgrade a library, or if it's a distributed program they're just going to grab the latest version- being able to upgrade components individual is not a use case that I see being all that important.
  - devereaux 8y ago
    Then I guess we have different opinions, because I would only update the parts of the container that need updating, and I wouldn't touch what isn't broken - just like how I do a apt-get upgrade more often than a apt-get dist-upgrade: because even if it should all work fine in theory, in practice it doesn't. I am mostly interested by gains in security, gains in maintainability, and gains in simplicity. Dynamic libraries reduce the number of parts that need to be managed, reducing overall complexity. Also, if I remember correctly, UWP apps in windows are managed a bit like that, with the equivalent of a xdelta (binary diff) Being able to update just one or a few libraries would be handy in other cases - not just critical exploits. On top of my head, the first use case I see is old apps, those which have deployed say years ago, where the compilation scripts/recipe/call it whatever you want have been lost and the employees left, and you need to change one piece of the whole for whatever reason. There's totally a place for static binaries, but it is a very specific one, where you have special constraints (ex: when I update a remote server, I want to have a static busybox)
    - mitchty 8y ago
      > I am mostly interested by gains in security, gains in maintainability, and gains in simplicity. Dynamic libraries reduce the number of parts that need to be managed, reducing overall complexity. I'd argue they do no such thing, the complexity is still there, only now your users can upgrade libraries and potentially break your app in ways you never expected. Lets take openssl as an example, i've seen upgrades of that break applications easily. Your app should take any dependency change as a new version or patch upgrade. And if you're releasing a new version of the application, what gain did a library provide? About none. I've seen it from both static and dynamic sides, I vastly prefer static binaries cons to the cons I've seen from dynamic libraries.
      - webmaven 8y ago
        How would this impact the sort of incremental app updates that Chrome (for example) does using Omaha? eg. would dynamic linking result in smaller or simpler deltas compared to static linking?
wahern 8y ago
DLL hell is mostly an issue of incompatible CRTs (aka libc in Unix parlance) on Windows. Historically (until circa 2015) Windows programs (particularly those compiled with Visual Studio) often loaded a multiplicity of incompatible CRTs. It was often the case that malloc'd memory from one DLL could not be free'd from another DLL because each linked to a different CRT. Compounding matters, applications would often globally install popular DLLs which, even if compiled from the exact same source code, would break apps if linked against a different CRT version at build time. Which would invariably be the case as Visual Studio by default links in its own version-specific CRT rather than the system CRT, the Linux equivalent of GCC shipping and statically linking in a new version of musl libc, making mixing-and-matching of shared libraries compiled in different environments a risky endeavor indeed. On Unix there's almost always a single, system-wide CRT. Moreover, commercial environments like Solaris as well as Linux/glibc have maintained strong backwards compatibility (mostly a matter of ABI stability) so that compiled programs continue to work correctly with future libc versions. Believe it or not, third-party dependencies aside, compiling a C program on Red Hat and successfully (and correctly) running on Debian is the norm, not the exception, assuming you never on execute on a system with an older version of glibc than during compilation.[1] (Even though musl libc doesn't do symbol versioning, I assume the story is similar as it rigorously sticks to exposing only POSIX interfaces and opaque structures as much as it can.) Therefore, DLL hell is a much less pronounced issue in Linux land. A typical example involves the loading of incompatible versions of third-party libraries. But most widely used third-party libraries like libz or libxml2 have maintained strong backward compatibility. The only culprit I've repeatedly encountered in this regard is OpenSSL, which until OpenSSL 1.1 never committed to a stable ABI, largely because it relied so heavily on exposed, preprocessor generated structures as opposed to idiomatic opaque structure pointers. But IME this was more an issue on macOS as the norm in Linux is to simply use the system OpenSSL version. (Who here bothers to download, build, and install OpenSSL in their containers rather than using the system version of their Debian- or RedHat-derived container base image? It's common enough at large companies but hardly so common that it was a substantial factor in the adoption of containers.) The ELF toolchain on modern Linux systems is actually sufficiently advanced that you could theoretically support the loading of multiple versions of a library like OpenSSL (as long as you didn't pass objects between them), such as the SONAME linker tag and the far more powerful but little known ELF symbol versioning. See, e.g., https://www.berrange.com/posts/2011/01/13/versioning-in-the-libvirt-library/ https://www.berrange.com/posts/2011/01/13/versioning-in-the-... Alas, SONAME is used inconsistently and ELF symbol versioning almost not at all--only symbol versioning users of note AFAIK are glibc and libvirt. It's at this point that executing in containers becomes the path of least resistance. It's true that installing scripting language packages or building complex C++ programs which have evolved nightmarish dependency trees as impressive as Node.js/npm is a huge motivator for containerization, but that's not the kind of thing that is meant by DLL hell. And historically this issue was sufficiently addressed by packaging systems. It's no coincidence Ubuntu is the most popular container base image--the Ubuntu package repository is multiple times the size of Red Hat's repository, and much more up-to-date. (It's larger even than CentOS' or Fedora's repositories, even when including third-party repositories like EPEL.) In other words, even in the era of containers people still largely rely on the older packaging infrastructure. [1] Maintaining this "never older than" invariant is an important caveat, but IME I've run into more issues with kernel versions (e.g. unsupported syscalls) than with libc versions. Containers don't solve the kernel version issue.

jasonhansel 8y ago

A question: do tools like this count as "static linking" for the purposes of LGPL licensing? If so, there may be legal issues at stake.

elsonrodriguez 8y ago

An application consists of its code and dependencies. If you change a dynamic dependency, then you are changing the application, and this merits a new release of the application and a version bump. Many shops arrived at this pattern (with bash scripts or config management) after trying a simple `apt-get upgrade` for some security patches, only to realize the application behavior changed or broke entirely. Static linking simply codifies this practice in a tighter construct, as do containers in general.