5 ms·

Shared, dynamically linked libraries. This was understandable in a time when disk space was scarce, but it causes all kinds of dependency hell when you want to

Shared, dynamically linked libraries. This was understandable in a time when disk space was scarce, but it causes all kinds of dependency hell when you want to install programs that need different versions of the same library, or when you upgrade one of those libraries and break a bunch of programs. Plan 9 got this right with static linking a long time ago: https://9p.io/wiki/plan9/why_static/index.html https://9p.io/wiki/plan9/why_static/index.html. Sadly this has still not been adopted by major Unixy operating systems. Flatpak and Snap seem promising for isolation of dependencies in Linux without requiring full containerization.

dev_dull 8y ago
From here on out I want static, compiled binaries from every new programming language.
kbenson 8y ago
The downside, and I view it as a fairly big downside, is that security updates are complicated. When there's a problem in libjpeg or zlib, you can get 99% of the affected programs by updating the shared lib and restarting the affected applications. What's more, determining if you are affected is much easier. Compare that to trying to figure out which of your statically compiled applications happen to include the affected version of the library. As with so many choices we need to make in software development, it doesn't distill down to one way simply being better. It's a trade off, and in this case it's not just for disk space, but also for maintainability and easily discovering how exposed you are with respect to security.
- GordonS 8y ago
  I was going to say much the same thing - static and dynamic linking both have their pros and cons, you can't really say one is categorically better than the other.
  - zzzcpan 8y ago
    Dynamic linking doesn't have much pros beyond saving some disk space and some insignificant amount of build time.
    - GordonS 8y ago
      I agree those are small things these days, but IMO the biggest pro is security related - it makes it easier to update libraries when security issues arise. For example, if a new issue is found in OpenSSL, you can update the OS's shared library, but for statically linked apps you're dependant on vendors rebuilding and publishing those apps to link to the new version.
  - twblalock 8y ago
    Two solutions may each have pros and cons, but that does not mean that one of the solutions is not better overall.
- zzzcpan 8y ago
  This is a package management problem, linking has nothing to do with it. Security updates, just like library updates, could simply trigger recreation and rebuild of affected packages. Of course if certain package managers still do this kind of thing manually, it's on them. People shouldn't be manually bumping versions in package sources.
  - kbenson 8y ago
    That's only a solution for people using package managers, and people using package managers for everything. People manually build stuff. People also manually install binaries than have their own dependencies (even installed wireshark for Windows? I know for a long time, libpcap was installed separately as a sub-process during install). For example, probably 99% of the people that build their own toolchain for development and pushing to production. People also install separate package management systems (brew, cygwin) that might not have their update mechanisms automatically run, or are infrequently run. This is a compilation and deployment strategy downside. You can mitigate (but not eliminate) this downside through a strong package management system that tracks all this (BTW, is there a package management system in current use that tracks statically compiled dependencies?), but only if users and administrators adhere to strict practices.
    - zzzcpan 8y ago
      If you don't track updates in an automatic way, you are not going to even know whether you need to update something. So there is no point to talk about security updates without implying some reliance on a package management system. There are plenty of package management systems in current use where linking is an ideological or historical choice and has no relevance to security updates.
      - kbenson 8y ago
        > If you don't track updates in an automatic way, you are not going to even know whether you need to update something. With dynamic linking, you may get updates for programs that need them without knowing that it was needed. If you manually build Nginx with shared libraries, and later on some fix for PCRE comes down the line and you update libpcre, your Nginx install takes advantage of that the next time it restarts. This happens whether you manually updated PCRE or a package management system does it for you. This happens for things like Chrome and Firefox too, even though they ship with quite a bit statically compiled. There are still portions they expect the system to provide (if you've even tried to get chrome working on an older system, this becomes obvious). This is an advantage, it exists, and people make use of it all the time. You can try to argue down it's relevance, but without successfully doing so first, I think saying it's pointless is getting ahead of yourself. > There are plenty of package management systems in current use where linking is an ideological or historical choice and has no relevance to security updates. At the very least it would have relevance in that they would need to accurately track package dependencies and would not have the additional benefit of the package failing to install if they got it wrong. I've had plenty of times in the past where package management systems haven't even gotten that right, where everything installs but there's errors about missing libraries because of the way things are loaded on demand for some systems. There are benefits to not relying on some automated or manual tracking of this, where instead if you get it wrong it doesn't work. The fact that it requires a package management to mitigate this problem is itself a downside. To be clear, my argument is there are both upsides and downsides to dynamic linking, and that it's not purely all negative. By arguing against that you're positioning yourself as stating that it's not a trade off, so it's all bad (or all good, but that doesn't seem to be your point). Is that really the position you intend to stand on? It seems rather absolutist.
- AnIdiotOnTheNet 8y ago
  I agree it's a tradeoff, and if the system is designed competently it'll use a mix of the two, with the base system providing common shared libraries for system interfaces, crypto, networking, widgets, sound, etc. and guaranteeing that they won't break compatibility over a long time frame (unless there's a flaw so huge it warrants it). Then application developers are free to rely on these base components and static link their other dependencies.
- twblalock 8y ago
  You can patch all of your programs at once when a vulnerability is discovered, and that is nice. However, the vulnerability may have become widespread in the first place because it was distributed in a shared library that everyone had to use. This is a double-edged sword that makes vulnerabilities just as easy to distribute as patches.
Ace17 8y ago
Doesn't generalized static linking makes fixing security holes a maintainer's nightmare? What about keeping libraries dynamic, but allowing different versions to co-exist? (basically, it boils down to considering different versions of the same lib as independent libraries)
- twblalock 8y ago
  The problem for Unix is that C dependencies don't work that way, and it would take a huge amount of effort and culture change in the C ecosystem to make something like that happen. It would probably involve more effort than just statically linking the binaries. If this worked, developers would have no incentive to upgrade library versions that are working properly. The number of programs on your system that share the same library version would decrease over time until most of them use different versions, so you will miss out on the advantage of dynamic linking anyway -- updating a library would only fix security holes in a subset of your programs. You might as well just statically link everything and call it a day.
  - Ace17 8y ago
    > until most of them use different versions, so you will miss out on the advantage of dynamic linking anyway -- updating a library would only fix security holes in a subset of your programs. You might as well just statically link everything and call it a day. Please don't. Especially if I don't have your source code. A reduced amount of binaries to upgrade is only the tip of the iceberg. Even if I have 10 programs using 10 different versions of the same library, dynamic linking is a better choice than static linking. Because it allows security upgrades, independently of any collaboration with the vendors of these 10 programs.
skrebbel 8y ago
Windows got this mostly right these days, not by design but by convention. Because dependency hell sucks, vendors have started shipping all their shared libraries and dependencies inside the installer, to simply be extracted into the application directory. This way it's automatically resolved by the relevant LoadLibrary call. Nobody in their right mind would install stuff into Windows\System32 anymore. Similarly, increasingly many programs now ship an entire VM. Eg Eclipse has shipped with its own JRE for years now. Every Electron app comes bundled with a browser and a JS runtime, Dropbox includes Python and half its battery pack, and so on. Strangely, .NET is the odd one out, still requiring system wide installation. Well, at least it has a good backward compat story. Sure, this makes installers huge, but nobody really cares about that anymore. Similarly the potential space and cache efficiency savings of two programs sharing the same iconv.dll is just not worth the headache. Command line "package managers" like https://scoop.sh https://scoop.sh take full advantage of this trend by downloading these installers and not even running them. Instead, scoop just extracts their files into a directory somewhere deep in the homedir (= no admin rights needed) and assumes that said program is fully self-contained. In my opinion, this is lovely.
- AnIdiotOnTheNet 8y ago
  For the same reason, it is often relatively straight-forward to make completely portable versions of Windows applications, which is great. Last time Win 10 hosed itself and needed a reinstall, getting most of my workflow going again was a simple matter of creating a new toolbar and pointing it at my portable application folder. It's become my firm opinion that the concept of "installation" of applications is silly, causing far more trouble than it is worth.
- sebazzz 8y ago
  > Strangely, .NET is the odd one out, still requiring system wide installation. .NET Core allows both system-wide side-by-side and local deployments of the framework. Regarding side-by-side libraries, there is WinSxS
rlkf 8y ago
Shared objects are not only shared on disk, but their code segments are also shared in memory between executables that depends on the same libraries, lowering footprint and loading time. The reason dependency hell arises is because there is a lack of discipline in setting the soname correctly according to a semantic versioning (i.e. someone change the interface without changing the soversion) When soname is set correctly, there isn't really that much difference between /usr/lib and the GAC in Windows.

dev_dull 8y ago

From here on out I want static, compiled binaries from every new programming language.

kbenson 8y ago

The downside, and I view it as a fairly big downside, is that security updates are complicated. When there's a problem in libjpeg or zlib, you can get 99% of the affected programs by updating the shared lib and restarting the affected applications. What's more, determining if you are affected is much easier. Compare that to trying to figure out which of your statically compiled applications happen to include the affected version of the library. As with so many choices we need to make in software development, it doesn't distill down to one way simply being better. It's a trade off, and in this case it's not just for disk space, but also for maintainability and easily discovering how exposed you are with respect to security.

GordonS 8y ago
I was going to say much the same thing - static and dynamic linking both have their pros and cons, you can't really say one is categorically better than the other.
- zzzcpan 8y ago
  Dynamic linking doesn't have much pros beyond saving some disk space and some insignificant amount of build time.
  - GordonS 8y ago
    I agree those are small things these days, but IMO the biggest pro is security related - it makes it easier to update libraries when security issues arise. For example, if a new issue is found in OpenSSL, you can update the OS's shared library, but for statically linked apps you're dependant on vendors rebuilding and publishing those apps to link to the new version.
- twblalock 8y ago
  Two solutions may each have pros and cons, but that does not mean that one of the solutions is not better overall.
zzzcpan 8y ago
This is a package management problem, linking has nothing to do with it. Security updates, just like library updates, could simply trigger recreation and rebuild of affected packages. Of course if certain package managers still do this kind of thing manually, it's on them. People shouldn't be manually bumping versions in package sources.
- kbenson 8y ago
  That's only a solution for people using package managers, and people using package managers for everything. People manually build stuff. People also manually install binaries than have their own dependencies (even installed wireshark for Windows? I know for a long time, libpcap was installed separately as a sub-process during install). For example, probably 99% of the people that build their own toolchain for development and pushing to production. People also install separate package management systems (brew, cygwin) that might not have their update mechanisms automatically run, or are infrequently run. This is a compilation and deployment strategy downside. You can mitigate (but not eliminate) this downside through a strong package management system that tracks all this (BTW, is there a package management system in current use that tracks statically compiled dependencies?), but only if users and administrators adhere to strict practices.
  - zzzcpan 8y ago
    If you don't track updates in an automatic way, you are not going to even know whether you need to update something. So there is no point to talk about security updates without implying some reliance on a package management system. There are plenty of package management systems in current use where linking is an ideological or historical choice and has no relevance to security updates.
    - kbenson 8y ago
      > If you don't track updates in an automatic way, you are not going to even know whether you need to update something. With dynamic linking, you may get updates for programs that need them without knowing that it was needed. If you manually build Nginx with shared libraries, and later on some fix for PCRE comes down the line and you update libpcre, your Nginx install takes advantage of that the next time it restarts. This happens whether you manually updated PCRE or a package management system does it for you. This happens for things like Chrome and Firefox too, even though they ship with quite a bit statically compiled. There are still portions they expect the system to provide (if you've even tried to get chrome working on an older system, this becomes obvious). This is an advantage, it exists, and people make use of it all the time. You can try to argue down it's relevance, but without successfully doing so first, I think saying it's pointless is getting ahead of yourself. > There are plenty of package management systems in current use where linking is an ideological or historical choice and has no relevance to security updates. At the very least it would have relevance in that they would need to accurately track package dependencies and would not have the additional benefit of the package failing to install if they got it wrong. I've had plenty of times in the past where package management systems haven't even gotten that right, where everything installs but there's errors about missing libraries because of the way things are loaded on demand for some systems. There are benefits to not relying on some automated or manual tracking of this, where instead if you get it wrong it doesn't work. The fact that it requires a package management to mitigate this problem is itself a downside. To be clear, my argument is there are both upsides and downsides to dynamic linking, and that it's not purely all negative. By arguing against that you're positioning yourself as stating that it's not a trade off, so it's all bad (or all good, but that doesn't seem to be your point). Is that really the position you intend to stand on? It seems rather absolutist.
AnIdiotOnTheNet 8y ago
I agree it's a tradeoff, and if the system is designed competently it'll use a mix of the two, with the base system providing common shared libraries for system interfaces, crypto, networking, widgets, sound, etc. and guaranteeing that they won't break compatibility over a long time frame (unless there's a flaw so huge it warrants it). Then application developers are free to rely on these base components and static link their other dependencies.
twblalock 8y ago
You can patch all of your programs at once when a vulnerability is discovered, and that is nice. However, the vulnerability may have become widespread in the first place because it was distributed in a shared library that everyone had to use. This is a double-edged sword that makes vulnerabilities just as easy to distribute as patches.

Ace17 8y ago

Doesn't generalized static linking makes fixing security holes a maintainer's nightmare? What about keeping libraries dynamic, but allowing different versions to co-exist? (basically, it boils down to considering different versions of the same lib as independent libraries)

twblalock 8y ago
The problem for Unix is that C dependencies don't work that way, and it would take a huge amount of effort and culture change in the C ecosystem to make something like that happen. It would probably involve more effort than just statically linking the binaries. If this worked, developers would have no incentive to upgrade library versions that are working properly. The number of programs on your system that share the same library version would decrease over time until most of them use different versions, so you will miss out on the advantage of dynamic linking anyway -- updating a library would only fix security holes in a subset of your programs. You might as well just statically link everything and call it a day.
- Ace17 8y ago
  > until most of them use different versions, so you will miss out on the advantage of dynamic linking anyway -- updating a library would only fix security holes in a subset of your programs. You might as well just statically link everything and call it a day. Please don't. Especially if I don't have your source code. A reduced amount of binaries to upgrade is only the tip of the iceberg. Even if I have 10 programs using 10 different versions of the same library, dynamic linking is a better choice than static linking. Because it allows security upgrades, independently of any collaboration with the vendors of these 10 programs.

skrebbel 8y ago

Windows got this mostly right these days, not by design but by convention. Because dependency hell sucks, vendors have started shipping all their shared libraries and dependencies inside the installer, to simply be extracted into the application directory. This way it's automatically resolved by the relevant LoadLibrary call. Nobody in their right mind would install stuff into Windows\System32 anymore. Similarly, increasingly many programs now ship an entire VM. Eg Eclipse has shipped with its own JRE for years now. Every Electron app comes bundled with a browser and a JS runtime, Dropbox includes Python and half its battery pack, and so on. Strangely, .NET is the odd one out, still requiring system wide installation. Well, at least it has a good backward compat story. Sure, this makes installers huge, but nobody really cares about that anymore. Similarly the potential space and cache efficiency savings of two programs sharing the same iconv.dll is just not worth the headache. Command line "package managers" like https://scoop.sh https://scoop.sh take full advantage of this trend by downloading these installers and not even running them. Instead, scoop just extracts their files into a directory somewhere deep in the homedir (= no admin rights needed) and assumes that said program is fully self-contained. In my opinion, this is lovely.

AnIdiotOnTheNet 8y ago
For the same reason, it is often relatively straight-forward to make completely portable versions of Windows applications, which is great. Last time Win 10 hosed itself and needed a reinstall, getting most of my workflow going again was a simple matter of creating a new toolbar and pointing it at my portable application folder. It's become my firm opinion that the concept of "installation" of applications is silly, causing far more trouble than it is worth.
sebazzz 8y ago
> Strangely, .NET is the odd one out, still requiring system wide installation. .NET Core allows both system-wide side-by-side and local deployments of the framework. Regarding side-by-side libraries, there is WinSxS

rlkf 8y ago

Shared objects are not only shared on disk, but their code segments are also shared in memory between executables that depends on the same libraries, lowering footprint and loading time. The reason dependency hell arises is because there is a lack of discipline in setting the soname correctly according to a semantic versioning (i.e. someone change the interface without changing the soversion) When soname is set correctly, there isn't really that much difference between /usr/lib and the GAC in Windows.