11 ms·

> I don't understand why this isn't done Because when there's a security update to (say) OpenSSL, it's better for the maintainers of just that library to push

> I don't understand why this isn't done Because when there's a security update to (say) OpenSSL, it's better for the maintainers of just that library to push an update, as opposed to forcing every single dependent to rebuild & push a new release.

otterley 7y ago
I would have agreed with this statement about five years ago. (Even though you would have had to restart all the dependent binaries after updating the shared libs.) Today, with containers becoming increasingly the de facto means of deploying software, it's not so important anymore. The upgrade process is now: (1) build an updated image; (2) upgrade your deployment manifest; (3) upload your manifest to your control plane. The control plane manages the rest. The other reason to use shared libs is for memory conservation, but except on the smallest devices, I'm not sure the average person cares about conserving a few MB of memory on 4GB+ machines anymore.
- MereInterest 7y ago
  Some of us use linux as a desktop environment, and like having the security patches be applied as soon as the relevant package has updated.
  - AnIdiotOnTheNet 7y ago
    As a user of the Linux desktop, I really love it when library updates break compatibility with the software I use too. Or can't be installed because of dependency conflicts. Containers are popular because shared libraries cause more trouble than they are worth.
- rumanator 7y ago
  > Today, with containers becoming increasingly the de facto means This assertion makes no sense at all and entirely misses the whole point of shared/dynamic libraries. It's like a buzzword is a magic spell that makes some people forget the entire history and design requirements up to that very moment.
  - ithkuil 7y ago
    Sometimes buzzwords make sense, in the right context. This was the right context. Assuming you use containers, you're likely to not log into them and keep them up to date and secure by running apt-get upgrade. The most common workflow is indeed: build your software in your CI system, in the last step create a container with your software and its dependencies. Then update your deployment with a new version of the whole image. A container image is for all intents and purposes the new "static binary". Yes, technically you can look inside it, yes technically you can (and you do) use dynamic linking inside the container itself. But as long as the workflow is the one depicted above, the environment no longer has the requirements that led to the design of dynamic linking. It's possible to have alternative workflows for building containers: you could fiddle with layers and swap an updated base OS under a layer containing your compiled application. I don't how common is that, but I'm sure somebody will want/have to do it. It all boils down to whether developers still maintain control over the full deployment pipeline as containers penetrate the enterprises (i.e. whether re retain the "shift to the left", another buzzword for you). Containers are not just a technical solution, they are the embodiment of the desire of developers to free themselves from the tyranny of filing tickets and waiting days to deploy their apps. But that leaves the security departments in enterprises understandably worried as most of those developers are focused on shipping features and often neglecting (or ignoring) security concerns around things that live one layer below the application they write.
  - AnIdiotOnTheNet 7y ago
    Shared libraries have largely proven that they aren't a good idea, which is why containers are so popular. Between conflicts and broken compatibility between updates, shared libraries have become more trouble than they are worth. I think they still make sense for base-system libraries, but unfortunately there is no agreed upon definition of 'base-system' in the wild west of Linux.
- GordonS 7y ago
  > Today, with containers becoming increasingly the de facto means of deploying software I think that's something of an exaggeration. Yes, containers are popular for server software, but even then it's a huge stretch to claim they are becoming de facto.
  - mikepurvis 7y ago
    App bundles on MacOS and iOS are basically big containers, though there is some limited external linking through Apple's frameworks scheme. And obviously video game distribution has looked like this since basically forever as well.
    - saagarjha 7y ago
      > App bundles on MacOS and iOS are basically big containers, though there is some limited external linking through Apple's frameworks scheme. There's a file hundreds of megabytes large containing all the dynamically-linked system libraries on iOS to make your apps work.
    - jfkebwjsbx 7y ago
      Video games do not run on/as containers. Quite the opposite, in fact.
      - pjmlp 7y ago
        Kind of. They do on XBox, Swift, iOS, Android sandboxes.
      - yellowapple 7y ago
        In addition to pjmlp's list, Steam is pushing toward this for Linux games (and one could argue that Steam has been this for as long as it's been available on Linux, given that it maintains its own runtime specifically so that games don't have to take distro-specific quirks into account). Beyond containers / isolated runtime environments, the parent comment is correct about games (specifically of the console variety) being historically nearly-always statically-linked never-updated monoliths (which is how I interpreted that comment). "Patching" a game after-the-fact was effectively unheard of until around the time of the PS3 / Xbox 360 / Wii (when Internet connectivity became more of a norm for game consoles), with the sole exception of perhaps releasing a new edition of it entirely (which would have little to no impact on the copies already sold).
- sitkack 7y ago
  Containers most likely wouldn't have existed if we had a proper ecosystem around static linking and resolution of dependencies. Containers solve the problem of the lack of executable state control, mostly caused by dynamic linking.
  - deleted 7y ago
    [deleted]
  - sjwright 7y ago
    More broadly, containers solve the problem of reproducibility. No longer does software get to vomit crap all over your file system in ways that make reproducing a functioning environment frustrating and troublesome. They have the effect of side-stepping the dependencies problem, but that isn’t the core benefit.
    - senderista 7y ago
      But the images themselves are not easily reproducible with standard build tooling.
      - sjwright 7y ago
        True—but that's far less of a problem, because it rarely occurs unexpectedly and under a time crunch. Diffing two docker images to determine the differences between builds would be far less onerous than attempting to diff a new deployment against a long-lived production server.
  - SAI_Peregrinus 7y ago
    Dynamic linking isn't the issue. Shared libraries are the issue. You could bundle a bunch of .so files with your executable & stick it in a directory, and have the executable link using those. That's basically how Windows does it, and it's why there's no "dependency hell" there despite having .dlls (dynamically linked libraries) all over the place. Shared libraries are shared (obviously) and get updated, so they're mutable. Linux systems depend on a substantial amount of shared mutable state being kept consistent. This causes lots of headaches, just as it does in concurrent programming.
- tannhaeuser 7y ago
  And the reason we're using containers in the first place is precisely because we've messed up and traded shared libs for having a proven-interworking set of them, something that can trivially be achieved using static linking.
  - rumanator 7y ago
    Actually the main selling point of containers has nothing to do with "proven interworking", but the ability to deploy and run entire applications in a fully controlled and fully configurable environment. Static libraries do nothing of the sort. In fact, they make it practically impossible to pull it off. There's far more to deploying software than mindlessly binding libraries.
    - hawski 7y ago
      > the ability to deploy and run entire applications in a fully controlled and fully configurable environment But isn't the reason to have this fully controlled and fully configurable environment to have a proof of interworking? Because when environment is in any form different you can, and people already do, say that it's not supported.
      - rumanator 7y ago
        > But isn't the reason to have this fully controlled and fully configurable environment to have a proof of interworking? No, because there's far more to deploying apps than copying libraries somewhere.
    - yellowapple 7y ago
      > Actually the main selling point of containers has nothing to do with "proven interworking", but the ability to deploy and run entire applications in a fully controlled and fully configurable environment. Which is exactly the same selling point as for static linking.
    - Wowfunhappy 7y ago
      On Windows, I don't need to use Docker in order to run a program in a reproducible way. I just download a program, and in 90% of cases it "just works" whether I'm running Windows 10, Windows 8, or the decade-old Windows 7. Furthermore, installing that program will (again, in 90% of cases at least) not affect my overall system configuration in any way. I can be confident that all of my other programs will continue to work as they have. Why? Because any libraries which aren't included in the least-common-denominator version of Windows are included with the download, and are used only for that download. The libraries may shipped as DLLs next to the executable, which are technically dynamic, but it's the same concept—those DLL's are program-specific. This ability is what I really miss when I try to switch to desktop Linux. I don't want to set up Docker containers for random desktop apps, and I don't want a given app to affect the state of my overall system. I want to download and run stuff. --- I realize there's a couple of big caveats here. Since Windows programs aren't sandboxed, misbehaving programs absolutely can hose a system—but at least that's not the intended way things are supposed to work. I'm also skipping over runtimes such as Visual C++, but as I see it, those can almost be considered part of the OS at this point. And I can a ridiculous number of versions of MSVC installed simultaneously without issue.
StreamBright 7y ago
Based on my experience this is very rarely the case unless you have an extremely disciplined SecOps team.
- rumanator 7y ago
  > Based on my experience this is very rarely the case You must have close to zero experience them because that's the norm on any software that depends on, say, third-party libraries that ship with a OS/distro. Recommended reading: Debian's openssl package. https://tracker.debian.org/pkg/openssl https://tracker.debian.org/pkg/openssl
  - StreamBright 7y ago
    You are talking about a FOSS project, I am talking about a company that has a service that uses OpenSSL in production.
    - boomlinde 7y ago
      These are not diametrically opposed. Your company can have a service that uses OpenSSL in production that runs on Debian to automatically take advantage of Debian patches and updates if it's linked dynamically to the system provided OpenSSL. You can either employ an extremely disciplined SecOps team to carefully track updates and CVEs (you'd need this whether you're linking statically or dynamically) or you can use e.g. Debian to take advantage of their work to that end.
      - StreamBright 7y ago
        Every single company that I used to work for had an internal version of Linux that they approved for production. Internal release cycles are disconnected from external release cycles. On the top of that, some of these companies were not using system-wide packages at all, you had to reference a version of packages (like OpenSSL) during your build process. We had to do emergency patching for CVEs and bump the versions in every service. This way you can have 100% confidence what a particular service is running with a particular version of OpenSSL. This process do not depend on Debian's (or other FOSS vendor's) release cycles and the dependencies are explicit, therefore the vulnerability assessment is simpler (as opposed to go to every server and check which version is installed). Don't you think?
yellowapple 7y ago
My main issue with this rationale is that, in the vast majority of production environments (at least the ones I've seen in the wild, and indeed the ones I've built), updating dependencies for dynamically-linked dependents is part of the "release" process just like doing so for a statically-linked dependent, so this ends up being a distinction without a difference; in either circumstance, there's a "rebuild" as the development and/or operations teams test and deploy the new application and runtime environment. This is only slightly more relevant for pure system administration scenarios where the machine is exclusively running software prebuilt by some third-party vendor (e.g. your average Linux distro package repo). Even then, unless you're doing blind automatic upgrades (which some shops do, but it carries its own set of risks), you're still hopefully at least testing new versions and employing some sort of well-defined deployment workflow. Also, if that "security update" introduces a breaking change (which Shouldn't Happen™, but something something Murphy's Law something something), then - again - retesting and rebuilding a runtime environment for a dynamically-linked dependent v. rebuilding a statically-linked dependent is a distinction without a difference.

otterley 7y ago

I would have agreed with this statement about five years ago. (Even though you would have had to restart all the dependent binaries after updating the shared libs.) Today, with containers becoming increasingly the de facto means of deploying software, it's not so important anymore. The upgrade process is now: (1) build an updated image; (2) upgrade your deployment manifest; (3) upload your manifest to your control plane. The control plane manages the rest. The other reason to use shared libs is for memory conservation, but except on the smallest devices, I'm not sure the average person cares about conserving a few MB of memory on 4GB+ machines anymore.

MereInterest 7y ago
Some of us use linux as a desktop environment, and like having the security patches be applied as soon as the relevant package has updated.
- AnIdiotOnTheNet 7y ago
  As a user of the Linux desktop, I really love it when library updates break compatibility with the software I use too. Or can't be installed because of dependency conflicts. Containers are popular because shared libraries cause more trouble than they are worth.
rumanator 7y ago
> Today, with containers becoming increasingly the de facto means This assertion makes no sense at all and entirely misses the whole point of shared/dynamic libraries. It's like a buzzword is a magic spell that makes some people forget the entire history and design requirements up to that very moment.
- ithkuil 7y ago
  Sometimes buzzwords make sense, in the right context. This was the right context. Assuming you use containers, you're likely to not log into them and keep them up to date and secure by running apt-get upgrade. The most common workflow is indeed: build your software in your CI system, in the last step create a container with your software and its dependencies. Then update your deployment with a new version of the whole image. A container image is for all intents and purposes the new "static binary". Yes, technically you can look inside it, yes technically you can (and you do) use dynamic linking inside the container itself. But as long as the workflow is the one depicted above, the environment no longer has the requirements that led to the design of dynamic linking. It's possible to have alternative workflows for building containers: you could fiddle with layers and swap an updated base OS under a layer containing your compiled application. I don't how common is that, but I'm sure somebody will want/have to do it. It all boils down to whether developers still maintain control over the full deployment pipeline as containers penetrate the enterprises (i.e. whether re retain the "shift to the left", another buzzword for you). Containers are not just a technical solution, they are the embodiment of the desire of developers to free themselves from the tyranny of filing tickets and waiting days to deploy their apps. But that leaves the security departments in enterprises understandably worried as most of those developers are focused on shipping features and often neglecting (or ignoring) security concerns around things that live one layer below the application they write.
- AnIdiotOnTheNet 7y ago
  Shared libraries have largely proven that they aren't a good idea, which is why containers are so popular. Between conflicts and broken compatibility between updates, shared libraries have become more trouble than they are worth. I think they still make sense for base-system libraries, but unfortunately there is no agreed upon definition of 'base-system' in the wild west of Linux.
GordonS 7y ago
> Today, with containers becoming increasingly the de facto means of deploying software I think that's something of an exaggeration. Yes, containers are popular for server software, but even then it's a huge stretch to claim they are becoming de facto.
- mikepurvis 7y ago
  App bundles on MacOS and iOS are basically big containers, though there is some limited external linking through Apple's frameworks scheme. And obviously video game distribution has looked like this since basically forever as well.
  - saagarjha 7y ago
    > App bundles on MacOS and iOS are basically big containers, though there is some limited external linking through Apple's frameworks scheme. There's a file hundreds of megabytes large containing all the dynamically-linked system libraries on iOS to make your apps work.
  - jfkebwjsbx 7y ago
    Video games do not run on/as containers. Quite the opposite, in fact.
    - pjmlp 7y ago
      Kind of. They do on XBox, Swift, iOS, Android sandboxes.
    - yellowapple 7y ago
      In addition to pjmlp's list, Steam is pushing toward this for Linux games (and one could argue that Steam has been this for as long as it's been available on Linux, given that it maintains its own runtime specifically so that games don't have to take distro-specific quirks into account). Beyond containers / isolated runtime environments, the parent comment is correct about games (specifically of the console variety) being historically nearly-always statically-linked never-updated monoliths (which is how I interpreted that comment). "Patching" a game after-the-fact was effectively unheard of until around the time of the PS3 / Xbox 360 / Wii (when Internet connectivity became more of a norm for game consoles), with the sole exception of perhaps releasing a new edition of it entirely (which would have little to no impact on the copies already sold).
sitkack 7y ago
Containers most likely wouldn't have existed if we had a proper ecosystem around static linking and resolution of dependencies. Containers solve the problem of the lack of executable state control, mostly caused by dynamic linking.
- deleted 7y ago
  [deleted]
- sjwright 7y ago
  More broadly, containers solve the problem of reproducibility. No longer does software get to vomit crap all over your file system in ways that make reproducing a functioning environment frustrating and troublesome. They have the effect of side-stepping the dependencies problem, but that isn’t the core benefit.
  - senderista 7y ago
    But the images themselves are not easily reproducible with standard build tooling.
    - sjwright 7y ago
      True—but that's far less of a problem, because it rarely occurs unexpectedly and under a time crunch. Diffing two docker images to determine the differences between builds would be far less onerous than attempting to diff a new deployment against a long-lived production server.
- SAI_Peregrinus 7y ago
  Dynamic linking isn't the issue. Shared libraries are the issue. You could bundle a bunch of .so files with your executable & stick it in a directory, and have the executable link using those. That's basically how Windows does it, and it's why there's no "dependency hell" there despite having .dlls (dynamically linked libraries) all over the place. Shared libraries are shared (obviously) and get updated, so they're mutable. Linux systems depend on a substantial amount of shared mutable state being kept consistent. This causes lots of headaches, just as it does in concurrent programming.
tannhaeuser 7y ago
And the reason we're using containers in the first place is precisely because we've messed up and traded shared libs for having a proven-interworking set of them, something that can trivially be achieved using static linking.
- rumanator 7y ago
  Actually the main selling point of containers has nothing to do with "proven interworking", but the ability to deploy and run entire applications in a fully controlled and fully configurable environment. Static libraries do nothing of the sort. In fact, they make it practically impossible to pull it off. There's far more to deploying software than mindlessly binding libraries.
  - hawski 7y ago
    > the ability to deploy and run entire applications in a fully controlled and fully configurable environment But isn't the reason to have this fully controlled and fully configurable environment to have a proof of interworking? Because when environment is in any form different you can, and people already do, say that it's not supported.
    - rumanator 7y ago
      > But isn't the reason to have this fully controlled and fully configurable environment to have a proof of interworking? No, because there's far more to deploying apps than copying libraries somewhere.
  - yellowapple 7y ago
    > Actually the main selling point of containers has nothing to do with "proven interworking", but the ability to deploy and run entire applications in a fully controlled and fully configurable environment. Which is exactly the same selling point as for static linking.
  - Wowfunhappy 7y ago
    On Windows, I don't need to use Docker in order to run a program in a reproducible way. I just download a program, and in 90% of cases it "just works" whether I'm running Windows 10, Windows 8, or the decade-old Windows 7. Furthermore, installing that program will (again, in 90% of cases at least) not affect my overall system configuration in any way. I can be confident that all of my other programs will continue to work as they have. Why? Because any libraries which aren't included in the least-common-denominator version of Windows are included with the download, and are used only for that download. The libraries may shipped as DLLs next to the executable, which are technically dynamic, but it's the same concept—those DLL's are program-specific. This ability is what I really miss when I try to switch to desktop Linux. I don't want to set up Docker containers for random desktop apps, and I don't want a given app to affect the state of my overall system. I want to download and run stuff. --- I realize there's a couple of big caveats here. Since Windows programs aren't sandboxed, misbehaving programs absolutely can hose a system—but at least that's not the intended way things are supposed to work. I'm also skipping over runtimes such as Visual C++, but as I see it, those can almost be considered part of the OS at this point. And I can a ridiculous number of versions of MSVC installed simultaneously without issue.

StreamBright 7y ago

Based on my experience this is very rarely the case unless you have an extremely disciplined SecOps team.

rumanator 7y ago
> Based on my experience this is very rarely the case You must have close to zero experience them because that's the norm on any software that depends on, say, third-party libraries that ship with a OS/distro. Recommended reading: Debian's openssl package. https://tracker.debian.org/pkg/openssl https://tracker.debian.org/pkg/openssl
- StreamBright 7y ago
  You are talking about a FOSS project, I am talking about a company that has a service that uses OpenSSL in production.
  - boomlinde 7y ago
    These are not diametrically opposed. Your company can have a service that uses OpenSSL in production that runs on Debian to automatically take advantage of Debian patches and updates if it's linked dynamically to the system provided OpenSSL. You can either employ an extremely disciplined SecOps team to carefully track updates and CVEs (you'd need this whether you're linking statically or dynamically) or you can use e.g. Debian to take advantage of their work to that end.
    - StreamBright 7y ago
      Every single company that I used to work for had an internal version of Linux that they approved for production. Internal release cycles are disconnected from external release cycles. On the top of that, some of these companies were not using system-wide packages at all, you had to reference a version of packages (like OpenSSL) during your build process. We had to do emergency patching for CVEs and bump the versions in every service. This way you can have 100% confidence what a particular service is running with a particular version of OpenSSL. This process do not depend on Debian's (or other FOSS vendor's) release cycles and the dependencies are explicit, therefore the vulnerability assessment is simpler (as opposed to go to every server and check which version is installed). Don't you think?

yellowapple 7y ago

My main issue with this rationale is that, in the vast majority of production environments (at least the ones I've seen in the wild, and indeed the ones I've built), updating dependencies for dynamically-linked dependents is part of the "release" process just like doing so for a statically-linked dependent, so this ends up being a distinction without a difference; in either circumstance, there's a "rebuild" as the development and/or operations teams test and deploy the new application and runtime environment. This is only slightly more relevant for pure system administration scenarios where the machine is exclusively running software prebuilt by some third-party vendor (e.g. your average Linux distro package repo). Even then, unless you're doing blind automatic upgrades (which some shops do, but it carries its own set of risks), you're still hopefully at least testing new versions and employing some sort of well-defined deployment workflow. Also, if that "security update" introduces a breaking change (which Shouldn't Happen™, but something something Murphy's Law something something), then - again - retesting and rebuilding a runtime environment for a dynamically-linked dependent v. rebuilding a statically-linked dependent is a distinction without a difference.