4 ms·

this is really about static vs dynamic linkage, regardless of the mechanism you use to effect run or build time binding of dependencies. there are as always tr

this is really about static vs dynamic linkage, regardless of the mechanism you use to effect run or build time binding of dependencies. there are as always tradeoffs, but a few more arguments against dynamic: by opening up a path (automatic updates) to a third party, you're providing a wide vector for those updates to cause new* problems (correctness, security) for your deployment, as well as fix old ones transitive api compatibility across these upgrades is more of a statement of faith and effort of will than any real guarentee. in reality the compatibilities between the various components are piecewise, and if the graph gets big enough it can be really hard to find a compatible set of libraries. runtime binary integration is always strictly worse than build time source integration, since there is tooling in the latter promoting consistency, and basically none in the former since you'd really like to be able to deploy changes in your own* software with very little effort - dependency updates can ride on the same train for free. presumably you have to have some intervention to restart your service against the new dependencies regardless. at that point its a question of the relative complexity of a dynamic restart of an existing service vs a build and deploy. in the limit these are both buttons. from that perspective, dynamic updates are a helpful crutch for people who don't have a good handle on their own build, test and deploy. thats pretty far from being an arguable fundamental good.

wrs 9y ago
>runtime binary integration is always strictly worse than build time source integration That is true, except in the case where you aren't empowered to do the building. Then dynamic update is better than no update.
pjc50 9y ago
> runtime binary integration is always strictly worse than build time source integration This is rather harsh on "plugin" architectures, COM, and OS libraries.
- pm90 9y ago
  It may be harsh but its true. Don't get me wrong: plugin architectures, OS libraries etc. do make the problem much easier to solve and 99% of the time you will be OK. Its that 1% outlier, when a dependency you thought was rock-solid-stable that needs an immediate security patch but fails in weird ways when you apply it that it really shows up. And because its so infrequent you may not even have any mechanism in place to fix it quickly.
hinkley 9y ago
I can't help but feel that twenty years from now we'll be tut-tutting because someone discovered that some old company is still running containers built in 2018 that have massive security vulnerabilities in them. There's a certain positive aspect to the treadmill effect you get with operating system upgrades that gets blunted once you can run another version of everything in a container nobody has rebuilt in ages.
- mkozlows 9y ago
  Although the counter-argument is, I've seen a whole bunch of cases where OSes weren't updated because this one old app wouldn't work on the new thing, so unmaintained legacy apps are always a problem.
  - hinkley 9y ago
    True. Can't update python because we are running our internal tool Dingus 2.1 on this machine. But in that case you might have 3 teams putting pressure on the Dingus team to upgrade their stuff. If they can just retort with "run it in a container dummy" then they have no incentive to ever fix it.
  - lowbloodsugar 9y ago
    side-eyes windows xp box controlling a machine
pxc 9y ago
> this is really about static vs dynamic linkage, regardless of the mechanism you use to effect run or build time binding of dependencies Are you familiar with the functional package management approach of tools like Nix (https://nixos.org/nix/ https://nixos.org/nix/) and Guix (https://www.gnu.org/software/guix/ https://www.gnu.org/software/guix/)? They use dynamic linking but store all packages in a content-addressed format, outside the usual FHS. Packages are free to rely on separate versions of their dependencies as needed. Packages are defined in a 'source-based' format, but because of the content-addressable storage strategy, both Nix and Guix can offer transparent binary caching. Guix folks in particular are working on verifying the integrity of binary caches, so you don't have to blindly trust them, and they've developed a system they call 'grafting' which can safely and quickly replace vulnerable packages without recompiling. Both Nix and Guix have some container infrastructure as well. I'm not as familiar with the Guix side, but Nix has tooling for deterministically generating Docker images and systemd-nspawn containers, so you can leverage these packaging tools at the same time as existing container deployment infrastructure. I like your analysis of 'build time' vs 'runtime' integration of dependencies, and I think you might enjoy playing with/critiquing Nix and Guix, because they address these problems in a strange and powerful way. They use dynamic linking for deduplication and small upgrades, but try very hard to integrate everything they can at build time. I'd be interested to see where you think their approach is strong and weak, if you ever get a chance to try it out.
- convolvatron 9y ago
  i dont really deal with these issues directly anymore, so no, I haven't had a chance to use any tools in the Nix family. but sure, once you eliminate the ambiguity around 'which bytes' then it hardly matters whether you distribute them in one chunk or several chunks. What makes Nix more interesting than just that is that it actually provides tools to manage the dependencies and build. As much as I think owning your whole world is important, it can really consume alot of engineering resource just due to the state of the world. Since Nix signatures are transitive (as I understand it) it also captures the versions of the included tools, which is usually too far for most organizations to go. And while issues having to do with compiler versions are rare, they can really consume alot of energy trying to track down. So, without having any firsthand experience, it seem like its as dependable as static linkage, and may be quite a bit more convenient.
eeZah7Ux 9y ago
> this is really about static vs dynamic linkage, regardless of the mechanism you use to effect run or build time binding of dependencies. No. Distributions have been ensuring security for 20+ years by: - using dynamic linking - unbundling/unvendoring dependencies into dedicated packages - having a security team that updates vulnerable packages in a timely manner - having tools to update vulnerable packages without having to rebuild/reinstall/reconfigure the world This is what happens when you don't do that: https://blog.acolyer.org/2017/04/03/a-study-of-security-vulnerabilities-on-docker-hub/ https://blog.acolyer.org/2017/04/03/a-study-of-security-vuln...

wrs 9y ago

>runtime binary integration is always strictly worse than build time source integration That is true, except in the case where you aren't empowered to do the building. Then dynamic update is better than no update.

pjc50 9y ago

> runtime binary integration is always strictly worse than build time source integration This is rather harsh on "plugin" architectures, COM, and OS libraries.

pm90 9y ago
It may be harsh but its true. Don't get me wrong: plugin architectures, OS libraries etc. do make the problem much easier to solve and 99% of the time you will be OK. Its that 1% outlier, when a dependency you thought was rock-solid-stable that needs an immediate security patch but fails in weird ways when you apply it that it really shows up. And because its so infrequent you may not even have any mechanism in place to fix it quickly.

hinkley 9y ago

I can't help but feel that twenty years from now we'll be tut-tutting because someone discovered that some old company is still running containers built in 2018 that have massive security vulnerabilities in them. There's a certain positive aspect to the treadmill effect you get with operating system upgrades that gets blunted once you can run another version of everything in a container nobody has rebuilt in ages.

mkozlows 9y ago
Although the counter-argument is, I've seen a whole bunch of cases where OSes weren't updated because this one old app wouldn't work on the new thing, so unmaintained legacy apps are always a problem.
- hinkley 9y ago
  True. Can't update python because we are running our internal tool Dingus 2.1 on this machine. But in that case you might have 3 teams putting pressure on the Dingus team to upgrade their stuff. If they can just retort with "run it in a container dummy" then they have no incentive to ever fix it.
- lowbloodsugar 9y ago
  side-eyes windows xp box controlling a machine

pxc 9y ago

> this is really about static vs dynamic linkage, regardless of the mechanism you use to effect run or build time binding of dependencies Are you familiar with the functional package management approach of tools like Nix (https://nixos.org/nix/ https://nixos.org/nix/) and Guix (https://www.gnu.org/software/guix/ https://www.gnu.org/software/guix/)? They use dynamic linking but store all packages in a content-addressed format, outside the usual FHS. Packages are free to rely on separate versions of their dependencies as needed. Packages are defined in a 'source-based' format, but because of the content-addressable storage strategy, both Nix and Guix can offer transparent binary caching. Guix folks in particular are working on verifying the integrity of binary caches, so you don't have to blindly trust them, and they've developed a system they call 'grafting' which can safely and quickly replace vulnerable packages without recompiling. Both Nix and Guix have some container infrastructure as well. I'm not as familiar with the Guix side, but Nix has tooling for deterministically generating Docker images and systemd-nspawn containers, so you can leverage these packaging tools at the same time as existing container deployment infrastructure. I like your analysis of 'build time' vs 'runtime' integration of dependencies, and I think you might enjoy playing with/critiquing Nix and Guix, because they address these problems in a strange and powerful way. They use dynamic linking for deduplication and small upgrades, but try very hard to integrate everything they can at build time. I'd be interested to see where you think their approach is strong and weak, if you ever get a chance to try it out.

convolvatron 9y ago
i dont really deal with these issues directly anymore, so no, I haven't had a chance to use any tools in the Nix family. but sure, once you eliminate the ambiguity around 'which bytes' then it hardly matters whether you distribute them in one chunk or several chunks. What makes Nix more interesting than just that is that it actually provides tools to manage the dependencies and build. As much as I think owning your whole world is important, it can really consume alot of engineering resource just due to the state of the world. Since Nix signatures are transitive (as I understand it) it also captures the versions of the included tools, which is usually too far for most organizations to go. And while issues having to do with compiler versions are rare, they can really consume alot of energy trying to track down. So, without having any firsthand experience, it seem like its as dependable as static linkage, and may be quite a bit more convenient.

eeZah7Ux 9y ago

> this is really about static vs dynamic linkage, regardless of the mechanism you use to effect run or build time binding of dependencies. No. Distributions have been ensuring security for 20+ years by: - using dynamic linking - unbundling/unvendoring dependencies into dedicated packages - having a security team that updates vulnerable packages in a timely manner - having tools to update vulnerable packages without having to rebuild/reinstall/reconfigure the world This is what happens when you don't do that: https://blog.acolyer.org/2017/04/03/a-study-of-security-vulnerabilities-on-docker-hub/ https://blog.acolyer.org/2017/04/03/a-study-of-security-vuln...