5 ms·

Correct me if I'm wrong (really) but isn't static linking mostly a problem for packagers? The less a package maintainer modifies the application the better IMO.

Correct me if I'm wrong (really) but isn't static linking mostly a problem for packagers? The less a package maintainer modifies the application the better IMO. If application A and B rely on dependency D, which turns out to have a vulnerability, fixed in D', Then _why_ do we think it is anyone but A & B's developers responsibility to update to D' and distribute patched versions? If the packager tries to do it, the chances of diverging A and B due to some other incompatibility are too high. Either, you're running well maintained software (OSS or commercial), and get the update in a timely manner, or you're running unmaintained software and don't get the update. Only in the second case does it make sense to patch in a dynamically linked dependency. And I'm sure there's plenty of examples of this, but the real issue is running un-maintained software! I am much happier keeping in close sync with timely releases from Go/Rust projects than I am with Debian et. al's style of freezing the world.

bombcar 6y ago
The sad truth is without the package maintaining too many people would be running horribly out of date and insecure software - as there is zero incentive to upgrade a working system.
- jay_kyburz 6y ago
  I think if the software was labeled "insecure - requires update" people would update. If you don't know that something is insecure then there is zero incentive.
evmar 6y ago
The OP is talking about technical details like static linking and language ecosystems, but if you zoom one level out, this comment correctly pins where the underlying problem actually lies: that the distribution maintainers inject themselves into the development process of all the software they ship. When A/B are not well-maintained upstream, distributions can assist by updating its dependency D, but even when A/B are well-maintained upstream, distributions still might monkey with them when D changes, even when that change to D actually breaks the software. To me, the root cause is that the distributions are effectively attempting to participate the software they ship, but do so by getting in the middle, rather than participating upstream. As a former author of some well-maintained upstream software I found their involvement made the software overall worse, but as a user of a distribution I find their maintenance of otherwise unmantained software sometimes helpful. In other words, I think the goal is admirable but the mechanism is wrong, and doing things like adding more dynamic linking only further enable the bad behavior.
- choeger 6y ago
  I think it would help if distributions could snatch a piece of real estate in upstream software. Something like, "in every project, the debian/ root folder belongs to the debian project and follows their rules". The packaging could then verify this folder and put their patches, build scripts, etc. there. This would help upstream communication a lot, I guess.
  - CJefferson 6y ago
    The problem is there isn't just Debian, there is Debian and arch and Gentoo and nix and guix and freebsd and cygwin (at least). Then, how do I check who should have merge access to all these directories?
    - tgbugs 6y ago
      The problem is actually far worse than that. While the gp has what seems to be a reasonable solution, the issue is that you can't actually know what distribution is going to package your software, or what environment it is going to run in. There are code bases out there that still work decades later without any active maintenance, but if you need maintenance just to be able to build software in a new environment something is wrong. The underlying issue is that there are no standards for how to package software in a multi language environment. If I want to go from a state where (require 'module-name) fails to a state where (require 'module-name) succeeds, there are a potentially infinite number of ways that that could be accomplished, a single software project cannot ever specify all the possible ways for building software. What they can try to do use uniform interfaces and standard patterns for building their software, in a way that delegates dependency management to an external system. It seems that it is hard for software developers to admit that other people know more about how and where their software will be running than they do. Good engineering practice seems to dictate that dependency and environment management should be completely orthogonal to the development of an individual component or individual functionality. The op is 3 stories about what happens when the two are not kept orthogonal. The fundamental problem is that it is often easier for individual projects to make decisions that conflate the individual project with its dependencies (no longer orthogonal). To my knowledge, there is not a universal or well understood set of requirements that could be used to specify what a stable interface between an individual software project and its dependencies looks like. There are a number of candidates, such as gentoo ebuilds, rpm spec files, etc. however I have not seen one that effectively accommodates all of them. Further, there are languages where the implementation (or even design) makes it impossible to keep dependencies and individual projects orthogonal. The end result of non-orthognal systems is more work for everyone, more wasted cpu cycles, and worse security. Distros can't stop people from using languages that conflate the two, but they can tell them that they are on their own, and that the distros can't depend on components written in such languages in the core of the OS. To everyone pushing the rewrite it in Rust meme this should be a wakeup call. The current design decisions in the language and limitations of the implementation make it less secure than C or C++ because swapping out dependencies is bottlenecked by the centralized primary development team, and maintainers and users can't take orthogonal action to fix an issue.
choeger 6y ago
The distribution provides support for much longer than upstream does. Also, packaging is actual work. Just because some hipsters in some company consider it cool to release new features every 4 weeks, it does not mean that the volunteers of some linux distribution can keep up with thag. So if you want your distribution to work well, you should focus on workable and transparent interfaces. If you are happy with the support cycles of your upstream and can live with a black box, on the other hand you don't need a distribution in the first place. Besides, the whole "vendor everything, link statically" idea works well only for the leafs of the tree. Guess what Rust would do if llvm was not a readily usable library but a bunch of C++ files directly used inside clang? Thanks to the Rust mode of operation, it is impossible to do with the rust compiler what they did with llvm.
- anotherhue 6y ago
  > on the other hand you don't need a distribution in the first place. I think you're right about that, I personally want as little distribution as possible. FreeBSD ports or Arch AUR work well for me. The idea of solidifying other people's software into a bundle and then maintaining it (with necessarily limited expertise) seems like a loosing battle.
  - beojan 6y ago
    As someone who uses the AUR a lot, it's really the perfect example of how you just won't update software if it isn't done automatically by the distro package manager, particularly when you use -git packages.
nwallin 6y ago
> Correct me if I'm wrong (really) but isn't static linking mostly a problem for packagers? The less a package maintainer modifies the application the better IMO. IMHO that's precisely why dependencies should be unpinned. Let's say application A relies on dependencies B, C, and D, and dependencies B, C, and D depend on dependency E. Let's say dependency E has a critical security vulnerability and needs to be updated today to E'. Let's say you have unpinned versions: The packager for E updates E to E'. The end. Let's say you have pinned versions: The developer for B updates package B to depend on E', the developer for C updates C to depend on E', the developer for D updates D to depend on E', the developer for A updates A to depend on B', C', and D'. The packager for B updates the package for B, the packager for C updates the package for C, the packager for D updates the package for D, and the packager for A updates the package for A. You'll notice that there's a timing issue here. The packager for C cannot move until the developer for C has done the work, the developer for A cannot move until the developer for B, C, and D have done their work, and the packager for A cannot do anything until everyone has completed their work. If, for instance, the developers for C all live in Texas and their power's been out for a few days, and when they get power back they're busy with other stuff for a while, it might take quite some time for C's developers to get an official package posted. But it's that important that A gets updated, because A is a network service with a port open to the internet and E is openssl or whatever. So now what? In a perfect world, all software dependencies would have active, attentive, prompt maintainers, but it tends to not be that way. Lots of critical internet infrastructure packages have a maintainer who's just some random person in Nebraska, and they go on vacation, or lose interest, go to sleep at night, go to little league games on the weekend, some of them have day jobs. If we lived in a world where Apache can't be updated to use the latest dynamic library for openssl because the developer for leftpad is watching a movie and has their phone turned off, that's a very serious problem, and it's a crazy world I would not want to be a sysadmin in. Certainly, maybe the packager for E is gonna be off this week, but a distro's packaging team tends to have a much easier time filling in for a maintainer who's away if the package is loosely coupled with the application's build process. 90% of the time, if a package in Gentoo requires an update, all you need to do is `mv foo-1.2.3.ebuild foo-1.2.4.ebuild`, `repoman manifest` and git commit+push. (I can't speak for other distros.) The system isn't perfect, but IMHO it's much more robust to the unfortunate realities of the ugly, soft underbelly of the world than static linking is.

bombcar 6y ago

The sad truth is without the package maintaining too many people would be running horribly out of date and insecure software - as there is zero incentive to upgrade a working system.

jay_kyburz 6y ago
I think if the software was labeled "insecure - requires update" people would update. If you don't know that something is insecure then there is zero incentive.

evmar 6y ago

The OP is talking about technical details like static linking and language ecosystems, but if you zoom one level out, this comment correctly pins where the underlying problem actually lies: that the distribution maintainers inject themselves into the development process of all the software they ship. When A/B are not well-maintained upstream, distributions can assist by updating its dependency D, but even when A/B are well-maintained upstream, distributions still might monkey with them when D changes, even when that change to D actually breaks the software. To me, the root cause is that the distributions are effectively attempting to participate the software they ship, but do so by getting in the middle, rather than participating upstream. As a former author of some well-maintained upstream software I found their involvement made the software overall worse, but as a user of a distribution I find their maintenance of otherwise unmantained software sometimes helpful. In other words, I think the goal is admirable but the mechanism is wrong, and doing things like adding more dynamic linking only further enable the bad behavior.

choeger 6y ago
I think it would help if distributions could snatch a piece of real estate in upstream software. Something like, "in every project, the debian/ root folder belongs to the debian project and follows their rules". The packaging could then verify this folder and put their patches, build scripts, etc. there. This would help upstream communication a lot, I guess.
- CJefferson 6y ago
  The problem is there isn't just Debian, there is Debian and arch and Gentoo and nix and guix and freebsd and cygwin (at least). Then, how do I check who should have merge access to all these directories?
  - tgbugs 6y ago
    The problem is actually far worse than that. While the gp has what seems to be a reasonable solution, the issue is that you can't actually know what distribution is going to package your software, or what environment it is going to run in. There are code bases out there that still work decades later without any active maintenance, but if you need maintenance just to be able to build software in a new environment something is wrong. The underlying issue is that there are no standards for how to package software in a multi language environment. If I want to go from a state where (require 'module-name) fails to a state where (require 'module-name) succeeds, there are a potentially infinite number of ways that that could be accomplished, a single software project cannot ever specify all the possible ways for building software. What they can try to do use uniform interfaces and standard patterns for building their software, in a way that delegates dependency management to an external system. It seems that it is hard for software developers to admit that other people know more about how and where their software will be running than they do. Good engineering practice seems to dictate that dependency and environment management should be completely orthogonal to the development of an individual component or individual functionality. The op is 3 stories about what happens when the two are not kept orthogonal. The fundamental problem is that it is often easier for individual projects to make decisions that conflate the individual project with its dependencies (no longer orthogonal). To my knowledge, there is not a universal or well understood set of requirements that could be used to specify what a stable interface between an individual software project and its dependencies looks like. There are a number of candidates, such as gentoo ebuilds, rpm spec files, etc. however I have not seen one that effectively accommodates all of them. Further, there are languages where the implementation (or even design) makes it impossible to keep dependencies and individual projects orthogonal. The end result of non-orthognal systems is more work for everyone, more wasted cpu cycles, and worse security. Distros can't stop people from using languages that conflate the two, but they can tell them that they are on their own, and that the distros can't depend on components written in such languages in the core of the OS. To everyone pushing the rewrite it in Rust meme this should be a wakeup call. The current design decisions in the language and limitations of the implementation make it less secure than C or C++ because swapping out dependencies is bottlenecked by the centralized primary development team, and maintainers and users can't take orthogonal action to fix an issue.

choeger 6y ago

The distribution provides support for much longer than upstream does. Also, packaging is actual work. Just because some hipsters in some company consider it cool to release new features every 4 weeks, it does not mean that the volunteers of some linux distribution can keep up with thag. So if you want your distribution to work well, you should focus on workable and transparent interfaces. If you are happy with the support cycles of your upstream and can live with a black box, on the other hand you don't need a distribution in the first place. Besides, the whole "vendor everything, link statically" idea works well only for the leafs of the tree. Guess what Rust would do if llvm was not a readily usable library but a bunch of C++ files directly used inside clang? Thanks to the Rust mode of operation, it is impossible to do with the rust compiler what they did with llvm.

anotherhue 6y ago
> on the other hand you don't need a distribution in the first place. I think you're right about that, I personally want as little distribution as possible. FreeBSD ports or Arch AUR work well for me. The idea of solidifying other people's software into a bundle and then maintaining it (with necessarily limited expertise) seems like a loosing battle.
- beojan 6y ago
  As someone who uses the AUR a lot, it's really the perfect example of how you just won't update software if it isn't done automatically by the distro package manager, particularly when you use -git packages.

nwallin 6y ago

> Correct me if I'm wrong (really) but isn't static linking mostly a problem for packagers? The less a package maintainer modifies the application the better IMO. IMHO that's precisely why dependencies should be unpinned. Let's say application A relies on dependencies B, C, and D, and dependencies B, C, and D depend on dependency E. Let's say dependency E has a critical security vulnerability and needs to be updated today to E'. Let's say you have unpinned versions: The packager for E updates E to E'. The end. Let's say you have pinned versions: The developer for B updates package B to depend on E', the developer for C updates C to depend on E', the developer for D updates D to depend on E', the developer for A updates A to depend on B', C', and D'. The packager for B updates the package for B, the packager for C updates the package for C, the packager for D updates the package for D, and the packager for A updates the package for A. You'll notice that there's a timing issue here. The packager for C cannot move until the developer for C has done the work, the developer for A cannot move until the developer for B, C, and D have done their work, and the packager for A cannot do anything until everyone has completed their work. If, for instance, the developers for C all live in Texas and their power's been out for a few days, and when they get power back they're busy with other stuff for a while, it might take quite some time for C's developers to get an official package posted. But it's that important that A gets updated, because A is a network service with a port open to the internet and E is openssl or whatever. So now what? In a perfect world, all software dependencies would have active, attentive, prompt maintainers, but it tends to not be that way. Lots of critical internet infrastructure packages have a maintainer who's just some random person in Nebraska, and they go on vacation, or lose interest, go to sleep at night, go to little league games on the weekend, some of them have day jobs. If we lived in a world where Apache can't be updated to use the latest dynamic library for openssl because the developer for leftpad is watching a movie and has their phone turned off, that's a very serious problem, and it's a crazy world I would not want to be a sysadmin in. Certainly, maybe the packager for E is gonna be off this week, but a distro's packaging team tends to have a much easier time filling in for a maintainer who's away if the package is loosely coupled with the application's build process. 90% of the time, if a package in Gentoo requires an update, all you need to do is `mv foo-1.2.3.ebuild foo-1.2.4.ebuild`, `repoman manifest` and git commit+push. (I can't speak for other distros.) The system isn't perfect, but IMHO it's much more robust to the unfortunate realities of the ugly, soft underbelly of the world than static linking is.