7 ms·

> The distribution / maintainer / package manager approach has proven to be an extremely reliable way to get trustworthy software. Many of us love it and want t

by hctaw 6y ago

> The distribution / maintainer / package manager approach has proven to be an extremely reliable way to get trustworthy software. Many of us love it and want to see it stick around. I disagree, it's proven to be inadequate for modern software development and that's why these new languages/ecosystems are springing up. The least reliable way to package and distribute software is by relying on traditional package managers. > Do what you can to give their applications the ability to easily split dependencies and support system provided libraries This is unrealistic. I do not trust system provided libraries to function with my applications because I've been burned so many times in the past. > how can I make this program as easy for maintainers to distribute as possible By statically linking everything as much as possible and shipping everything else in a self contained bundle with a launcher that overrides any symbols that might inadvertently be pulled in from the system. The universe I'd like to live in is where the only use case for dynamic linking are OS vendor APIs and cryptographically secure functions like TLS. My dream package manager would whitelist those system librarie and forbid distribution for any bundle that does contain the shared objects with the symbols it needs.

bscphil 6y ago
> I disagree, it's proven to be inadequate for modern software development Well, that's exactly why the OP (and my essay) are "anti" modern software development in many ways. The view is that we're moving away from the traditional open source ecosystem and methods of software development with these new technologies, which (to be clear) are good technologies, but were created mostly to solve problems that some large corporations have, not to solve the problems that the open source ecosystem has. > The least reliable way to package and distribute software is by relying on traditional package managers. Not sure what you mean by this, but it's entirely untrue in my experience. Anything I install with a package manager just works, 100% of the time. Stuff I try to get any other way is a shitshow, and the lack of "quality control" provided by maintainers speaks for itself. I mean, just look at Android apps or the Chrome extension store. Heaven forbid we go back to the days of curl | bash off someone's website. > By statically linking everything as much as possible and shipping everything else in a self contained bundle with a launcher that overrides any symbols that might inadvertently be pulled in from the system. I know you know this, but just to be clear, that's not a solution to the problem of "making things easier for maintainers to distribute", that's cutting maintainers out of the loop. The whole point of my focus on ecosystems is that this is something that I, as a user, don't want to happen.
- oblio 6y ago
  The open source solutions are primarily for C/C++. A bit for Perl, a bit for Python. But they haven't really moved on. Java has been tacked on since forever. Same for .NET, JavaScript, whatever. And if you wanted your program propagated to all major distros you'd have to wait a decade. Nobody has time for that. Not corporations, not mom and pop stores, and I doubt many hobbyists.
  - ticviking 6y ago
    Then you always have the option to package your software for the ones you use and not worry about the rest. If you want to make your program popular then packaging is part of the process needed.
    - bfrog 6y ago
      Not anymore
    - lifthrasiir 6y ago
      That's not the problem. (*) The actual problem is that the distro maintainers want split packages (for security and so on), not vendored, and this requirement was already burdensome for many languages other than C/C++. If vendored packages were acceptable I believe people would have of course contributed them. Maintainers made this obstacle themselves (for a good cause, arguably) and it seems farfetched for them to then complain other languages are uncooperative. * Was "isn't that a problem?", which was not I meant to say.
    - jholman 6y ago
      > If you want to make your program popular then packaging is part of the process needed. That is demonstrably not true. I'm not making a value judgment (maybe it SHOULD be true, I dunno), but it's not true. There's a lot of popular unpackaged software out there.
    - virgo_eye 6y ago
      I mean the whole argument here is that distro packaging is not needed, that there's a better way of doing it.
lucideer 6y ago
> The universe I'd like to live in is where the only use case for dynamic linking are OS vendor APIs and cryptographically secure functions like TLS. My dream package manager would whitelist those system librarie and forbid distribution for any bundle that does contain the shared objects with the symbols it needs. This idea seems to be predicated on a belief that OS vendor APIs and cryptographic libraries are the only attack surface for serious user-affecting software exploits.
- hctaw 6y ago
  They're obviously not, but they're the ones that package managers can help mitigate automatically. The rest is going to be up to the developers to patch.
nextos 6y ago
Some of the things you mention are not incompatible with package managers. Have you considered Nix? I actually agree with the parent post in terms of strongly preferring package managers to other means of distributing software. I've always found Linux much easier to admin than other OSes simply because of package managers.
- kuratkull 6y ago
  Was really pumped after playing with Nix for a bit. But got bitten twice in the first day. I tried to run my work project using Nix. It needs Pythons python-prctl library - that doesn't work in Nix. So that's a dud. Next I try to use it as a Nim environment - Nim is unable to compile anything in Nix due to some missing (bundled) Glibc symbol. (Nim works nicely in my standard installation). The cool "export full container" thing mentioned in the tutorial failed for all, even the simplest cases. So I am kinda disillusioned by Nix.
  - nextos 6y ago
    You can always emulate FHS environments for things that are hard to turn into a Nix package without significant effort. Aside from that, Guix is an alternative implementation that might have what you need.
erik_seaberg 6y ago
Are you saying Debian stable isn’t slow and careful enough in your experience? I usually hear just the opposite.
rcxdude 6y ago
> I do not trust system provided libraries to function with my applications because I've been burned so many times in the past. I agree. About half the issues I've had with dependencies has been due to distributions fiddling with upstream for some reason or another. Probably the main reason I like Arch (which has a policy of just following vanilla upstream, though they aren't immune: 'python' being python 3 is probably the biggest pain point)
pdonis 6y ago
>I disagree, it's proven to be inadequate for modern software development You're not disgreeing with the post you responded to; you're just stating a different priority. hctaw said the distro/maintainer/pm approach is extremely reliable at producing trustworthy software. That means trustworthy to the user. It says nothing at all about how hard producing that software is for the developer. You are saying that the distro/maintainer/pm approach makes it harder for the developer. That's true. But it doesn't contradict the above at all. And anyway, as a user, I don't care. I want my software to work, to be stable, and to not have security flaws; and if a security flaw is found, I want a fix to be pushed to me ASAP. The distro/maintainer/pm approach does that. If instead I have umpteen zillion different statically linked applications installed, each of which packages all of its own dependencies, then instead of just relying on my distro to push security fixes to shared libraries that everyone uses, I have to rely on every single one of those developers to do it for their own packages. And most of them won't do it, or they'll do it when they get around to it instead of when I, the user, need it. > The universe I'd like to live in is where the only use case for dynamic linking are OS vendor APIs and cryptographically secure functions like TLS This won't work either, because those are certainly not the only places where security flaws can happen that I, the user, need a fix for ASAP.
- secabeen 6y ago
  > And anyway, as a user, I don't care. I want my software to work, to be stable, and to not have security flaws; and if a security flaw is found, I want a fix to be pushed to me ASAP. The distro/maintainer/pm approach does that. If instead I have umpteen zillion different statically linked applications installed, each of which packages all of its own dependencies, then instead of just relying on my distro to push security fixes to shared libraries that everyone uses, I have to rely on every single one of those developers to do it for their own packages. And most of them won't do it, or they'll do it when they get around to it instead of when I, the user, need it. This is my main worry as well. We are currently in the early, easy, period of this new development paradigm, where developers are constantly releasing new code, and fixes are easy to deploy. I worry that in 10 or 15 years, when a security bug is found in a critical imported function, the developers aren't going to be around anymore to fix them, they will have moved on to the next new, hot, language, and will have as much interested in maintaining their old Go/Rust code as developers today do in maintaining their old C code.
0xbadcafebee 6y ago
> it's proven to be inadequate for modern software development Sure, because it's not meant for software development. It's meant for users to run software. Software development packagaing/distribution is pretty bad. Most languages do it their own way and never seem to learn the lessons of previous ones. And on top of that, they encourage poor habits, like writing your own module that's the same as somebody else's with one new function, rather than contributing to the existing module, or writing an extension for it. > I do not trust system provided libraries to function with my applications because I've been burned so many times in the past. Probably this is because both you + the library developer are not coordinating with the distributions on how you release your code. Distros get a lot of flack for breaking changes, but they're working from software released by developers, and rely entirely on their own user base to test changes. If developers cared about their software working they'd be more involved in its packaging & distribution.
jjav 6y ago
> The least reliable way to package and distribute software is by relying on traditional package managers. Checks and balances produce reliability. Having a packaging process and people distinct from the code author validating and enforcing it, most certainly produces much more reliable, secure and stable packages and distributions. Sure, it's slower. Slower is actually a feature in this use case. Having anyone push out their code to the world without any constraints or care, breaking compatibility day to day, that is certainly faster and easier. But reliable, certainly not. This is the culture that produces leftpad debacle and assorted malicious packages regularly.
- thu2111 6y ago
  It really doesn't. I've worked on upstream projects where a significant fraction of all bugs reported were created by distributions screwing up packaging and patching in ways they weren't at all qualified to understand and frequently led to non-obvious failures. When we tried to work with them to fix this, about half the time they flamed us and quoted distro 'policy' as a reason not to fix their bugs, so we just refused to accept bug reports from anyone using those packages anymore. The fact is that a lot of old-school Linux distributions are built by people who have only a very vague understanding of the software they're packaging, and frequently are closer to the sysadmin side of things than the large-scale software development side. It makes the relationships very frustrating and that's why proprietary software vendors invariably opt-out of distro packaging. Even with apps statically linked to the max possible level Linux users generate disproportionate levels of support tickets due to the general flakiness of the distros they use, so allowing them to modify tested software even further is a losing proposition. Basically the whole concept of a Linux distribution is obsolete, fading away and irretrievably broken. Hence the proliferation of containers.
drran 6y ago
> By statically linking everything as much as possible and shipping everything else in a self contained bundle with a launcher that overrides any symbols that might inadvertently be pulled in from the system. So, you propose to ship your own OS, as single image, for your application? What stops you from doing this? Drivers? Ship your own hardware then, like smartphone vendors do.

by hctaw 6y ago

bscphil 6y ago

> I disagree, it's proven to be inadequate for modern software development Well, that's exactly why the OP (and my essay) are "anti" modern software development in many ways. The view is that we're moving away from the traditional open source ecosystem and methods of software development with these new technologies, which (to be clear) are good technologies, but were created mostly to solve problems that some large corporations have, not to solve the problems that the open source ecosystem has. > The least reliable way to package and distribute software is by relying on traditional package managers. Not sure what you mean by this, but it's entirely untrue in my experience. Anything I install with a package manager just works, 100% of the time. Stuff I try to get any other way is a shitshow, and the lack of "quality control" provided by maintainers speaks for itself. I mean, just look at Android apps or the Chrome extension store. Heaven forbid we go back to the days of curl | bash off someone's website. > By statically linking everything as much as possible and shipping everything else in a self contained bundle with a launcher that overrides any symbols that might inadvertently be pulled in from the system. I know you know this, but just to be clear, that's not a solution to the problem of "making things easier for maintainers to distribute", that's cutting maintainers out of the loop. The whole point of my focus on ecosystems is that this is something that I, as a user, don't want to happen.

oblio 6y ago
The open source solutions are primarily for C/C++. A bit for Perl, a bit for Python. But they haven't really moved on. Java has been tacked on since forever. Same for .NET, JavaScript, whatever. And if you wanted your program propagated to all major distros you'd have to wait a decade. Nobody has time for that. Not corporations, not mom and pop stores, and I doubt many hobbyists.
- ticviking 6y ago
  Then you always have the option to package your software for the ones you use and not worry about the rest. If you want to make your program popular then packaging is part of the process needed.
  - bfrog 6y ago
    Not anymore
  - lifthrasiir 6y ago
    That's not the problem. (*) The actual problem is that the distro maintainers want split packages (for security and so on), not vendored, and this requirement was already burdensome for many languages other than C/C++. If vendored packages were acceptable I believe people would have of course contributed them. Maintainers made this obstacle themselves (for a good cause, arguably) and it seems farfetched for them to then complain other languages are uncooperative. * Was "isn't that a problem?", which was not I meant to say.
  - jholman 6y ago
    > If you want to make your program popular then packaging is part of the process needed. That is demonstrably not true. I'm not making a value judgment (maybe it SHOULD be true, I dunno), but it's not true. There's a lot of popular unpackaged software out there.
  - virgo_eye 6y ago
    I mean the whole argument here is that distro packaging is not needed, that there's a better way of doing it.

lucideer 6y ago

> The universe I'd like to live in is where the only use case for dynamic linking are OS vendor APIs and cryptographically secure functions like TLS. My dream package manager would whitelist those system librarie and forbid distribution for any bundle that does contain the shared objects with the symbols it needs. This idea seems to be predicated on a belief that OS vendor APIs and cryptographic libraries are the only attack surface for serious user-affecting software exploits.

hctaw 6y ago
They're obviously not, but they're the ones that package managers can help mitigate automatically. The rest is going to be up to the developers to patch.

nextos 6y ago

Some of the things you mention are not incompatible with package managers. Have you considered Nix? I actually agree with the parent post in terms of strongly preferring package managers to other means of distributing software. I've always found Linux much easier to admin than other OSes simply because of package managers.

kuratkull 6y ago
Was really pumped after playing with Nix for a bit. But got bitten twice in the first day. I tried to run my work project using Nix. It needs Pythons python-prctl library - that doesn't work in Nix. So that's a dud. Next I try to use it as a Nim environment - Nim is unable to compile anything in Nix due to some missing (bundled) Glibc symbol. (Nim works nicely in my standard installation). The cool "export full container" thing mentioned in the tutorial failed for all, even the simplest cases. So I am kinda disillusioned by Nix.
- nextos 6y ago
  You can always emulate FHS environments for things that are hard to turn into a Nix package without significant effort. Aside from that, Guix is an alternative implementation that might have what you need.

erik_seaberg 6y ago

Are you saying Debian stable isn’t slow and careful enough in your experience? I usually hear just the opposite.

rcxdude 6y ago

> I do not trust system provided libraries to function with my applications because I've been burned so many times in the past. I agree. About half the issues I've had with dependencies has been due to distributions fiddling with upstream for some reason or another. Probably the main reason I like Arch (which has a policy of just following vanilla upstream, though they aren't immune: 'python' being python 3 is probably the biggest pain point)

pdonis 6y ago

>I disagree, it's proven to be inadequate for modern software development You're not disgreeing with the post you responded to; you're just stating a different priority. hctaw said the distro/maintainer/pm approach is extremely reliable at producing trustworthy software. That means trustworthy to the user. It says nothing at all about how hard producing that software is for the developer. You are saying that the distro/maintainer/pm approach makes it harder for the developer. That's true. But it doesn't contradict the above at all. And anyway, as a user, I don't care. I want my software to work, to be stable, and to not have security flaws; and if a security flaw is found, I want a fix to be pushed to me ASAP. The distro/maintainer/pm approach does that. If instead I have umpteen zillion different statically linked applications installed, each of which packages all of its own dependencies, then instead of just relying on my distro to push security fixes to shared libraries that everyone uses, I have to rely on every single one of those developers to do it for their own packages. And most of them won't do it, or they'll do it when they get around to it instead of when I, the user, need it. > The universe I'd like to live in is where the only use case for dynamic linking are OS vendor APIs and cryptographically secure functions like TLS This won't work either, because those are certainly not the only places where security flaws can happen that I, the user, need a fix for ASAP.

secabeen 6y ago
> And anyway, as a user, I don't care. I want my software to work, to be stable, and to not have security flaws; and if a security flaw is found, I want a fix to be pushed to me ASAP. The distro/maintainer/pm approach does that. If instead I have umpteen zillion different statically linked applications installed, each of which packages all of its own dependencies, then instead of just relying on my distro to push security fixes to shared libraries that everyone uses, I have to rely on every single one of those developers to do it for their own packages. And most of them won't do it, or they'll do it when they get around to it instead of when I, the user, need it. This is my main worry as well. We are currently in the early, easy, period of this new development paradigm, where developers are constantly releasing new code, and fixes are easy to deploy. I worry that in 10 or 15 years, when a security bug is found in a critical imported function, the developers aren't going to be around anymore to fix them, they will have moved on to the next new, hot, language, and will have as much interested in maintaining their old Go/Rust code as developers today do in maintaining their old C code.

0xbadcafebee 6y ago

> it's proven to be inadequate for modern software development Sure, because it's not meant for software development. It's meant for users to run software. Software development packagaing/distribution is pretty bad. Most languages do it their own way and never seem to learn the lessons of previous ones. And on top of that, they encourage poor habits, like writing your own module that's the same as somebody else's with one new function, rather than contributing to the existing module, or writing an extension for it. > I do not trust system provided libraries to function with my applications because I've been burned so many times in the past. Probably this is because both you + the library developer are not coordinating with the distributions on how you release your code. Distros get a lot of flack for breaking changes, but they're working from software released by developers, and rely entirely on their own user base to test changes. If developers cared about their software working they'd be more involved in its packaging & distribution.

jjav 6y ago

> The least reliable way to package and distribute software is by relying on traditional package managers. Checks and balances produce reliability. Having a packaging process and people distinct from the code author validating and enforcing it, most certainly produces much more reliable, secure and stable packages and distributions. Sure, it's slower. Slower is actually a feature in this use case. Having anyone push out their code to the world without any constraints or care, breaking compatibility day to day, that is certainly faster and easier. But reliable, certainly not. This is the culture that produces leftpad debacle and assorted malicious packages regularly.

thu2111 6y ago
It really doesn't. I've worked on upstream projects where a significant fraction of all bugs reported were created by distributions screwing up packaging and patching in ways they weren't at all qualified to understand and frequently led to non-obvious failures. When we tried to work with them to fix this, about half the time they flamed us and quoted distro 'policy' as a reason not to fix their bugs, so we just refused to accept bug reports from anyone using those packages anymore. The fact is that a lot of old-school Linux distributions are built by people who have only a very vague understanding of the software they're packaging, and frequently are closer to the sysadmin side of things than the large-scale software development side. It makes the relationships very frustrating and that's why proprietary software vendors invariably opt-out of distro packaging. Even with apps statically linked to the max possible level Linux users generate disproportionate levels of support tickets due to the general flakiness of the distros they use, so allowing them to modify tested software even further is a losing proposition. Basically the whole concept of a Linux distribution is obsolete, fading away and irretrievably broken. Hence the proliferation of containers.

drran 6y ago

> By statically linking everything as much as possible and shipping everything else in a self contained bundle with a launcher that overrides any symbols that might inadvertently be pulled in from the system. So, you propose to ship your own OS, as single image, for your application? What stops you from doing this? Drivers? Ship your own hardware then, like smartphone vendors do.