5 ms·

I've argued this stance once before, but I feel it needs reiterating on this topic: I for one am 100% convinced that whatever the (FOSS) software future has in

by c0l0 5y ago

I've argued this stance once before, but I feel it needs reiterating on this topic: I for one am 100% convinced that whatever the (FOSS) software future has in store for the world MUST have strong support for dynamic linking. If you look at something like `apt-cache show podman | grep ^Built-Using:` (Output: https://paste.debian.net/plain/1225449 https://paste.debian.net/plain/1225449) on Debian 11, you will see why. Imagine a few of those components shared between tens of packages, and security problems discovered im some. It's got to be any package maintainer's worst nightmare. "Traditional" distros run by volunteers and in participation of small-ish companies will not be able to cope with that kind of need for rebuild churn - and yet they will live amongst us for many years, if not decades, to come.

infogulch 5y ago
First, can this not be reduced to an automation and compute problem? A new version of a library with a bug fix or security fix is released, so rebuild a bunch of packages. Where's the issue? Statically linked binaries can be produced from dynamic libraries, so the "rebuild a bunch of packages" can be further reduced to "re-link a bunch of packages". Optimize bandwidth consumption by building locally and validating against a reproduceable build transparency log. Second, shouldn't the majority of the time rebuilding packages be dedicated to testing each application with the new library? Would you just skip that step if you used dynamic libraries?
duped 5y ago
I've argued the opposite stance numerous times, because shared dependencies are far rarer than they might appear to be while requiring custom packaging for different distros is a burden I don't want to bear as a package author. I want to have one build for all distros, not custom packaging for every one that has a slightly different definition of how software is supposed to be packaged. In my experience, non-developer users don't care about package managers (they want it to be invisible) but do care about one click install. The only way I've found to satisfy those users is to distribute applications outside the distros' package managers and statically link everything as much as possible. It saves me and my users time and money. Package management with inverted dependency trees is user and developer hostile, and it's why shit like podman and docker have to exist in the first place.
- convolvatron 5y ago
  maybe we can also mention that semver is more of a rough guidance than anything else. you dont really know if an X+Y that runs fine will really continue to run with X+Y' unless you tests that cover that workload and you've actually run them.
- chlorion 5y ago
  Creating packages for various distros is not your job as a software developer, and it never was intended to be your job in the first place. At least part of this stance is based on a misunderstanding of how Linux package manager style software distribution works. You most likely don't have experience creating and maintaining packages for every distro that exists, expecting you to be able to do this would be silly. Typically the users or developers of the countless different distros will be who handles packaging your software though, not yourself! As a developer you just need to provide the source code, use a standardized build system such as make, cmake, meson, cargo, etc, and a list of libraries that your software depends on. If you do these things, creating a distro specific package for your software will be totally trivial for whoever feels like doing it and contributing to their distro's repo. Most distros have decades worth of tooling accumulated to make packaging software that uses standard build systems absolutely trivial! Optionally, you can provide a flatpak image (or similar) for users who want to circumvent the system package manager.
  - duped 5y ago
    This is divorced from reality. You're right, I do not have the experience to package it on every distro. I do however get bug reports that "$app does not work on $distro" that I have to fix as the software developer of $app, because "just wait for another software developer that understands $distro to package it for you" is not an acceptable workaround for users. Even something that seems innocuous like, "oh sorry it's not available for that by default, here's a tarball" is too much friction for user applications. Flatpak is the closest to a solution we have, but it has its own issues. It's infinitely superior to "just use the standard build tool like $(N different build tools) and list your dependencies, then pretend it's someone else's problem to solve!"
  - 5y ago
jahewson 5y ago
If traditional distos don’t scale then perhaps we really should be looking at alternative approaches? We need systems that serve the users ahead of the maintainers.
- ciprian_craciun 5y ago
  At the moment, given that the majority of open-source software is hosted on GitHub, perhaps GitHub releases (as in downloads from the releases tab) might be one such possible "alternative software repository". It's only missing an official "installer"; but there are a few unofficial alternatives like `wget` and `curl`. :) ---- I'm not endorsing GitHub, I'm quite neutral about it, however at the moment it does offer a "standardized" experience. So much so, that whenever I see a project announced on HN and the link takes me to a landing page, I immediately look for a GitHub link and switch to that.
bscphil 5y ago
Well said. Dynamically linked packages managed by a package manager and created and distributed by maintainers is the Linux Way of doing things for multiple very good reasons. In fact, the article brilliant describes one of them: > Modern software distribution: AKA "actually free with no-ads apps store" of the Linux and BSD worlds. That's it. That's the advantage summed up in a single description that wasn't even intended as a description of the advantages of the current way of doing things! The article has a list of "cons", but it utterly fails to consider the possibility that this overwhelming advantage of what it calls "modern software distribution" is simply and inescapably tied to the maintainer-oriented approach that currently underlies it! Attempts to create modes of distribution that are even slightly different from maintainer-distribution (think Google Play) suck ass. For all its faults, F-droid is a vastly better platform because it insists that the software must be built and distributed by an F-droid maintainer. Stepping even further away into the realm of developer built opaque binaries is begging for chaos and misery. As you say, it's ultimately a security concern [1]. The article claims that a change is necessary because of "increased usage of languages and run-times that don't fit the current build model", "some of these new projects have large numbers ... of dependencies", but these are themselves problems with modern software development. Even putting aside distribution, bloated nodejs dependency trees create security vulnerabilities. The inability to develop software without pinning exact versions of your dependencies (which then need to be manually upgraded by the developer) creates security vulnerabilities and fragility. These are problems, not good reasons for changing our current way of distributing software! I'm convinced that there are some (like me) willing to die on this particular hill. Come what may, even if half of the developers out there switch to Go and only ship static binaries, we're going to continue working on and using traditional Linux distributions with maintainer controlled software. (To be clear: Go and static builds are warranted in many cases. For example, closed-source rarely updated software like games, and software that is "deployed" rather than installed. But these are not the base case for Linux distributions.) Further reading: http://kmkeen.com/maintainers-matter/ http://kmkeen.com/maintainers-matter/ [1] https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packagers-security-nightmare/ https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packag...
- ripley12 5y ago
  > Attempts to create modes of distribution that are even slightly different from maintainer-distribution (think Google Play) suck ass. I'm not sure Google Play is a great comparison here. Package managers like Homebrew and Scoop are probably better ones; when I just want the latest version of a CLI tool, they make that experience much better on macOS and Windows than on Linux (I know Homebrew sorta supports Linux, but it's still early days).

by c0l0 5y ago

infogulch 5y ago

First, can this not be reduced to an automation and compute problem? A new version of a library with a bug fix or security fix is released, so rebuild a bunch of packages. Where's the issue? Statically linked binaries can be produced from dynamic libraries, so the "rebuild a bunch of packages" can be further reduced to "re-link a bunch of packages". Optimize bandwidth consumption by building locally and validating against a reproduceable build transparency log. Second, shouldn't the majority of the time rebuilding packages be dedicated to testing each application with the new library? Would you just skip that step if you used dynamic libraries?

duped 5y ago

I've argued the opposite stance numerous times, because shared dependencies are far rarer than they might appear to be while requiring custom packaging for different distros is a burden I don't want to bear as a package author. I want to have one build for all distros, not custom packaging for every one that has a slightly different definition of how software is supposed to be packaged. In my experience, non-developer users don't care about package managers (they want it to be invisible) but do care about one click install. The only way I've found to satisfy those users is to distribute applications outside the distros' package managers and statically link everything as much as possible. It saves me and my users time and money. Package management with inverted dependency trees is user and developer hostile, and it's why shit like podman and docker have to exist in the first place.

convolvatron 5y ago
maybe we can also mention that semver is more of a rough guidance than anything else. you dont really know if an X+Y that runs fine will really continue to run with X+Y' unless you tests that cover that workload and you've actually run them.
chlorion 5y ago
Creating packages for various distros is not your job as a software developer, and it never was intended to be your job in the first place. At least part of this stance is based on a misunderstanding of how Linux package manager style software distribution works. You most likely don't have experience creating and maintaining packages for every distro that exists, expecting you to be able to do this would be silly. Typically the users or developers of the countless different distros will be who handles packaging your software though, not yourself! As a developer you just need to provide the source code, use a standardized build system such as make, cmake, meson, cargo, etc, and a list of libraries that your software depends on. If you do these things, creating a distro specific package for your software will be totally trivial for whoever feels like doing it and contributing to their distro's repo. Most distros have decades worth of tooling accumulated to make packaging software that uses standard build systems absolutely trivial! Optionally, you can provide a flatpak image (or similar) for users who want to circumvent the system package manager.
- duped 5y ago
  This is divorced from reality. You're right, I do not have the experience to package it on every distro. I do however get bug reports that "$app does not work on $distro" that I have to fix as the software developer of $app, because "just wait for another software developer that understands $distro to package it for you" is not an acceptable workaround for users. Even something that seems innocuous like, "oh sorry it's not available for that by default, here's a tarball" is too much friction for user applications. Flatpak is the closest to a solution we have, but it has its own issues. It's infinitely superior to "just use the standard build tool like $(N different build tools) and list your dependencies, then pretend it's someone else's problem to solve!"
- 5y ago

jahewson 5y ago

If traditional distos don’t scale then perhaps we really should be looking at alternative approaches? We need systems that serve the users ahead of the maintainers.

ciprian_craciun 5y ago
At the moment, given that the majority of open-source software is hosted on GitHub, perhaps GitHub releases (as in downloads from the releases tab) might be one such possible "alternative software repository". It's only missing an official "installer"; but there are a few unofficial alternatives like `wget` and `curl`. :) ---- I'm not endorsing GitHub, I'm quite neutral about it, however at the moment it does offer a "standardized" experience. So much so, that whenever I see a project announced on HN and the link takes me to a landing page, I immediately look for a GitHub link and switch to that.

bscphil 5y ago

Well said. Dynamically linked packages managed by a package manager and created and distributed by maintainers is the Linux Way of doing things for multiple very good reasons. In fact, the article brilliant describes one of them: > Modern software distribution: AKA "actually free with no-ads apps store" of the Linux and BSD worlds. That's it. That's the advantage summed up in a single description that wasn't even intended as a description of the advantages of the current way of doing things! The article has a list of "cons", but it utterly fails to consider the possibility that this overwhelming advantage of what it calls "modern software distribution" is simply and inescapably tied to the maintainer-oriented approach that currently underlies it! Attempts to create modes of distribution that are even slightly different from maintainer-distribution (think Google Play) suck ass. For all its faults, F-droid is a vastly better platform because it insists that the software must be built and distributed by an F-droid maintainer. Stepping even further away into the realm of developer built opaque binaries is begging for chaos and misery. As you say, it's ultimately a security concern [1]. The article claims that a change is necessary because of "increased usage of languages and run-times that don't fit the current build model", "some of these new projects have large numbers ... of dependencies", but these are themselves problems with modern software development. Even putting aside distribution, bloated nodejs dependency trees create security vulnerabilities. The inability to develop software without pinning exact versions of your dependencies (which then need to be manually upgraded by the developer) creates security vulnerabilities and fragility. These are problems, not good reasons for changing our current way of distributing software! I'm convinced that there are some (like me) willing to die on this particular hill. Come what may, even if half of the developers out there switch to Go and only ship static binaries, we're going to continue working on and using traditional Linux distributions with maintainer controlled software. (To be clear: Go and static builds are warranted in many cases. For example, closed-source rarely updated software like games, and software that is "deployed" rather than installed. But these are not the base case for Linux distributions.) Further reading: http://kmkeen.com/maintainers-matter/ http://kmkeen.com/maintainers-matter/ [1] https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packagers-security-nightmare/ https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packag...

ripley12 5y ago
> Attempts to create modes of distribution that are even slightly different from maintainer-distribution (think Google Play) suck ass. I'm not sure Google Play is a great comparison here. Package managers like Homebrew and Scoop are probably better ones; when I just want the latest version of a CLI tool, they make that experience much better on macOS and Windows than on Linux (I know Homebrew sorta supports Linux, but it's still early days).