6 ms·

It's frustrating to see the "rewrite it in Rust" meme continue spread to all sorts of projects when there is no reasonable packaging story for the language and

It's frustrating to see the "rewrite it in Rust" meme continue spread to all sorts of projects when there is no reasonable packaging story for the language and no solution in sight because the community does not see it as a problem. Cargo has introduced a huge problem for distros that didn't exist with languages like C. Compared to Rust, packaging C projects for distros is easy. Because of these problems, developers are receiving more issue reports from distro maintainers and are becoming more hostile to distro packaging, adopting a "use my binary or go away" mentality. Of course, this is a general problem that has been happening for all languages that come with a language-specific package manager; developers love them but they're typically unaware of the massive downstream problems they cause. It's getting harder to harder to exercise freedom 2 of free software and that's a shame.

ahupp 2y ago
It’s not like this is unique to rust; you see similar issues with node and python. Distributions have many jobs, but one was solving the lack of package management in C. Now that every modern language a package manager, trying to apply the C package management philosophy is untenable. Specifically, the idea of a single version, globally installed, and producing distro packages for every language specific packages.
- llm_trw 2y ago
  Apart from the fact building it the 'non-C way' results in a magic meat package that you have no idea what it contains: https://hpc.guix.info/blog/2021/09/whats-in-a-package/ https://hpc.guix.info/blog/2021/09/whats-in-a-package/ Guix is also a distro that allows for any number of versions of the same package globally, something that language specific dependancy managers do not. Distors are there for a reason, and anyone who doesn't understand that reason is just another contributor to the ongoing collapse of the tower of abstractions we've built.
  - ahupp 2y ago
    Using language-native packaging doesn't imply that you have to use binaries from wherever. In the pytorch example you can still build it as a regular part of the distribution, using the C++ dependencies/toolchain, it just means you don't try to stuff it into a versioning/distribution/install model that doesn't match the languages expectations.
  - tcfhgj 2y ago
    > Distors are there for a reason for me: make an os out of the kernel
  - pjmlp 2y ago
    Making an userspace to Linux kernel, and sorting out disagreements by creating yet another fork.
  - kpcyrd 2y ago
    This is outdated information. Debian (and other distros) already had their own SBOM format called buildinfo files that encodes this kind of information. In Debian stable ripgrep on amd64 is currently on version 13.0.0-4+b2. The relevant buildinfo file can be found here: https://buildinfos.debian.net/buildinfo-pool/r/rust-ripgrep/rust-ripgrep_13.0.0-4+b2_amd64.buildinfo https://buildinfos.debian.net/buildinfo-pool/r/rust-ripgrep/... It encodes the entire Rust dependency graph that was used for this binary, with exact versions of each crate.
- XorNot 2y ago
  Except from a management and maintenance perspective...this is a nightmare. When a security vulnerability drops somewhere, everywhere needs to be patched ASAP. Distros (and the people who run most scales of IT org) want to be able to deploy and verify that the fix is in place - and its a huge advantage if it's a linked library that you can just deploy an upgrade for. But if it's tons and tons of monolithic binaries, then the problem goes viral - every single one has to be recompiled, redeployed etc. And frequently at the cost of "are you only compatible with this specific revision, or was it just really easy to put that in?" It's worth noting that docker and friends also while still suffering from this problem, don't quite suffer from it in the same way - they're shipping entire dynamically linked environments, so while not as automatic, being able to simply scan for and replace the library you know is bad is a heck of a lot easier then recompiling a statically linked exe. People are okay with really specific dependencies when it's part of the business critical application they're supporting - i.e. the nodejs or python app which runs the business, that can do anything it wants we'll keep it running no matter what. Having this happen to the underlying distributions though? (of note: I've run into this issue with Go - love the static deploys, but if someone finds a vulnerability in the TLS stack of Go suddenly we're rushing out rebuilds).
  - curt15 2y ago
    >When a security vulnerability drops somewhere, everywhere needs to be patched ASAP. Is that ultimately the responsibility of the application developer or the OS developer?
    - XorNot 2y ago
      It's "whoever turns up to do the work" but I would point out that distros generally have more people in the process who can pick up the work. The issue is one way or another it needs to happen ASAP: so either the distro is haranguing upstream to "approve" a change, or they're going to be going in and carrying patches to Cargo.toml anyway - so this idea of "don't you dare touch my dependencies" lasts exactly as long as until you need a fix in ASAP.
      - 2y ago
- rlpb 2y ago
  > Now that every modern language a package manager... ...they fail to integrate with dependencies written in any other language. It's fine if you just want to sit a monoculture language software stack on top of a multilingual base platform. You can't make a functional system with one language alone, yet those who criticise distribution packaging architecture do so while simultaneously depending on this ability that language-specific package managers do not have. There is no viable alternative today. Most critics think they understand the general problem but only have narrow practical experience, so end up believing that their solution is superior while not considering the general multilingual software supply problem. Nix isn't a solution either, because in the general case Nix isn't security-supporting arbitrary and multiple dependency versions either.
zozbot234 2y ago
The "problems" are the same as with any statically-linked package (or, for that matter, any usage of AppImage, Flatpak, Snap, container images etc. as a binary distribution format). You can use Rust to build a "C project" (either a dynamic library exporting a plain C API/ABI, or an application linking to "C" dynamic libraries) and its packaging story will be the same as previous projects that were literally written in C. The language is just not very relevant here, if anything Rust has a better story than comparable languages like Golang, Haskell, Ocaml etc. that are just as challenging for distros.
- zajio1am 2y ago
  This is unrelated to whether it is statically-linked or dynamically-linked, it is about maintaining compatibility of API for libraries. In C, it is generally assumed that libraries maintain compatibility within one major version, so programs rarely have tight version intervals and maintainers could just use the newest available version (in each major series) for all packages depending on it. If the build system (for Rust and some other languages) makes it easy to depend on specific minor/patch versions (or upper-bound intervals of versions), it encourages developers to do so instead of working on fixing the mess in the ecosystem.
  - rtpg 2y ago
    A rust program depends on various libraries. It releases a specific version. Not pinning to specific dependencies for that specific version is weird for everyone who gets that program outside of their OS distribution package manager! I release foo 1.2.3 as a package, when doing it depending on bar 4.5.6. Perhaps foo still compiles with bar 4.5.2 or 4.6.8... but that's _not_ foo 1.2.3 right? And if there are bug fixes in some of those but not others, if I had my deps set up to just package with "bar 4.x" , suddenly I'm getting bug reports for "1.2.3" without a real association to "the" 1.2.3. I'm saying this, perhaps this is as easy as having two sets of deps (like deps for repackagers, deps for people wanting to reproduce what I built)... but if a binary is being shipped _not_ being explicit about the exact dependencies used is upstream of a whole lot of problems! Is there a great answer from OS distro package managers for this? Is the thing that should happen here be that there's a configure step in rust builds that would mess around with deps based on what you have?
forrestthewoods 2y ago
> and are becoming more hostile to distro packaging This to me sounds like the distro packaging model isn’t a good one.
- davexunit 2y ago
  Someone always makes this comment and it's always wrong.
  - forrestthewoods 2y ago
    The Linux model of global shared libraries is an objective failure. Everyone is forced to hack around this bad and broken design by using tools like Docker.
    - davexunit 2y ago
      It's not the "Linux model". It's an antiquated distro model that has been superseded by distros like Guix and NixOS that have shown you can still have an understandable dependency graph of your entire system without resorting to opaque binary blobs with Docker.
      - forrestthewoods 2y ago
        [flagged]
        davexunit 2y ago
        I use Guix. Have a nice day.
      - pjmlp 2y ago
        When will AWS, Azure, GCP adopt such great distros?
        davexunit 2y ago
lmm 2y ago
It's the distro packaging that makes it hard to exercise freedom 2, and it always has been - frankly when distro packagers intervene in language packaging it's always done more harm than good. Distributions still don't have good answers to installing packages locally, or even installing two versions of the same library side by side, and it's not reasonable for that to hold back packaging and development progress. (There are good solutions to this problem, but they look more like Nix than like traditional distro packaging).
- davexunit 2y ago
  Nix and Guix solve the problem, I don't see why they shouldn't count. Traditional global install distros are too inflexible and imperative.
  - xedrac 2y ago
    There is definite friction between Nix and Rust dependency management. Nix wants all dependencies to be a Nix package, but Rust wants Cargo to dynamically manage all dependencies. It seems like the same sort of problem. Or am I missing something?
    - xyzsparetimexyz 2y ago
      https://github.com/ipetkov/crane https://github.com/ipetkov/crane makes integrating rust and nix incredibly easy
phire 2y ago
The major source of conflict is that the disto model for c/c++ dependencies is simply not good. Developers only tolerate it because the problem was previously unsolved before linux package managers. We are talking about a time before even autoconf, almost everyone was rolling their own configure scripts, or creating makefiles which self-configured. It was a mess, and linux package managers were first to actually solve the problem. I fully believe that the explosive growth of linux in the early years had little to do with the kernel, and was much more about the proper package managers introduced by Debian, Red Hat, and friends. But just because package managers were massively better than nothing, and nothing (universally) better has come along since, doesn't mean they are actually good. The user experience is pretty good. But for developers, the drive to dynamically link every single dependency, and only have one (maybe two) versions of each dependency, and to randomly update dependencies causes massive problems for developers. It only works as well as it does for C/C++ applications because they have gotten used to working within the restrictive framework. And maybe a bit of Stockholm syndrome. ----------------- Let me tell you a story: Back in 2014, I was working on Dolphin Emulator (which is entirely c++), trying to fix regressions for a Dolphin 5.0 release. One of these regressions was that on linux, launching a game a second time (or opening the controller config window twice) would cause a crash. I quickly tracked the bug down to a bug introduced in SDL 2.0.3. Most consumers of SDL weren't impacted by this bug, because they only ever initialised SDL once. The fix was simple enough, it had already been fixed in the latest reversion and so would be fixed whenever 2.0.4 was released. But we wanted to release before that, so how to fix the bug? If this was cargo, I would just point it at a git revision (possibly a custom branch, with just the fix back ported). But that wasn't an option. I spent ages trying to work out a workaround that avoided the bug, but couldn't find one. I considered including a patched version of SDL with dolphin; Dolphin's source tree already contained a bunch of libraries which we statically linked in. Some patched, others were just there for easier builds on Windows/Mac). But I knew from experience that any distro would simply patch our build system to use the distro version. Dolphin's build system already had an extensive set of manual checks to use the distro version of a library if it existed. Many of these of these checks were contributed upstreams by distro package maintainers. I considered throwing an error message (twice, both at build and runtime) if dolphin was linked with a bugged version of SDL 2... But I figured the distro maintainers would probably patch out both error message too. Remember, we are talking about a bug that wouldn't even be fixed to the next release of SDL, with an uncertain release date. And there would probably be distros which stuck with the buggy version for way too long (Debian actually avoided the bug, because it was still shipping SDL 2.0.1, before the bug was introduced) Eventually I got frustrated. And I realised that SDL was only used for controller input, and only on linux/bsd. SDL already wasn't meeting our needs, we had a dedicated core input backend for MacOS and both DInput and XInput backends for Windows because they worked better. I had also just gone though the entire SDL input code, back when I was trying to find a workaround, and had learned how it work. So... to workaround the limitations of Linux Disto packaging (and a stupid, easy to fix SDL bug), I simply deleted SDL out of the Dolphin codebase. Only took me three days to completely reimplement all the controller input functionality, directly accessing the lower-level linux libraries: libudev for controller detection/hotpugging, open the correct `/dev/input/eventX` file, libevdev for parsing the raw input events. Three days to implement it. A few weeks of testing to find bugs... Not only did this workaround the bug. But it allowed Dolphin to access controller functionality that SDL wasn't exposing at the time (suddenly, the Dual Shock 4's touchpad would work, and the pressure sensitive buttons on Dual Sock 3). I don't think SDL was even exposing the gyroscope/accelerometers of those controllers at the time. --------------------- I was very happy with the result, but it was crazy that I was forced to go to such extremes, just because of the linux package manager insistence on enforcing one shared version of a dependency. I'm not sure how common it is for developers to rip-out and replace an entire dependency because of distro packaging, but workarounds for buggy library versions are common. These days, I'm really enjoying programming with rust; The fact that cargo always gives me the exact version of the package I asked for is a massive win. And why should we give it up, just because distros want to do something worse. And it would be so hard to make the transition, because a lot of rust code has been written under the assumption that cargo will give you the version (or version range) than you ask for. If you think about it, it's kind of insane that packages will be build and tested against one version of a library, just to have it randomly replaced with a later version, potentially introducing bugs. In some edge cases, packages even get downgraded to an older version between build/test and being installed on a user's system. --------------------- What's the solution for rust? I have no idea. Distros do have a point that statically linking everything isn't the best idea. Maybe we can reach some kind of middle ground where some dependencies are statically linked and others are dynamically linked, but with explicit support for multiple versions of each dependency co-existing on the system. Maybe we also add explicit support to package managers to track which dependencies have been statically linked into them, so we can force a rebuild of them all when there is a security issue.
kpcyrd 2y ago
> when there is no reasonable packaging story for the language For context: I've been around in the Debian Rust team since 2018, but I'm also a very active package maintainer in both Arch Linux and Alpine. Rust packaging is absolutely trivial with both Arch Linux and Alpine. For Debian specifically there's the policy of "all build inputs need to be present in the Debian archive", which means the source code needs to be spoon-fed from crates.io into the Debian archive. This is not a problem in itself, and cargo is actually incredibly helpful when building an operating system, since things are very streamlined and machine-readable instead of everybody handrolling their own build systems with Makefiles. Debian explicitly has cargo-based tooling to create source packages. The only manual step is often annotating copyright attributions, since this can not be sufficiently done automatically. The much bigger hurdle is the bureaucratic overhead. The librust-*-dev namespace is for the most part very well defined, but adding a new crate still requires an explicit approval process, even when uploads are sponsored by seasoned Debian Developers. There was a request for auto-approval for this namespace, like there is for llvm-* or linux-image-*, but back then (many years ago) this was declined. With this auto-approval rule in place it would also be easier to have (temporarily) multiple versions of a crate in Debian, to make library upgrades easier. This needs to be done sparsely however, since it takes up space in Packages.xz which is also downloaded by all users with every `apt update`. There's currently no way to make a package available only for build servers (and people who want to be one), but this concept has been discussed on mailing lists for this exact reason. This is all very specific to Debian however, I'm surprised you're blaming Rust developers for this.
- blub 2y ago
  > but adding a new crate still requires an explicit approval process, even when uploads are sponsored by seasoned Debian Developers. What’s the additional security benefit of the explicit approval? A major problem with Rust (for me) is the multitude of dependencies required even for trivial software, which complicates supply-chain monitoring. Auto-approval for crates gives me a bad feeling.
  - kpcyrd 2y ago
    It's not a security control, it's for copyright compliance and in case of incorrectly filled out package metadata.

ahupp 2y ago

It’s not like this is unique to rust; you see similar issues with node and python. Distributions have many jobs, but one was solving the lack of package management in C. Now that every modern language a package manager, trying to apply the C package management philosophy is untenable. Specifically, the idea of a single version, globally installed, and producing distro packages for every language specific packages.

llm_trw 2y ago
Apart from the fact building it the 'non-C way' results in a magic meat package that you have no idea what it contains: https://hpc.guix.info/blog/2021/09/whats-in-a-package/ https://hpc.guix.info/blog/2021/09/whats-in-a-package/ Guix is also a distro that allows for any number of versions of the same package globally, something that language specific dependancy managers do not. Distors are there for a reason, and anyone who doesn't understand that reason is just another contributor to the ongoing collapse of the tower of abstractions we've built.
- ahupp 2y ago
  Using language-native packaging doesn't imply that you have to use binaries from wherever. In the pytorch example you can still build it as a regular part of the distribution, using the C++ dependencies/toolchain, it just means you don't try to stuff it into a versioning/distribution/install model that doesn't match the languages expectations.
- tcfhgj 2y ago
  > Distors are there for a reason for me: make an os out of the kernel
- pjmlp 2y ago
  Making an userspace to Linux kernel, and sorting out disagreements by creating yet another fork.
- kpcyrd 2y ago
  This is outdated information. Debian (and other distros) already had their own SBOM format called buildinfo files that encodes this kind of information. In Debian stable ripgrep on amd64 is currently on version 13.0.0-4+b2. The relevant buildinfo file can be found here: https://buildinfos.debian.net/buildinfo-pool/r/rust-ripgrep/rust-ripgrep_13.0.0-4+b2_amd64.buildinfo https://buildinfos.debian.net/buildinfo-pool/r/rust-ripgrep/... It encodes the entire Rust dependency graph that was used for this binary, with exact versions of each crate.
XorNot 2y ago
Except from a management and maintenance perspective...this is a nightmare. When a security vulnerability drops somewhere, everywhere needs to be patched ASAP. Distros (and the people who run most scales of IT org) want to be able to deploy and verify that the fix is in place - and its a huge advantage if it's a linked library that you can just deploy an upgrade for. But if it's tons and tons of monolithic binaries, then the problem goes viral - every single one has to be recompiled, redeployed etc. And frequently at the cost of "are you only compatible with this specific revision, or was it just really easy to put that in?" It's worth noting that docker and friends also while still suffering from this problem, don't quite suffer from it in the same way - they're shipping entire dynamically linked environments, so while not as automatic, being able to simply scan for and replace the library you know is bad is a heck of a lot easier then recompiling a statically linked exe. People are okay with really specific dependencies when it's part of the business critical application they're supporting - i.e. the nodejs or python app which runs the business, that can do anything it wants we'll keep it running no matter what. Having this happen to the underlying distributions though? (of note: I've run into this issue with Go - love the static deploys, but if someone finds a vulnerability in the TLS stack of Go suddenly we're rushing out rebuilds).
- curt15 2y ago
  >When a security vulnerability drops somewhere, everywhere needs to be patched ASAP. Is that ultimately the responsibility of the application developer or the OS developer?
  - XorNot 2y ago
    It's "whoever turns up to do the work" but I would point out that distros generally have more people in the process who can pick up the work. The issue is one way or another it needs to happen ASAP: so either the distro is haranguing upstream to "approve" a change, or they're going to be going in and carrying patches to Cargo.toml anyway - so this idea of "don't you dare touch my dependencies" lasts exactly as long as until you need a fix in ASAP.
    - 2y ago
rlpb 2y ago
> Now that every modern language a package manager... ...they fail to integrate with dependencies written in any other language. It's fine if you just want to sit a monoculture language software stack on top of a multilingual base platform. You can't make a functional system with one language alone, yet those who criticise distribution packaging architecture do so while simultaneously depending on this ability that language-specific package managers do not have. There is no viable alternative today. Most critics think they understand the general problem but only have narrow practical experience, so end up believing that their solution is superior while not considering the general multilingual software supply problem. Nix isn't a solution either, because in the general case Nix isn't security-supporting arbitrary and multiple dependency versions either.

zozbot234 2y ago

The "problems" are the same as with any statically-linked package (or, for that matter, any usage of AppImage, Flatpak, Snap, container images etc. as a binary distribution format). You can use Rust to build a "C project" (either a dynamic library exporting a plain C API/ABI, or an application linking to "C" dynamic libraries) and its packaging story will be the same as previous projects that were literally written in C. The language is just not very relevant here, if anything Rust has a better story than comparable languages like Golang, Haskell, Ocaml etc. that are just as challenging for distros.

zajio1am 2y ago
This is unrelated to whether it is statically-linked or dynamically-linked, it is about maintaining compatibility of API for libraries. In C, it is generally assumed that libraries maintain compatibility within one major version, so programs rarely have tight version intervals and maintainers could just use the newest available version (in each major series) for all packages depending on it. If the build system (for Rust and some other languages) makes it easy to depend on specific minor/patch versions (or upper-bound intervals of versions), it encourages developers to do so instead of working on fixing the mess in the ecosystem.
- rtpg 2y ago
  A rust program depends on various libraries. It releases a specific version. Not pinning to specific dependencies for that specific version is weird for everyone who gets that program outside of their OS distribution package manager! I release foo 1.2.3 as a package, when doing it depending on bar 4.5.6. Perhaps foo still compiles with bar 4.5.2 or 4.6.8... but that's _not_ foo 1.2.3 right? And if there are bug fixes in some of those but not others, if I had my deps set up to just package with "bar 4.x" , suddenly I'm getting bug reports for "1.2.3" without a real association to "the" 1.2.3. I'm saying this, perhaps this is as easy as having two sets of deps (like deps for repackagers, deps for people wanting to reproduce what I built)... but if a binary is being shipped _not_ being explicit about the exact dependencies used is upstream of a whole lot of problems! Is there a great answer from OS distro package managers for this? Is the thing that should happen here be that there's a configure step in rust builds that would mess around with deps based on what you have?

forrestthewoods 2y ago

> and are becoming more hostile to distro packaging This to me sounds like the distro packaging model isn’t a good one.

davexunit 2y ago
Someone always makes this comment and it's always wrong.
- forrestthewoods 2y ago
  The Linux model of global shared libraries is an objective failure. Everyone is forced to hack around this bad and broken design by using tools like Docker.
  - davexunit 2y ago
    It's not the "Linux model". It's an antiquated distro model that has been superseded by distros like Guix and NixOS that have shown you can still have an understandable dependency graph of your entire system without resorting to opaque binary blobs with Docker.
    - forrestthewoods 2y ago
      [flagged]
      - davexunit 2y ago
        I use Guix. Have a nice day.
    - pjmlp 2y ago
      When will AWS, Azure, GCP adopt such great distros?
      - davexunit 2y ago

lmm 2y ago

It's the distro packaging that makes it hard to exercise freedom 2, and it always has been - frankly when distro packagers intervene in language packaging it's always done more harm than good. Distributions still don't have good answers to installing packages locally, or even installing two versions of the same library side by side, and it's not reasonable for that to hold back packaging and development progress. (There are good solutions to this problem, but they look more like Nix than like traditional distro packaging).

davexunit 2y ago
Nix and Guix solve the problem, I don't see why they shouldn't count. Traditional global install distros are too inflexible and imperative.
- xedrac 2y ago
  There is definite friction between Nix and Rust dependency management. Nix wants all dependencies to be a Nix package, but Rust wants Cargo to dynamically manage all dependencies. It seems like the same sort of problem. Or am I missing something?
  - xyzsparetimexyz 2y ago
    https://github.com/ipetkov/crane https://github.com/ipetkov/crane makes integrating rust and nix incredibly easy

phire 2y ago

The major source of conflict is that the disto model for c/c++ dependencies is simply not good. Developers only tolerate it because the problem was previously unsolved before linux package managers. We are talking about a time before even autoconf, almost everyone was rolling their own configure scripts, or creating makefiles which self-configured. It was a mess, and linux package managers were first to actually solve the problem. I fully believe that the explosive growth of linux in the early years had little to do with the kernel, and was much more about the proper package managers introduced by Debian, Red Hat, and friends. But just because package managers were massively better than nothing, and nothing (universally) better has come along since, doesn't mean they are actually good. The user experience is pretty good. But for developers, the drive to dynamically link every single dependency, and only have one (maybe two) versions of each dependency, and to randomly update dependencies causes massive problems for developers. It only works as well as it does for C/C++ applications because they have gotten used to working within the restrictive framework. And maybe a bit of Stockholm syndrome. ----------------- Let me tell you a story: Back in 2014, I was working on Dolphin Emulator (which is entirely c++), trying to fix regressions for a Dolphin 5.0 release. One of these regressions was that on linux, launching a game a second time (or opening the controller config window twice) would cause a crash. I quickly tracked the bug down to a bug introduced in SDL 2.0.3. Most consumers of SDL weren't impacted by this bug, because they only ever initialised SDL once. The fix was simple enough, it had already been fixed in the latest reversion and so would be fixed whenever 2.0.4 was released. But we wanted to release before that, so how to fix the bug? If this was cargo, I would just point it at a git revision (possibly a custom branch, with just the fix back ported). But that wasn't an option. I spent ages trying to work out a workaround that avoided the bug, but couldn't find one. I considered including a patched version of SDL with dolphin; Dolphin's source tree already contained a bunch of libraries which we statically linked in. Some patched, others were just there for easier builds on Windows/Mac). But I knew from experience that any distro would simply patch our build system to use the distro version. Dolphin's build system already had an extensive set of manual checks to use the distro version of a library if it existed. Many of these of these checks were contributed upstreams by distro package maintainers. I considered throwing an error message (twice, both at build and runtime) if dolphin was linked with a bugged version of SDL 2... But I figured the distro maintainers would probably patch out both error message too. Remember, we are talking about a bug that wouldn't even be fixed to the next release of SDL, with an uncertain release date. And there would probably be distros which stuck with the buggy version for way too long (Debian actually avoided the bug, because it was still shipping SDL 2.0.1, before the bug was introduced) Eventually I got frustrated. And I realised that SDL was only used for controller input, and only on linux/bsd. SDL already wasn't meeting our needs, we had a dedicated core input backend for MacOS and both DInput and XInput backends for Windows because they worked better. I had also just gone though the entire SDL input code, back when I was trying to find a workaround, and had learned how it work. So... to workaround the limitations of Linux Disto packaging (and a stupid, easy to fix SDL bug), I simply deleted SDL out of the Dolphin codebase. Only took me three days to completely reimplement all the controller input functionality, directly accessing the lower-level linux libraries: libudev for controller detection/hotpugging, open the correct `/dev/input/eventX` file, libevdev for parsing the raw input events. Three days to implement it. A few weeks of testing to find bugs... Not only did this workaround the bug. But it allowed Dolphin to access controller functionality that SDL wasn't exposing at the time (suddenly, the Dual Shock 4's touchpad would work, and the pressure sensitive buttons on Dual Sock 3). I don't think SDL was even exposing the gyroscope/accelerometers of those controllers at the time. --------------------- I was very happy with the result, but it was crazy that I was forced to go to such extremes, just because of the linux package manager insistence on enforcing one shared version of a dependency. I'm not sure how common it is for developers to rip-out and replace an entire dependency because of distro packaging, but workarounds for buggy library versions are common. These days, I'm really enjoying programming with rust; The fact that cargo always gives me the exact version of the package I asked for is a massive win. And why should we give it up, just because distros want to do something worse. And it would be so hard to make the transition, because a lot of rust code has been written under the assumption that cargo will give you the version (or version range) than you ask for. If you think about it, it's kind of insane that packages will be build and tested against one version of a library, just to have it randomly replaced with a later version, potentially introducing bugs. In some edge cases, packages even get downgraded to an older version between build/test and being installed on a user's system. --------------------- What's the solution for rust? I have no idea. Distros do have a point that statically linking everything isn't the best idea. Maybe we can reach some kind of middle ground where some dependencies are statically linked and others are dynamically linked, but with explicit support for multiple versions of each dependency co-existing on the system. Maybe we also add explicit support to package managers to track which dependencies have been statically linked into them, so we can force a rebuild of them all when there is a security issue.

kpcyrd 2y ago

> when there is no reasonable packaging story for the language For context: I've been around in the Debian Rust team since 2018, but I'm also a very active package maintainer in both Arch Linux and Alpine. Rust packaging is absolutely trivial with both Arch Linux and Alpine. For Debian specifically there's the policy of "all build inputs need to be present in the Debian archive", which means the source code needs to be spoon-fed from crates.io into the Debian archive. This is not a problem in itself, and cargo is actually incredibly helpful when building an operating system, since things are very streamlined and machine-readable instead of everybody handrolling their own build systems with Makefiles. Debian explicitly has cargo-based tooling to create source packages. The only manual step is often annotating copyright attributions, since this can not be sufficiently done automatically. The much bigger hurdle is the bureaucratic overhead. The librust-*-dev namespace is for the most part very well defined, but adding a new crate still requires an explicit approval process, even when uploads are sponsored by seasoned Debian Developers. There was a request for auto-approval for this namespace, like there is for llvm-* or linux-image-*, but back then (many years ago) this was declined. With this auto-approval rule in place it would also be easier to have (temporarily) multiple versions of a crate in Debian, to make library upgrades easier. This needs to be done sparsely however, since it takes up space in Packages.xz which is also downloaded by all users with every `apt update`. There's currently no way to make a package available only for build servers (and people who want to be one), but this concept has been discussed on mailing lists for this exact reason. This is all very specific to Debian however, I'm surprised you're blaming Rust developers for this.

blub 2y ago
> but adding a new crate still requires an explicit approval process, even when uploads are sponsored by seasoned Debian Developers. What’s the additional security benefit of the explicit approval? A major problem with Rust (for me) is the multitude of dependencies required even for trivial software, which complicates supply-chain monitoring. Auto-approval for crates gives me a bad feeling.
- kpcyrd 2y ago
  It's not a security control, it's for copyright compliance and in case of incorrectly filled out package metadata.