6 ms·

This would be a good one for Rust enthusiasts to weigh in on. The issue raised is that some program written in rust insists on a very specific version of vario

This would be a good one for Rust enthusiasts to weigh in on. The issue raised is that some program written in rust insists on a very specific version of various dependencies. If other people change the metadata, it builds and seems to run ok with different versions. (Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solution). Linux likes to pack a finite set of versions of libraries (ideally just one version at a time) and use that dependency for multiple programs, totally at odds with the vendoring specific versions strategy. I'm not clear what a solution is to this one. In the specific case, sure, drop it from Debian. But this isn't the only program choosing to vendor dependencies instead of use them from the environment.

viraptor 2y ago
This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access. But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really want strict versions - I understand why the author wants to keep it that way. It's more of a self-inflicted Debian policy pain rather than anything to do with Rust. The author could be nice and vendor, but also he's completely right to refuse. The request is basically "we put handcuffs on ourselves, so please you do the work instead".
- agwa 2y ago
  The author would be fine with vendoring. The problem is that Debian doesn't allow it.
  - AshamedCaptain 2y ago
    And that's because the users of Debian are not fine with vendoring.
  - cesarb 2y ago
    > The problem is that Debian doesn't allow it. Vendoring used to be more common, even in Debian. But then there was an important zlib vulnerability (which IIRC could be exploited through manipulated compressed data), and they had to chase and fix all the many copies of zlib embedded all over the place. To make things worse, some of them were not only old versions of zlib, but also had modified that zlib code, so each had to be reviewed before applying the fix. Debian (and other traditional distributions) learned from that incident, and started a strong push to not only remove vendored copies of libraries, but also when possible use a single version of these libraries. Which means that, whenever an important security issue is once again found in a widely used library, only a single copy has to be fixed and updated.
    - chandlerswift 2y ago
      I'm interested in hearing more about the history behind Debian and zlib! I did some searching and the closest thing I could find was a nod to the same incident in the Upstream Guide[0]. Do you know of a place where I could read more about it? [0]: https://wiki.debian.org/UpstreamGuide#No_inclusion_of_third_party_code https://wiki.debian.org/UpstreamGuide#No_inclusion_of_third_...
      - cesarb 2y ago
        It's been so long ago, that it's hard to find all the discussions I had seen back then. I recall that it was after a long time without any zlib release, so looking at the zlib history, I think it was this one fixed in zlib 1.1.4 from 11 March 2002: http://www.zlib.org/advisory-2002-03-11.txt http://www.zlib.org/advisory-2002-03-11.txt Looking at the debian-devel archives around that date, I found a Debian developer complaining about the vendored zlib copies (https://lists.debian.org/debian-devel/2002/03/msg00716.html https://lists.debian.org/debian-devel/2002/03/msg00716.html), but not the full discussion about getting rid of vendored libraries, so it must have happened elsewhere.
        chandlerswift 2y ago
        Thanks for taking the time these references up for me!
- JackSlateur 2y ago
  if you deal with filesystems, you really want strict versions How are doing everybody else ? What is specific with filesystems, here ? This is a bad dependency management, nothing more. Vendoring is a PITA.
  - viraptor 2y ago
    Level of potentially messing up. If you have a chat app, relax the deps and it crashes - oh well. But if you have a tool which can cause data loss, you really want to be sure everyone's running what you tested. It's similar to why restic (backup software) has official, static, reproducible binaries in their releases.
    - JackSlateur 2y ago
      And if my kernel crashes ? And if whatever library bugged, oh well, random people can access my systems ? And if my bgpd bugged, oh well, we just inject random routes et forward data randomly ? Makes no sense.
      - viraptor 2y ago
        Your kernel has no external library dependencies. Relaxing library requirements can introduce security issues when the dependency behaviour changes, so I'm not sure what the argument is. If your bgp is bugged, yeah it sucks, but it's recoverable. We're using encrypted traffic these days to ensure that even captured traffic is safe. The difference here is that data corruption is not recoverable.
        deleted 2y ago
        [deleted]
        JackSlateur 2y ago
        Linux uses at least libc
        2y ago
- 1oooqooq 2y ago
  it is rust specific because rust is the first attemp to replace proper system engineering languages with one that while nicer on memory management and overall ergonomics, is worse in tooking. cargo brings many malpractices from java/JavaScript (maven, npm, etc) that were always shunned in systems engineering and, mark my words, will be a security nightmare for linux in the near future.
  - viraptor 2y ago
    Go, D, Ocaml, Nim have the same issue, most are older than Rust and are arguably systems languages. C/C++ built with Bazel or Conan will have the same issue too.
    - 1oooqooq 2y ago
      those are userland system applications at most... which i grant is also what the original article talked about, but still.
    - rc00 2y ago
      Go would only be comparable if you had hundreds of vendored dependencies but Go's community and culture have encouraged use of the standard library over external dependencies where possible. A core problem with Rust is the lack of an adequate standard library. The problem with Cargo is when you have an application with hundreds of dependencies. That does not exist in any mainstream Go application. 1. Why Rust Libraries May Never Exist - https://www.youtube.com/watch?v=769VqNup21Q https://www.youtube.com/watch?v=769VqNup21Q 2. The Rust Cargo ecosystem is broken - https://www.youtube.com/watch?v=OdMQwDtumJ8 https://www.youtube.com/watch?v=OdMQwDtumJ8
      - viraptor 2y ago
        Having a big standard library doesn't help that much since it needs separate updates. Every other golang app I use requires updated crypto, net, or sync. In theory they come in the standard library, but the moment you need updates, they're exactly the same as other dependencies for packaging purposes. (it's very common, yes I do packaging)
- AshamedCaptain 2y ago
  In fact, I am tired of the same issue e.g. when evaluating machine learning tools with Python. It's as if everyone just says "dependency = ${exact_version_I_have}" and never considers anything else. Not helped by the fact that most of these projects break ABI every other day which I still think it is just plain evil. Please don't say "just use containers/virtualenvs/vendoring/whatever" because at some point you obviously want these "containers" to use your real hardware. And your GPU driver is only going to be at one version, no matter how many statically linked executables you have.
  - aragilar 2y ago
    It's 100% culture: this happens across all languages.
  - rfoo 2y ago
    > And your GPU driver is only going to be at one version, no matter how many statically linked executables you have. This is a non-issue. libcuda.so (which is part of the NVIDIA driver, their userland part) can not be statically linked into anything, since it's closed-source and no .a shipped at all, so no possibility of misconfigure it to be statically linked. And NVIDIA drivers, as a whole (including a pair of libcuda.so and corresponding .ko), is infinitely backward-compatible, you can always use new driver with old libraries. So the problem you mentioned does not exist at all. Statically linking to cuBLAS / cuDNN etc never add more pain and can only reduce pain. Unless you hold it wrong.
    - AshamedCaptain 2y ago
      That is hardly true for NVIDIA over the years, as I have experienced personally, and it's definitely not true for about every other GPU driver in existence -- particularly AMD's RoCM, which even recommends specific Python versions to go with each ROCM version. In addition, replace "GPU" with any other piece of hardware, such as NPU. Or even software, such as your desktop environment. Do you think a statically linked binary will run with Wayland+1 ? It's already a chore to run games that link with older SDL versions, and the only reason you can at all is that SDL was dynamically linked... Even the kernel itself does a pretty poor job at maintaining backwards compatibility (e.g. contents of /sys changing every other year). Statically linking just hardcodes one of these things, and then forgets about the rest, such as IPC APIs, desktop APIs, filesystem structure, etc. which due to the poor culture out there are moving as much as the libraries themselves.
- cogman10 2y ago
  It should be noted that, particularly for rust, it's a silly policy as rust does not support dynamic linking of rust libraries. All rust applications are statically linked. So trying to force all projects using foo onto the same version of foo is just a huge headache with no real benefit to anything.
  - haileys 2y ago
    It remains a problem for source packages, which are an important part of Debian necessary for offline and reproducible builds.
  - bonzini 2y ago
    Almost—even if you need to rebuild dependencies, you can still patch security issues just once instead of doing so in every vendored copy. That's the first point in the parent comment. The same applies to Go.
  - rlpb 2y ago
    > So trying to force all projects using foo onto the same version of foo is just a huge headache with no real benefit to anything. It creates a security nightmare. Distributions expect to be able to security patch without waiting for laggard upstreams to bump their pinned dependency versions. If the ecosystem insists that only upstream-sanctioned versions are acceptable, then that runs contrary to this expectation, because it blocks distributions from patching or bumping dependency versions. If this case should be an exception, then you end up with a security team having to patch myriad different versions. That's the security nightmare.
    - deleted 2y ago
      [deleted]
    - cogman10 2y ago
      Fair point. I do think we need to reevaluate how security is managed in terms of software. It's often the case that we say "this library version has a CVE, we need to update it!" However, it's not often the case that the CVE in the library is even exploitable from the parent application. I'm thinking of a recent 9 CVE against zlib... But actually it was a rarely used extension API for working with compressed files. Something like path tracing to see if a vulnerable API is even accessible from the parent application/library before instructing the universe to adopt a new version.
- nubinetwork 2y ago
  > This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level Can confirm, I occasionally use an ai program that has to hard code all of their python dependencies, because it's the only way to get it to compile, let alone run properly... and then they go and change the underlying package manager, and figure out they have to hard code even more... it's a bloody mess, but thankfully you can just run it as a docker container once it's all working...
  - ptero 2y ago
    Not disagreeing, but IME much more often I have to run something that locks specific versions of their python dependencies because ... well ... why not -- kool kids run everything in a virtual environment anyway. Version locking is a powerful tool that should be used very sparingly. My 2c.
  - greatgib 2y ago
    I think that 2 different things are mixed up. For Rust it is language specific, because the thing being statically built, doing differently is very hard. For other languages like Python, there is not a need and you should not do that. The dynamic part of it make it so that developer should be resilient to different versions. And python also provide a lot of conveniences for that (just think about six). But in your case, you faced the current situation of lot of "data scientist" jumping the band wagon of AI are creating a lot of things like apprentice sorcerers without a real experience of software engineering.
    - nubinetwork 2y ago
      > For other languages like Python, there is not a need and you should not do that. Normally I would agree, but when it comes to cuda and rocm, and the one that intel uses... you find that you have to pin torch/audio/video, and then you have to start pinning its dependencies (and their dependencies etc etc) or else you start getting random build or runtime failures...
    - rcxdude 2y ago
      No, in both cases the issue is the same: whether the actual interface of the library is staying the same or not, and over what time period. Python packages have plenty of the same kind of breakages. usually it's unintentional, a bug or similar, sometimes it's a deliberate breaking change in a point release. Either way the effect is the same. Version X works in project A but newer version Y doesn't, and the reverse can be true for project B. It's just that in python you're most likely to get the problem when you run the code, wheras compiled languages can also fail at the build step as well as at runtime. It can be pretty bad with the bigger ML/scientific computing packages, hence all the version pinning that happens, though I will concur that generally speaking AI researchers don't understand package management in the slightest and you get all kinds of horrors as a result (e.g. mixing anaconda and pip in the same project, bonus points if it's blasted over the distro's python packaging for even more chaos. works on one very broken machine if you are lucky and don't touch anything). Debian's approach is similarly frustrating: they will try to make your code work with an older version of a library than the one you developed with, which doesn't even work within semver (semver only defines backwards compatibility, not forwards). That's what caused the problem in this case: rolling forward dependencies was not recommended but didn't break anything, rolling a dependency backwards meant it stopped working, which is something absolutely no-one should be surprised or upset by (at least it meant the build broke as opposed to debian shipping a broken package like would have likely happened with python. It just meant that debian was shipping a version with bugs fixed in upstream). If you're going to do that you've got to be prepared to take responsibility for fixing the problem that you've introduced.
haileys 2y ago
I think this is the same shape of issue that I’ve experienced with Debian for as long as I’ve used it - close to 15 years now. Debian is a great OS, but it targets stability and long term support for its releases. That just isn’t compatible with newer, faster moving software that’s still working towards stability. I remember it being an issue when I was playing around with Mono around 2010 ish, and it’s an issue now with bcachefs - a very new and fast moving technology. For motivated users, the solution is to install bcachefs-tools direct from upstream, or from a third party packager (if one exists). When bcachefs stabilises, I’m sure it’ll find its way back into Debian official.
- aragilar 2y ago
  What I think is somewhat interesting in this case (if I've read the post correctly) half the changes were for newer versions of dependencies. If that's the case, then I'm inclined to start wondering what's going on with bcachefs upstream (and the whole dependency tree), if they're not keeping up with versions.
  - arzig 2y ago
    My read is that whatever subset of the ecosystem bcachefs-tools just hasn’t stabilized. Presumably it will, but once everything is functionally at 1.x even if they don’t reach it officially this will be become a non-issue. That said, having dabled in rust there’s a temptation to name breaking changes to improve ergonomics because the type system is so expressive. I almost wonder if this is an argument for simpler type systems to reduce churn.
  - SkiFire13 2y ago
    Imagine that an update of bcache-fs is released and 3 dependencies (could even be transitive) were updated and the newer versions were not packaged in Debian. The maintainer now has one package to update but another 3 to package for the first time. That's additional work. AFAIK Fedora has a tool to automatize this work, but maybe Debian has additional requirements that make this harder.
    - 2y ago
rlpb 2y ago
Maintenance is much more practical if everything in your dependency tree uses the same version of every dependency. Achieving this in practice is much of the work distribution maintainers have to do. Doing this has a couple of key advantages: 1. when a security vulnerability needs patching in a stable distribution release, the security team only have to patch one version instead of a dozen or more different versions bundled all over the place 2. a library that depends on dependencies A and B, both of which depend on X, can actually work properly if it needs to pass API objects created by X across between A and B, since X is of the same version In an ecosystem where it's considered acceptable to "pin" versions of dependencies and also call any system that doesn't use the pinned versions "unsupported", both of the above two cases become impractical. Whether you use shared libraries or static libraries, the above matters still exist.
- orra 2y ago
  > 2. a library that depends on dependencies A and B, both of which depend on X, can actually work properly if it needs to pass API objects created by X across between A and B, since X is of the same version That feels like an advantage for the developer, not the distro maintainer. I find that amusing, because you're saying it's easy if they can use one version of the library. Well, that's also true if they're writing an application, hence pinning.
  - rlpb 2y ago
    > That feels like an advantage for the developer, not the distro maintainer. It's an advantage for the ecosystem, yes. Not the developer of the library (X in my example), but a different developer. Someone who is trying to write an app to do something interesting with A and B. > ...because you're saying it's easy if they can use one version of the library. Well, that's also true if they're writing an application I don't think that's true. Perhaps library A has pinned dependencies for X such that it is in conflict with unrelated library B that also uses pinned dependencies for X. Both libraries are now less useful to the ecosystem, because they are now inadvertently incompatible with each other (if their APIs expose X's objects). My example demonstrates why this doesn't work in the general case, and therefore why it's wrong to expect that everyone does it this way.
foobarqux 2y ago
As others have said this isn't just Rust, distro "release" packaging is untenable now for non-base-system packages: there are an ever increasing number of packages, they release with updates users need/want at a rate far faster than the distro release schedule and have too many conflicting dependencies. To deal with this today you install a stable base system with an overlaid "user" package system so Debian/Fedora + nixpkgs/guix/brew/snap/flatpak/mpr/pip/cargo/etc. Unfortunately because because although you can get most packages on nixpkgs you can't get all and you'll need to install multiple so it becomes a nightmare to maintain (how do you remember all the package managers you have to update during the xz vulnerability?) and extremely bloated due to duplicated dependencies (especially for GUI packages that require gnome or kde). You do get pretty far for CLI-only packages by just adding nixpkgs though. Too bad it's so terrible to use. MPR is also pretty interesting: You basically leverage the AUR for Debian and with a few changes you could probably make the dependency names translation automatic and transparent. It solves the bloat problem (since you'll mostly use system packages) but doesn't help with library versions.
- ploxiln 2y ago
  Shouldn't the filesystem utilities be base-system packages, though? Like e2fsprogs, xfsprogs ... I guess rust just isn't appropriate for base-system packages, because the dependencies move too fast, and so rust components need to depend on different specific versions of all dependencies, and are "horrifically out-of-date" in just a few months ... and rust developers wouldn't stoop so low as to recognize that e.g. bindgen is a particularly hairy and touchy dependency so they should just vendor the bindings output by bindgen, rather than the entire bindgen tool plus all other dependencies in full ...
  - foobarqux 2y ago
    I'm not that familiar with bcachefs-tools and was speaking more generally but I thought bcachefs-tools wasn't critical for normal booting (On Ubuntu 24.04 nothing seems to depend on that package and it seems optional. The same for btrfs-tools). Some other comments suggested you need it to mount a degraded file system but it seems to me that is a case where either 1. your system can't boot and you need external recovery boot tools anyway or 2. you can boot and can run whatever tools you want (you can install the latest version of bcachefs-tools via nixpkgs for example). The real problem there seems to be that the program was silently broken due to the changes made to get it to build on Debian. Though I agree that it is closer to a system-critical package than most packages.
BoingBoomTschak 2y ago
Gentoo has SLOTs and they work well. But for Go and Rust, I think they chose the vendoring path because it's just too damn Don Quixotesque to try to mirror their NPM tier package repositories.
lambda 2y ago
> Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solutio Can you link to the post on Reddit you are referring to?
- dfc 2y ago
  I think this is the comment: https://www.reddit.com/r/bcachefs/comments/1f4erbg/comment/lkped4c/ https://www.reddit.com/r/bcachefs/comments/1f4erbg/comment/l...
pornel 2y ago
> rust insists on a very specific version of various dependencies It only insists on semver-compatible versions (if a Rust/Cargo package specifies libfoo=5.1, it will work with libfoo=5.9). It's one per major version, not that different from Debian packaging "libfoo5" and "libfoo7" when both are needed. The difference is that Cargo unifies versions by updating them to the newest compatible, and Debian unifies them by downgrading them to old unsupported versions, ignores compatibility, and reintroduces bugs, and disables required functionality.

viraptor 2y ago

This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access. But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really want strict versions - I understand why the author wants to keep it that way. It's more of a self-inflicted Debian policy pain rather than anything to do with Rust. The author could be nice and vendor, but also he's completely right to refuse. The request is basically "we put handcuffs on ourselves, so please you do the work instead".

agwa 2y ago
The author would be fine with vendoring. The problem is that Debian doesn't allow it.
- AshamedCaptain 2y ago
  And that's because the users of Debian are not fine with vendoring.
- cesarb 2y ago
  > The problem is that Debian doesn't allow it. Vendoring used to be more common, even in Debian. But then there was an important zlib vulnerability (which IIRC could be exploited through manipulated compressed data), and they had to chase and fix all the many copies of zlib embedded all over the place. To make things worse, some of them were not only old versions of zlib, but also had modified that zlib code, so each had to be reviewed before applying the fix. Debian (and other traditional distributions) learned from that incident, and started a strong push to not only remove vendored copies of libraries, but also when possible use a single version of these libraries. Which means that, whenever an important security issue is once again found in a widely used library, only a single copy has to be fixed and updated.
  - chandlerswift 2y ago
    I'm interested in hearing more about the history behind Debian and zlib! I did some searching and the closest thing I could find was a nod to the same incident in the Upstream Guide[0]. Do you know of a place where I could read more about it? [0]: https://wiki.debian.org/UpstreamGuide#No_inclusion_of_third_party_code https://wiki.debian.org/UpstreamGuide#No_inclusion_of_third_...
    - cesarb 2y ago
      It's been so long ago, that it's hard to find all the discussions I had seen back then. I recall that it was after a long time without any zlib release, so looking at the zlib history, I think it was this one fixed in zlib 1.1.4 from 11 March 2002: http://www.zlib.org/advisory-2002-03-11.txt http://www.zlib.org/advisory-2002-03-11.txt Looking at the debian-devel archives around that date, I found a Debian developer complaining about the vendored zlib copies (https://lists.debian.org/debian-devel/2002/03/msg00716.html https://lists.debian.org/debian-devel/2002/03/msg00716.html), but not the full discussion about getting rid of vendored libraries, so it must have happened elsewhere.
      - chandlerswift 2y ago
        Thanks for taking the time these references up for me!
JackSlateur 2y ago
if you deal with filesystems, you really want strict versions How are doing everybody else ? What is specific with filesystems, here ? This is a bad dependency management, nothing more. Vendoring is a PITA.
- viraptor 2y ago
  Level of potentially messing up. If you have a chat app, relax the deps and it crashes - oh well. But if you have a tool which can cause data loss, you really want to be sure everyone's running what you tested. It's similar to why restic (backup software) has official, static, reproducible binaries in their releases.
  - JackSlateur 2y ago
    And if my kernel crashes ? And if whatever library bugged, oh well, random people can access my systems ? And if my bgpd bugged, oh well, we just inject random routes et forward data randomly ? Makes no sense.
    - viraptor 2y ago
      Your kernel has no external library dependencies. Relaxing library requirements can introduce security issues when the dependency behaviour changes, so I'm not sure what the argument is. If your bgp is bugged, yeah it sucks, but it's recoverable. We're using encrypted traffic these days to ensure that even captured traffic is safe. The difference here is that data corruption is not recoverable.
      - deleted 2y ago
        [deleted]
      - JackSlateur 2y ago
        Linux uses at least libc
        2y ago
1oooqooq 2y ago
it is rust specific because rust is the first attemp to replace proper system engineering languages with one that while nicer on memory management and overall ergonomics, is worse in tooking. cargo brings many malpractices from java/JavaScript (maven, npm, etc) that were always shunned in systems engineering and, mark my words, will be a security nightmare for linux in the near future.
- viraptor 2y ago
  Go, D, Ocaml, Nim have the same issue, most are older than Rust and are arguably systems languages. C/C++ built with Bazel or Conan will have the same issue too.
  - 1oooqooq 2y ago
    those are userland system applications at most... which i grant is also what the original article talked about, but still.
  - rc00 2y ago
    Go would only be comparable if you had hundreds of vendored dependencies but Go's community and culture have encouraged use of the standard library over external dependencies where possible. A core problem with Rust is the lack of an adequate standard library. The problem with Cargo is when you have an application with hundreds of dependencies. That does not exist in any mainstream Go application. 1. Why Rust Libraries May Never Exist - https://www.youtube.com/watch?v=769VqNup21Q https://www.youtube.com/watch?v=769VqNup21Q 2. The Rust Cargo ecosystem is broken - https://www.youtube.com/watch?v=OdMQwDtumJ8 https://www.youtube.com/watch?v=OdMQwDtumJ8
    - viraptor 2y ago
      Having a big standard library doesn't help that much since it needs separate updates. Every other golang app I use requires updated crypto, net, or sync. In theory they come in the standard library, but the moment you need updates, they're exactly the same as other dependencies for packaging purposes. (it's very common, yes I do packaging)
AshamedCaptain 2y ago
In fact, I am tired of the same issue e.g. when evaluating machine learning tools with Python. It's as if everyone just says "dependency = ${exact_version_I_have}" and never considers anything else. Not helped by the fact that most of these projects break ABI every other day which I still think it is just plain evil. Please don't say "just use containers/virtualenvs/vendoring/whatever" because at some point you obviously want these "containers" to use your real hardware. And your GPU driver is only going to be at one version, no matter how many statically linked executables you have.
- aragilar 2y ago
  It's 100% culture: this happens across all languages.
- rfoo 2y ago
  > And your GPU driver is only going to be at one version, no matter how many statically linked executables you have. This is a non-issue. libcuda.so (which is part of the NVIDIA driver, their userland part) can not be statically linked into anything, since it's closed-source and no .a shipped at all, so no possibility of misconfigure it to be statically linked. And NVIDIA drivers, as a whole (including a pair of libcuda.so and corresponding .ko), is infinitely backward-compatible, you can always use new driver with old libraries. So the problem you mentioned does not exist at all. Statically linking to cuBLAS / cuDNN etc never add more pain and can only reduce pain. Unless you hold it wrong.
  - AshamedCaptain 2y ago
    That is hardly true for NVIDIA over the years, as I have experienced personally, and it's definitely not true for about every other GPU driver in existence -- particularly AMD's RoCM, which even recommends specific Python versions to go with each ROCM version. In addition, replace "GPU" with any other piece of hardware, such as NPU. Or even software, such as your desktop environment. Do you think a statically linked binary will run with Wayland+1 ? It's already a chore to run games that link with older SDL versions, and the only reason you can at all is that SDL was dynamically linked... Even the kernel itself does a pretty poor job at maintaining backwards compatibility (e.g. contents of /sys changing every other year). Statically linking just hardcodes one of these things, and then forgets about the rest, such as IPC APIs, desktop APIs, filesystem structure, etc. which due to the poor culture out there are moving as much as the libraries themselves.
cogman10 2y ago
It should be noted that, particularly for rust, it's a silly policy as rust does not support dynamic linking of rust libraries. All rust applications are statically linked. So trying to force all projects using foo onto the same version of foo is just a huge headache with no real benefit to anything.
- haileys 2y ago
  It remains a problem for source packages, which are an important part of Debian necessary for offline and reproducible builds.
- bonzini 2y ago
  Almost—even if you need to rebuild dependencies, you can still patch security issues just once instead of doing so in every vendored copy. That's the first point in the parent comment. The same applies to Go.
- rlpb 2y ago
  > So trying to force all projects using foo onto the same version of foo is just a huge headache with no real benefit to anything. It creates a security nightmare. Distributions expect to be able to security patch without waiting for laggard upstreams to bump their pinned dependency versions. If the ecosystem insists that only upstream-sanctioned versions are acceptable, then that runs contrary to this expectation, because it blocks distributions from patching or bumping dependency versions. If this case should be an exception, then you end up with a security team having to patch myriad different versions. That's the security nightmare.
  - deleted 2y ago
    [deleted]
  - cogman10 2y ago
    Fair point. I do think we need to reevaluate how security is managed in terms of software. It's often the case that we say "this library version has a CVE, we need to update it!" However, it's not often the case that the CVE in the library is even exploitable from the parent application. I'm thinking of a recent 9 CVE against zlib... But actually it was a rarely used extension API for working with compressed files. Something like path tracing to see if a vulnerable API is even accessible from the parent application/library before instructing the universe to adopt a new version.
nubinetwork 2y ago
> This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level Can confirm, I occasionally use an ai program that has to hard code all of their python dependencies, because it's the only way to get it to compile, let alone run properly... and then they go and change the underlying package manager, and figure out they have to hard code even more... it's a bloody mess, but thankfully you can just run it as a docker container once it's all working...
- ptero 2y ago
  Not disagreeing, but IME much more often I have to run something that locks specific versions of their python dependencies because ... well ... why not -- kool kids run everything in a virtual environment anyway. Version locking is a powerful tool that should be used very sparingly. My 2c.
- greatgib 2y ago
  I think that 2 different things are mixed up. For Rust it is language specific, because the thing being statically built, doing differently is very hard. For other languages like Python, there is not a need and you should not do that. The dynamic part of it make it so that developer should be resilient to different versions. And python also provide a lot of conveniences for that (just think about six). But in your case, you faced the current situation of lot of "data scientist" jumping the band wagon of AI are creating a lot of things like apprentice sorcerers without a real experience of software engineering.
  - nubinetwork 2y ago
    > For other languages like Python, there is not a need and you should not do that. Normally I would agree, but when it comes to cuda and rocm, and the one that intel uses... you find that you have to pin torch/audio/video, and then you have to start pinning its dependencies (and their dependencies etc etc) or else you start getting random build or runtime failures...
  - rcxdude 2y ago
    No, in both cases the issue is the same: whether the actual interface of the library is staying the same or not, and over what time period. Python packages have plenty of the same kind of breakages. usually it's unintentional, a bug or similar, sometimes it's a deliberate breaking change in a point release. Either way the effect is the same. Version X works in project A but newer version Y doesn't, and the reverse can be true for project B. It's just that in python you're most likely to get the problem when you run the code, wheras compiled languages can also fail at the build step as well as at runtime. It can be pretty bad with the bigger ML/scientific computing packages, hence all the version pinning that happens, though I will concur that generally speaking AI researchers don't understand package management in the slightest and you get all kinds of horrors as a result (e.g. mixing anaconda and pip in the same project, bonus points if it's blasted over the distro's python packaging for even more chaos. works on one very broken machine if you are lucky and don't touch anything). Debian's approach is similarly frustrating: they will try to make your code work with an older version of a library than the one you developed with, which doesn't even work within semver (semver only defines backwards compatibility, not forwards). That's what caused the problem in this case: rolling forward dependencies was not recommended but didn't break anything, rolling a dependency backwards meant it stopped working, which is something absolutely no-one should be surprised or upset by (at least it meant the build broke as opposed to debian shipping a broken package like would have likely happened with python. It just meant that debian was shipping a version with bugs fixed in upstream). If you're going to do that you've got to be prepared to take responsibility for fixing the problem that you've introduced.

haileys 2y ago

I think this is the same shape of issue that I’ve experienced with Debian for as long as I’ve used it - close to 15 years now. Debian is a great OS, but it targets stability and long term support for its releases. That just isn’t compatible with newer, faster moving software that’s still working towards stability. I remember it being an issue when I was playing around with Mono around 2010 ish, and it’s an issue now with bcachefs - a very new and fast moving technology. For motivated users, the solution is to install bcachefs-tools direct from upstream, or from a third party packager (if one exists). When bcachefs stabilises, I’m sure it’ll find its way back into Debian official.

aragilar 2y ago
What I think is somewhat interesting in this case (if I've read the post correctly) half the changes were for newer versions of dependencies. If that's the case, then I'm inclined to start wondering what's going on with bcachefs upstream (and the whole dependency tree), if they're not keeping up with versions.
- arzig 2y ago
  My read is that whatever subset of the ecosystem bcachefs-tools just hasn’t stabilized. Presumably it will, but once everything is functionally at 1.x even if they don’t reach it officially this will be become a non-issue. That said, having dabled in rust there’s a temptation to name breaking changes to improve ergonomics because the type system is so expressive. I almost wonder if this is an argument for simpler type systems to reduce churn.
- SkiFire13 2y ago
  Imagine that an update of bcache-fs is released and 3 dependencies (could even be transitive) were updated and the newer versions were not packaged in Debian. The maintainer now has one package to update but another 3 to package for the first time. That's additional work. AFAIK Fedora has a tool to automatize this work, but maybe Debian has additional requirements that make this harder.
  - 2y ago

rlpb 2y ago

Maintenance is much more practical if everything in your dependency tree uses the same version of every dependency. Achieving this in practice is much of the work distribution maintainers have to do. Doing this has a couple of key advantages: 1. when a security vulnerability needs patching in a stable distribution release, the security team only have to patch one version instead of a dozen or more different versions bundled all over the place 2. a library that depends on dependencies A and B, both of which depend on X, can actually work properly if it needs to pass API objects created by X across between A and B, since X is of the same version In an ecosystem where it's considered acceptable to "pin" versions of dependencies and also call any system that doesn't use the pinned versions "unsupported", both of the above two cases become impractical. Whether you use shared libraries or static libraries, the above matters still exist.

orra 2y ago
> 2. a library that depends on dependencies A and B, both of which depend on X, can actually work properly if it needs to pass API objects created by X across between A and B, since X is of the same version That feels like an advantage for the developer, not the distro maintainer. I find that amusing, because you're saying it's easy if they can use one version of the library. Well, that's also true if they're writing an application, hence pinning.
- rlpb 2y ago
  > That feels like an advantage for the developer, not the distro maintainer. It's an advantage for the ecosystem, yes. Not the developer of the library (X in my example), but a different developer. Someone who is trying to write an app to do something interesting with A and B. > ...because you're saying it's easy if they can use one version of the library. Well, that's also true if they're writing an application I don't think that's true. Perhaps library A has pinned dependencies for X such that it is in conflict with unrelated library B that also uses pinned dependencies for X. Both libraries are now less useful to the ecosystem, because they are now inadvertently incompatible with each other (if their APIs expose X's objects). My example demonstrates why this doesn't work in the general case, and therefore why it's wrong to expect that everyone does it this way.

foobarqux 2y ago

As others have said this isn't just Rust, distro "release" packaging is untenable now for non-base-system packages: there are an ever increasing number of packages, they release with updates users need/want at a rate far faster than the distro release schedule and have too many conflicting dependencies. To deal with this today you install a stable base system with an overlaid "user" package system so Debian/Fedora + nixpkgs/guix/brew/snap/flatpak/mpr/pip/cargo/etc. Unfortunately because because although you can get most packages on nixpkgs you can't get all and you'll need to install multiple so it becomes a nightmare to maintain (how do you remember all the package managers you have to update during the xz vulnerability?) and extremely bloated due to duplicated dependencies (especially for GUI packages that require gnome or kde). You do get pretty far for CLI-only packages by just adding nixpkgs though. Too bad it's so terrible to use. MPR is also pretty interesting: You basically leverage the AUR for Debian and with a few changes you could probably make the dependency names translation automatic and transparent. It solves the bloat problem (since you'll mostly use system packages) but doesn't help with library versions.

ploxiln 2y ago
Shouldn't the filesystem utilities be base-system packages, though? Like e2fsprogs, xfsprogs ... I guess rust just isn't appropriate for base-system packages, because the dependencies move too fast, and so rust components need to depend on different specific versions of all dependencies, and are "horrifically out-of-date" in just a few months ... and rust developers wouldn't stoop so low as to recognize that e.g. bindgen is a particularly hairy and touchy dependency so they should just vendor the bindings output by bindgen, rather than the entire bindgen tool plus all other dependencies in full ...
- foobarqux 2y ago
  I'm not that familiar with bcachefs-tools and was speaking more generally but I thought bcachefs-tools wasn't critical for normal booting (On Ubuntu 24.04 nothing seems to depend on that package and it seems optional. The same for btrfs-tools). Some other comments suggested you need it to mount a degraded file system but it seems to me that is a case where either 1. your system can't boot and you need external recovery boot tools anyway or 2. you can boot and can run whatever tools you want (you can install the latest version of bcachefs-tools via nixpkgs for example). The real problem there seems to be that the program was silently broken due to the changes made to get it to build on Debian. Though I agree that it is closer to a system-critical package than most packages.

BoingBoomTschak 2y ago

Gentoo has SLOTs and they work well. But for Go and Rust, I think they chose the vendoring path because it's just too damn Don Quixotesque to try to mirror their NPM tier package repositories.

lambda 2y ago

> Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solutio Can you link to the post on Reddit you are referring to?

dfc 2y ago
I think this is the comment: https://www.reddit.com/r/bcachefs/comments/1f4erbg/comment/lkped4c/ https://www.reddit.com/r/bcachefs/comments/1f4erbg/comment/l...

pornel 2y ago

> rust insists on a very specific version of various dependencies It only insists on semver-compatible versions (if a Rust/Cargo package specifies libfoo=5.1, it will work with libfoo=5.9). It's one per major version, not that different from Debian packaging "libfoo5" and "libfoo7" when both are needed. The difference is that Cargo unifies versions by updating them to the newest compatible, and Debian unifies them by downgrading them to old unsupported versions, ignores compatibility, and reintroduces bugs, and disables required functionality.