4 ms·

I would rather something not be packaged at all than packaged badly; this whole experience reads like a lesson in what not to do. What the author left out in h

I would rather something not be packaged at all than packaged badly; this whole experience reads like a lesson in what not to do. What the author left out in his blog post is that I specifically explained why this was going to be an issue and what was going to happen when he and I first talked about Debian packaging; and why for a package that needs to work correctly for the system to boot (fsck, mount helper, encryption unlock), playing these kinds of games with swapping dependencies out was not a good idea. But with Debian and Fedora people alike it's been like talking to a brick wall. "No, your concerns don't matter, this is our policy and we're going to do what we want with your code". They've made it very clear that I have no voice or say in how they package it. So then he breaks the build by switching to the packaged version of bindgen, an old unsupported version. And he makes no attempt to debug, doesn't even send me the build error until months later. As a result, Debian users were stuck on a broken version of -tools that wasn't passing mount options correctly, which meant that when a drive died and they needed to mount in degraded mode, they weren't able to. This happened to multiple people - that bothered to report it. This was quickly fixed upstream, but as far as I know it's still broken in Debian; and Jonathan Carter wasn't fielding any of those bug reports, I was. I'm tired, frustrated and demoralized by this drama and tired of constantly having to respond to it. This has taken up an enormous amount of my time. I really wish this guy hadn't packaged -tools for Debian in the first place. I never asked for it. I have enough on my plate to do without a bunch of distro drama to deal with as well.

haileys 2y ago
Thanks for taking the time to comment here! > So then he breaks the build by switching to the packaged version of bindgen, an old unsupported version. Could bcachefs-tools solve this problem upstream of distros by releasing a source tarball with the bindings already generated, so that you can use whatever version of bindgen you want, and bindgen is not needed at all when building from this release tarball? This would be similar in nature to traditional autotools approach where some amount of pre-compilation is done to produce a release tarball from what actually lives in the source repository. It's also what I do for some of my ffi crates. I have a script in the root of the repo which regenerates the bindings, but when published to crates.io, the bindings are already generated and bindgen is not part of the build downstream of my crate.
- koverstreet 2y ago
  I already provide tarballs with vendored dependencies - I did this for Fedora, so that package builds wouldn't have to download anything. I'm not going to special case the bindgen bindings because this really isn't a bindgen specific issue - this could just as easily been any other dependency.
  - haileys 2y ago
    Speaking from practical experience, bindgen is a particularly heavyweight dependency. It requires libclang to be available and also adds a considerable amount of time to the compile. If it makes compiling the software easier - and end users compiling the software themselves would benefit here as well as distro maintainers - then it would seem like a pragmatic move to me.
    - koverstreet 2y ago
      This really isn't about bindgen. Any time a dependency is changed, the package version should be bumped and retested. This is a massive improvement in reproducibility, bisectability and our ability to QA. Before this model, bisecting breakage that was a result of a change in library 'a' breaking something in consumer 'b' was effectively impossible; now it is. What Debian and Fedora have been doing is a big step backwards, this whole 'unbundling of dependencies' (that were going to be statically linked anyways) needs to die.
      - brnt 2y ago
        > What Debian and Fedora have been doing is a big step backwards, this whole 'unbundling of dependencies' (that were going to be statically linked anyways) needs to die. They, and most others, have been doing that since their respectively beginnings, and for good reasons: everybody expects distros to fix security issues, which is greatly aided by ensuring an-as-small-as-necesary dep tree and thus a single version of any particular dep. Changing that isn't impossible, but I am not aware of any distro that's managing this and be widely considered 'stable'. How can distros change while keeping up their end which is stability and robustness in the face of bugs and security issues?
smashed 2y ago
> I would rather something not be packaged at all than packaged badly; this whole experience reads like a lesson in what not to do. You might consider adding a big warning on your official documentation about unsupported distribution packages. Add links to relevant issue tracker/bug reports or mailing list discussions saying until So and so issue are resolved, the official statement is that distribution package is unsupported, not recommended and deemed dangerous to use. This is as much leverage as you can have with the distribution community. Then wait for them to upstream patches and attempt to fix the issues. Accept fixes as you consider appropriate or not. It's also important to at least respect the distribution's opinions in regards to having only one version of a library's major release . Just respect and understanding, you don't have to agree. Also, all the problematic libs cited by the packager are 0.x.x numbered which sounds like a very young and immature ecosystem of dependencies. Of course, this is bound to cause pain to packagers. I think this speaks volumes about the high level of interest for bcachefs, that they actually tried to make it work instead of not bothering.
- mahkoh 2y ago
  0.2 and 0.4 are different "major releases" of rust crates as you say. The major release is determined by the first non-0 component in the version number. The issue is that debian appears to only allow one version even if there are multiple major versions. If debian is fine with packaging versions 2.0 and 4.0 but not 0.2 and 0.4, then debian does not understand rust version numbers.
  - andrewshadura 2y ago
    Debian allows this. However, it's extra work for package maintainers and extra headache on the security front.
- koverstreet 2y ago
  nah, I'm not taking the fight to Jörg Schilling levels :)
eqvinox 2y ago
> this whole experience reads like a lesson in what not to do. What lessons have you drawn from it?

haileys 2y ago

Thanks for taking the time to comment here! > So then he breaks the build by switching to the packaged version of bindgen, an old unsupported version. Could bcachefs-tools solve this problem upstream of distros by releasing a source tarball with the bindings already generated, so that you can use whatever version of bindgen you want, and bindgen is not needed at all when building from this release tarball? This would be similar in nature to traditional autotools approach where some amount of pre-compilation is done to produce a release tarball from what actually lives in the source repository. It's also what I do for some of my ffi crates. I have a script in the root of the repo which regenerates the bindings, but when published to crates.io, the bindings are already generated and bindgen is not part of the build downstream of my crate.

koverstreet 2y ago
I already provide tarballs with vendored dependencies - I did this for Fedora, so that package builds wouldn't have to download anything. I'm not going to special case the bindgen bindings because this really isn't a bindgen specific issue - this could just as easily been any other dependency.
- haileys 2y ago
  Speaking from practical experience, bindgen is a particularly heavyweight dependency. It requires libclang to be available and also adds a considerable amount of time to the compile. If it makes compiling the software easier - and end users compiling the software themselves would benefit here as well as distro maintainers - then it would seem like a pragmatic move to me.
  - koverstreet 2y ago
    This really isn't about bindgen. Any time a dependency is changed, the package version should be bumped and retested. This is a massive improvement in reproducibility, bisectability and our ability to QA. Before this model, bisecting breakage that was a result of a change in library 'a' breaking something in consumer 'b' was effectively impossible; now it is. What Debian and Fedora have been doing is a big step backwards, this whole 'unbundling of dependencies' (that were going to be statically linked anyways) needs to die.
    - brnt 2y ago
      > What Debian and Fedora have been doing is a big step backwards, this whole 'unbundling of dependencies' (that were going to be statically linked anyways) needs to die. They, and most others, have been doing that since their respectively beginnings, and for good reasons: everybody expects distros to fix security issues, which is greatly aided by ensuring an-as-small-as-necesary dep tree and thus a single version of any particular dep. Changing that isn't impossible, but I am not aware of any distro that's managing this and be widely considered 'stable'. How can distros change while keeping up their end which is stability and robustness in the face of bugs and security issues?

smashed 2y ago

> I would rather something not be packaged at all than packaged badly; this whole experience reads like a lesson in what not to do. You might consider adding a big warning on your official documentation about unsupported distribution packages. Add links to relevant issue tracker/bug reports or mailing list discussions saying until So and so issue are resolved, the official statement is that distribution package is unsupported, not recommended and deemed dangerous to use. This is as much leverage as you can have with the distribution community. Then wait for them to upstream patches and attempt to fix the issues. Accept fixes as you consider appropriate or not. It's also important to at least respect the distribution's opinions in regards to having only one version of a library's major release . Just respect and understanding, you don't have to agree. Also, all the problematic libs cited by the packager are 0.x.x numbered which sounds like a very young and immature ecosystem of dependencies. Of course, this is bound to cause pain to packagers. I think this speaks volumes about the high level of interest for bcachefs, that they actually tried to make it work instead of not bothering.

mahkoh 2y ago
0.2 and 0.4 are different "major releases" of rust crates as you say. The major release is determined by the first non-0 component in the version number. The issue is that debian appears to only allow one version even if there are multiple major versions. If debian is fine with packaging versions 2.0 and 4.0 but not 0.2 and 0.4, then debian does not understand rust version numbers.
- andrewshadura 2y ago
  Debian allows this. However, it's extra work for package maintainers and extra headache on the security front.
koverstreet 2y ago
nah, I'm not taking the fight to Jörg Schilling levels :)

eqvinox 2y ago

> this whole experience reads like a lesson in what not to do. What lessons have you drawn from it?