4 ms·

What's the problem with focusing on upstreaming the patches and getting rid of rust's "staging fork"? Especially if they fix something as serious as miscompilat

What's the problem with focusing on upstreaming the patches and getting rid of rust's "staging fork"? Especially if they fix something as serious as miscompilations, wouldn't those patches be VERY much needed upstream? I'm asking out of genuine interest.

Rusky 6y ago
That's already done. The point is that it takes time, and there are always more of them, so at any point in time there are likely to be a handful of them still in-process of moving upstream.
steveklabnik 6y ago
Quoting from the documentation I linked: > Strongly prefer to upstream all patches to LLVM before including them in rustc. That is, this is already the case. We don't like maintaining a fork. We try to upstream as much as we can. But, at the same time, even when you do this, it takes tons of time. A contributor was talking about exactly this on Twitter earlier today, and estimated that, even if the patch was written and accepted as fast as possible, it would still take roughly a year for that patch to make it into the release used by Rust. This is the opposite side of the whole "move slow and only use old versions of things" tradeoff: that would take even longer to get into, say, the LLVM in Debian stable, as suggested in another comment chain. So our approach is, upstream the patches, keep them in our fork, and then remove them when they inevitably make it back downstream. (EDIT: Also, what Rusky said: there's basically always going to be more than one patch, in various stages of this process...)
- archi42 6y ago
  Ok, maybe I'm just to spoiled by my previous experience then. I never had any problems getting patches upstreamed relatively quickly, but then e.g. meson or picolibc are much smaller than LLVM and patches are simpler to reason about (I know some compiler construction). Thanks for the answer.
  - steveklabnik 6y ago
    Remember that in my comment above, it’s assuming a speedy review. Release schedules are still a thing; I’m not at my computer anymore, but IIRC, llvm releases once or twice a year, so that’s the inherent limit of making it into a release, not necessarily the review time.
- cycloptic 6y ago
  Can't there be a build option to not use the LLVM submodule, and instead use the system LLVM? Assuming there are tests for these LLVM bugs, and assuming the patches are indeed being merged, wouldn't a CI be able to catch when it is safe for downstream users that want to use upstream LLVM to update their Rust installation?
  - steveklabnik 6y ago
    There is. We even test that it builds as part of our CI, to make sure it works, IIRC.
    - cycloptic 6y ago
      Hmm, I couldn't find that last time I looked at building Rust -- all I saw was an option to grab precompiled builds of Rust LLVM from the CI server, but maybe I missed something.
      - steveklabnik 6y ago
        https://rustc-dev-guide.rust-lang.org/getting-started.html#configuring-the-compiler https://rustc-dev-guide.rust-lang.org/getting-started.html#c... links to https://rustc-dev-guide.rust-lang.org/building/suggested.html#skipping-llvm-build https://rustc-dev-guide.rust-lang.org/building/suggested.htm... ... which is broken. I'll file a docs bug. Can't do it right this moment, but will in a day or two. It is not my area of expertise, but I believe the way you do it is to set this option: https://github.com/rust-lang/rust/blob/master/config.toml.example#L596-L599 https://github.com/rust-lang/rust/blob/master/config.toml.ex... buuut the dockerfile for the llvm 9 build passes this flag as an argument: https://github.com/rust-lang/rust/blob/5a549d36ee81b226d16724240b88f0137f8a36af/src/ci/docker/host-x86_64/x86_64-gnu-llvm-9/Dockerfile#L31 https://github.com/rust-lang/rust/blob/5a549d36ee81b226d1672...
  - drran 6y ago
    Rust in Fedora is compiled with system LLVM (libLLVM-11.so). Enjoy.

Rusky 6y ago

That's already done. The point is that it takes time, and there are always more of them, so at any point in time there are likely to be a handful of them still in-process of moving upstream.

steveklabnik 6y ago

Quoting from the documentation I linked: > Strongly prefer to upstream all patches to LLVM before including them in rustc. That is, this is already the case. We don't like maintaining a fork. We try to upstream as much as we can. But, at the same time, even when you do this, it takes tons of time. A contributor was talking about exactly this on Twitter earlier today, and estimated that, even if the patch was written and accepted as fast as possible, it would still take roughly a year for that patch to make it into the release used by Rust. This is the opposite side of the whole "move slow and only use old versions of things" tradeoff: that would take even longer to get into, say, the LLVM in Debian stable, as suggested in another comment chain. So our approach is, upstream the patches, keep them in our fork, and then remove them when they inevitably make it back downstream. (EDIT: Also, what Rusky said: there's basically always going to be more than one patch, in various stages of this process...)

archi42 6y ago
Ok, maybe I'm just to spoiled by my previous experience then. I never had any problems getting patches upstreamed relatively quickly, but then e.g. meson or picolibc are much smaller than LLVM and patches are simpler to reason about (I know some compiler construction). Thanks for the answer.
- steveklabnik 6y ago
  Remember that in my comment above, it’s assuming a speedy review. Release schedules are still a thing; I’m not at my computer anymore, but IIRC, llvm releases once or twice a year, so that’s the inherent limit of making it into a release, not necessarily the review time.
cycloptic 6y ago
Can't there be a build option to not use the LLVM submodule, and instead use the system LLVM? Assuming there are tests for these LLVM bugs, and assuming the patches are indeed being merged, wouldn't a CI be able to catch when it is safe for downstream users that want to use upstream LLVM to update their Rust installation?
- steveklabnik 6y ago
  There is. We even test that it builds as part of our CI, to make sure it works, IIRC.
  - cycloptic 6y ago
    Hmm, I couldn't find that last time I looked at building Rust -- all I saw was an option to grab precompiled builds of Rust LLVM from the CI server, but maybe I missed something.
    - steveklabnik 6y ago
      https://rustc-dev-guide.rust-lang.org/getting-started.html#configuring-the-compiler https://rustc-dev-guide.rust-lang.org/getting-started.html#c... links to https://rustc-dev-guide.rust-lang.org/building/suggested.html#skipping-llvm-build https://rustc-dev-guide.rust-lang.org/building/suggested.htm... ... which is broken. I'll file a docs bug. Can't do it right this moment, but will in a day or two. It is not my area of expertise, but I believe the way you do it is to set this option: https://github.com/rust-lang/rust/blob/master/config.toml.example#L596-L599 https://github.com/rust-lang/rust/blob/master/config.toml.ex... buuut the dockerfile for the llvm 9 build passes this flag as an argument: https://github.com/rust-lang/rust/blob/5a549d36ee81b226d16724240b88f0137f8a36af/src/ci/docker/host-x86_64/x86_64-gnu-llvm-9/Dockerfile#L31 https://github.com/rust-lang/rust/blob/5a549d36ee81b226d1672...
- drran 6y ago
  Rust in Fedora is compiled with system LLVM (libLLVM-11.so). Enjoy.