2 ms·

Why are they "rebootstapping"? Or does that mean they're just recompiling all the .debs again, but using the existing environment? (For context I've bootstrap

by rwmj 3y ago

Why are they "rebootstapping"? Or does that mean they're just recompiling all the .debs again, but using the existing environment? (For context I've bootstrapped Fedora on riscv64 from scratch three times, and I wouldn't be keen to ever have to do it again.)

snvzz 3y ago
Debian unofficial ports are built separately. As RISC-V is now official, it needs to be rebuilt with the infrastructure that builds all official architectures. There should be no issues, as the unofficial port is already building over 95% of the Debian package library. Note that Debian is the biggest distribution, so this is no small feat. In a couple weeks, it should all be sorted, with the official port built to the same level the unofficial port was.
- rwmj 3y ago
  Sure, we also build most of Fedora on a separate build system (http://fedora.riscv.rocks/koji/ http://fedora.riscv.rocks/koji/) so it's a pretty similar situation.
  - pabs3 3y ago
    By built separately, they meant unofficial ports aren't built by Debian, but by random machines hosted in random places by members of the port team, who often aren't Debian members yet.
pabs3 3y ago
The rebootstrap is done because unofficial ports build servers usually are not owned or run by the Debian project, but by the porting teams themselves, who could be and often are random people who are not yet Debian members. So we don't necessarily want to use all of those .debs because they might not be rebuildable, or they might have artefacts from whatever hacky bootstrap process the first build used. The bootstrap process is a QA strategy to have more trust in the resulting binary packages. Longer term we want all new ports to be cross-built from an existing architecture, but we don't have cross-buildds yet. The rebootstrap tool is a work-in-progress QA tool for making cross-building bootstrap for new arches a viable strategy. https://wiki.debian.org/HelmutGrohne/rebootstrap https://wiki.debian.org/HelmutGrohne/rebootstrap https://wiki.debian.org/DebianBootstrap https://wiki.debian.org/DebianBootstrap https://crossqa.debian.net/ https://crossqa.debian.net/ https://wiki.debian.org/PortsDocs/New https://wiki.debian.org/PortsDocs/New Even longer term, we should do a bootstrap of all architectures from scratch using the approach of Guix, Bootstrappable Builds and Reproducible Builds; start with ~500 bytes of hand-written machine code plus all the source code and build enough to get a build server up, then go from there. https://bootstrappable.org/ https://bootstrappable.org/ https://reproducible-builds.org/ https://reproducible-builds.org/ https://www.aikidev.net/~vagrant/talks/2023/fossy/Breaking_the_Chains_of_Trusting_Trust.pdf https://www.aikidev.net/~vagrant/talks/2023/fossy/Breaking_t...
dima55 3y ago
"rebootstrap" is the tool used today to bring up a new architecture. My understanding is that most of the process is now automated.
- pabs3 3y ago
  The rebootstrap being referred to here is not Helmut's automated rebootstrap tool, but about the fairly manual process of rebuilding an existing unofficial port as on official port, the steps for this are on the new port docs wiki page: https://wiki.debian.org/PortsDocs/New#Official_port https://wiki.debian.org/PortsDocs/New#Official_port

by rwmj 3y ago

snvzz 3y ago

Debian unofficial ports are built separately. As RISC-V is now official, it needs to be rebuilt with the infrastructure that builds all official architectures. There should be no issues, as the unofficial port is already building over 95% of the Debian package library. Note that Debian is the biggest distribution, so this is no small feat. In a couple weeks, it should all be sorted, with the official port built to the same level the unofficial port was.

rwmj 3y ago
Sure, we also build most of Fedora on a separate build system (http://fedora.riscv.rocks/koji/ http://fedora.riscv.rocks/koji/) so it's a pretty similar situation.
- pabs3 3y ago
  By built separately, they meant unofficial ports aren't built by Debian, but by random machines hosted in random places by members of the port team, who often aren't Debian members yet.

pabs3 3y ago

The rebootstrap is done because unofficial ports build servers usually are not owned or run by the Debian project, but by the porting teams themselves, who could be and often are random people who are not yet Debian members. So we don't necessarily want to use all of those .debs because they might not be rebuildable, or they might have artefacts from whatever hacky bootstrap process the first build used. The bootstrap process is a QA strategy to have more trust in the resulting binary packages. Longer term we want all new ports to be cross-built from an existing architecture, but we don't have cross-buildds yet. The rebootstrap tool is a work-in-progress QA tool for making cross-building bootstrap for new arches a viable strategy. https://wiki.debian.org/HelmutGrohne/rebootstrap https://wiki.debian.org/HelmutGrohne/rebootstrap https://wiki.debian.org/DebianBootstrap https://wiki.debian.org/DebianBootstrap https://crossqa.debian.net/ https://crossqa.debian.net/ https://wiki.debian.org/PortsDocs/New https://wiki.debian.org/PortsDocs/New Even longer term, we should do a bootstrap of all architectures from scratch using the approach of Guix, Bootstrappable Builds and Reproducible Builds; start with ~500 bytes of hand-written machine code plus all the source code and build enough to get a build server up, then go from there. https://bootstrappable.org/ https://bootstrappable.org/ https://reproducible-builds.org/ https://reproducible-builds.org/ https://www.aikidev.net/~vagrant/talks/2023/fossy/Breaking_the_Chains_of_Trusting_Trust.pdf https://www.aikidev.net/~vagrant/talks/2023/fossy/Breaking_t...

dima55 3y ago

"rebootstrap" is the tool used today to bring up a new architecture. My understanding is that most of the process is now automated.

pabs3 3y ago
The rebootstrap being referred to here is not Helmut's automated rebootstrap tool, but about the fairly manual process of rebuilding an existing unofficial port as on official port, the steps for this are on the new port docs wiki page: https://wiki.debian.org/PortsDocs/New#Official_port https://wiki.debian.org/PortsDocs/New#Official_port