4 ms·

Rust in the NetBSD Kernel, and other odd decisions

37 points by jaypatelani 8mo ago

nextaccountic 8mo ago
> In general, the bootstrap relies on a binary package of the previous version. This is unacceptable for an otherwise source-only, self-contained distribution like the NetBSD sources. This does not paint the full picture. Rust can be bootstrapped with mrustc, which is written in C++ https://github.com/thepowersgang/mrustc https://github.com/thepowersgang/mrustc Now, mrustc supports only Rust 1.74. To build Rust 1.92, you need almost 20 builds. But this can be done from source Guix has written about bootstrapping Rust from source (they care a lot about this). Here is how it looked like in 2018 https://guix.gnu.org/nb-NO/blog/2018/bootstrapping-rust/ https://guix.gnu.org/nb-NO/blog/2018/bootstrapping-rust/
- stockerta 8mo ago
  And this is good because? Seriously, modern devs have lost their minds if they think that this state of affairs is okay.
  - BrouteMinou 8mo ago
    I laughed reading the comment above yours. I also laughed reading yours, you are so right on. Next, I am expecting someone telling that there is a javascript on github to automate that build process... This is not what I was expecting computer science to become, 30 years ago...
    - stockerta 8mo ago
      And that JS project needs only 1Gb of node.js modules to run. Aren't computers fun? /s
  - anthk 8mo ago
    Go need a smaller chainloading process from C/C++ too. But the good thing it's there are prebuilt self-contained binary ports from every OS to almost any OS so you can bootstrap it with very little.
    - Borg3 8mo ago
      O really? Please.. can I bootstrap it here? CYGWIN_NT-5.2 1.3.22(0.78/3/2)
      - anthk 8mo ago
        There are some Go backports to XP. https://github.com/BieHDC/go-backports https://github.com/BieHDC/go-backports
      - LeFantome 8mo ago
        https://github.com/felixmaker/thunk https://github.com/felixmaker/thunk
  - rcxdude 8mo ago
    For a good while this process was just straight up not possible for C and C++, until someone put in a lot of effort to recreate the bootstrap process for reasons of reproducible builds (and it's still a long and complicated process I think only a few people have done). For decades people were just building from source with compilers that had extremely long and undocumented bootstrap chains (they may have documentation for how to bootstrap from a different C or C++ compiler, but there wasn't one that would start from scratch).
Surac 8mo ago
if it can't be build with Ansi C or perhaps old style C++ it is not self contained i think. depending a Language on some fast moving binarys is in my eye a bad decission. Excuse my english. im not a native speaker.
- bandrami 8mo ago
  There are self-hosted Forths
  - anthk 8mo ago
    Subleq and Eforth can self-compile, yes. Being base.dec a subleq EForth image and eforth.fth the source code, to create a new one: ./subleq base.dec < tuned_eforth.fth > new.dec And Uxn partially. You can run the compiler as a UXN rom (Drimflim.rom). So by just having a recent working VM you can compile the rest with uxn2 or 'uxncli dribflim.rom app.tal app.rom'. Tal files are source files.
- rcxdude 8mo ago
  Why are C and C++ special in this regard? I could just as well claim it's not self-contained unless you can start from source and a snippet of inspectable machine code, like Guix can (which still has fairly significant gaps in what you could call 'completely self contained', if you wanted to poke holes in it).
  - ronsor 8mo ago
    C is special because it's possible to relatively easily implement a good-enough C compiler for bootstrapping if needed. It would be under 15,000 lines of code in most languages. C++ certainly isn't that special. It's a pain to implement, but so is Rust.
    - pjmlp 8mo ago
      And how did you got the binary for that language? This argument can be pushed all the way down to build your own CPUs from sand, get to build the first hexdump monitor with electric switches, followed by writting the first Assembler in hexdump, and only then the first compiler for one of thoses languages in Assembly, and finally you get C.
akagusu 8mo ago
We need memory safety but Rust is not the answer. It has no formal spec, changes too fast, depends on third party libraries that change faster than I can breath, and is controlled by a foundation that is controlled by big tech corps. What could go wrong?
- pjmlp 8mo ago
  It has a partial spec. https://ferrous-systems.com/blog/ferrocene-25-11-0/ https://ferrous-systems.com/blog/ferrocene-25-11-0/ Lets not forget not having a formal spec apparently wasn't an issue for C, which only got standardized in 1989, and even K&R C only specified a subset of its behaviours, which is a reason why there is so much UB, and implementation specific behaviours with YOLO C, as the Fil-C author likes to call it.
  - snovymgodym 8mo ago
    > Lets not forget not having a formal spec apparently wasn't an issue for C C emerged in the 1970s when there weren't many other options, and it was rapidly propagated because US antitrust laws forced AT&T to inexpensively license Unix (and by extension C) to universities and research institutions. That was half a century ago. I am a supporter of Rust adoption, but the line of reasoning that "C didn't need X, so Rust shouldn't either" is fundamentally flawed. The time difference between today and C's public emergence is the same as the time between C's emergence and the 1920s. The field of computing is radically different today and any language that wishes to see wider adoption now must play by a completely different set of rules than those of 50 years ago.
    - pjmlp 8mo ago
      There have been options since JOVIAL in 1958, as there have been OS written in high level language a decade before C came to be invented. C has to thank its adoption by being freely available with UNIX, that AT&T was forbidden to take commercial advantage of, for many years, the only reference implementation was whatever cc does.
tancop 8mo ago
this is the right way to do it. just explain why you think rust is not a good match for your project right now. no politics and no hate for the language.
LeFantome 8mo ago
> There are many architectures that NetBSD supports where Rust is not available Are there many architectures that NetBSD supports where GCC is not available? The rustc_codegen_gcc project allows using the rustc compiler to target any architecture that GCC supports. I think it is pretty far along. https://github.com/rust-lang/rustc_codegen_gcc https://github.com/rust-lang/rustc_codegen_gcc > getting a Rust compiler running in the first place is hard > bootstrap relies on a binary package > the compiler would also have to be part of the base system The gccrs project aims to make Rust a fully supported language in GCC. It is fully self-hosting and can be built from source. Once accepted, it will be one of the languages you get when you build GCC. https://github.com/Rust-GCC/gccrs https://github.com/Rust-GCC/gccrs When gccrs is ready, it will address all the above objections. Linux faces the same issues. Today, Rust is optional in Linux and can be skipped on any platform not supported in LLVM and rustc. For Rust to become mandatory in Linux, gccrs is going to be required (for some platforms). > the release cycles of Rust are not compatible with the NetBSD ones I do not fully grasp the objection here. Obviously NetBSD would not be forced to use any Rust features they cannot compile. And using gccrs for the NetBSD kernel would not prevent them from bundling LLVM and rustc as well if they want. Anyway, I am not advocating the use of Rust in NetBSD. I am merely pointing out that solutions to the issues raised are being worked on.
- LeFantome 8mo ago
  I just noticed that rustc_codegen_gcc has been accepted into the mainline Rust compiler, or at least the nightly builds. https://github.com/rust-lang/rust/pull/151156 https://github.com/rust-lang/rust/pull/151156 This means that you will be able to use either GCC or LLVM to build your Rust code with rustc (the main Rust compiler). That means you can compile your Rust for any platform that GCC supports. This only addresses the first point in the blog post (architecture support) but it is an interesting development.

37 points by jaypatelani 8mo ago

> In general, the bootstrap relies on a binary package of the previous version. This is unacceptable for an otherwise source-only, self-contained distribution like the NetBSD sources. This does not paint the full picture. Rust can be bootstrapped with mrustc, which is written in C++ https://github.com/thepowersgang/mrustc https://github.com/thepowersgang/mrustc Now, mrustc supports only Rust 1.74. To build Rust 1.92, you need almost 20 builds. But this can be done from source Guix has written about bootstrapping Rust from source (they care a lot about this). Here is how it looked like in 2018 https://guix.gnu.org/nb-NO/blog/2018/bootstrapping-rust/ https://guix.gnu.org/nb-NO/blog/2018/bootstrapping-rust/

stockerta 8mo ago
And this is good because? Seriously, modern devs have lost their minds if they think that this state of affairs is okay.
- BrouteMinou 8mo ago
  I laughed reading the comment above yours. I also laughed reading yours, you are so right on. Next, I am expecting someone telling that there is a javascript on github to automate that build process... This is not what I was expecting computer science to become, 30 years ago...
  - stockerta 8mo ago
    And that JS project needs only 1Gb of node.js modules to run. Aren't computers fun? /s
- anthk 8mo ago
  Go need a smaller chainloading process from C/C++ too. But the good thing it's there are prebuilt self-contained binary ports from every OS to almost any OS so you can bootstrap it with very little.
  - Borg3 8mo ago
    O really? Please.. can I bootstrap it here? CYGWIN_NT-5.2 1.3.22(0.78/3/2)
    - anthk 8mo ago
      There are some Go backports to XP. https://github.com/BieHDC/go-backports https://github.com/BieHDC/go-backports
    - LeFantome 8mo ago
      https://github.com/felixmaker/thunk https://github.com/felixmaker/thunk
- rcxdude 8mo ago
  For a good while this process was just straight up not possible for C and C++, until someone put in a lot of effort to recreate the bootstrap process for reasons of reproducible builds (and it's still a long and complicated process I think only a few people have done). For decades people were just building from source with compilers that had extremely long and undocumented bootstrap chains (they may have documentation for how to bootstrap from a different C or C++ compiler, but there wasn't one that would start from scratch).

Surac 8mo ago

if it can't be build with Ansi C or perhaps old style C++ it is not self contained i think. depending a Language on some fast moving binarys is in my eye a bad decission. Excuse my english. im not a native speaker.

bandrami 8mo ago
There are self-hosted Forths
- anthk 8mo ago
  Subleq and Eforth can self-compile, yes. Being base.dec a subleq EForth image and eforth.fth the source code, to create a new one: ./subleq base.dec < tuned_eforth.fth > new.dec And Uxn partially. You can run the compiler as a UXN rom (Drimflim.rom). So by just having a recent working VM you can compile the rest with uxn2 or 'uxncli dribflim.rom app.tal app.rom'. Tal files are source files.
rcxdude 8mo ago
Why are C and C++ special in this regard? I could just as well claim it's not self-contained unless you can start from source and a snippet of inspectable machine code, like Guix can (which still has fairly significant gaps in what you could call 'completely self contained', if you wanted to poke holes in it).
- ronsor 8mo ago
  C is special because it's possible to relatively easily implement a good-enough C compiler for bootstrapping if needed. It would be under 15,000 lines of code in most languages. C++ certainly isn't that special. It's a pain to implement, but so is Rust.
  - pjmlp 8mo ago
    And how did you got the binary for that language? This argument can be pushed all the way down to build your own CPUs from sand, get to build the first hexdump monitor with electric switches, followed by writting the first Assembler in hexdump, and only then the first compiler for one of thoses languages in Assembly, and finally you get C.

akagusu 8mo ago

We need memory safety but Rust is not the answer. It has no formal spec, changes too fast, depends on third party libraries that change faster than I can breath, and is controlled by a foundation that is controlled by big tech corps. What could go wrong?

pjmlp 8mo ago
It has a partial spec. https://ferrous-systems.com/blog/ferrocene-25-11-0/ https://ferrous-systems.com/blog/ferrocene-25-11-0/ Lets not forget not having a formal spec apparently wasn't an issue for C, which only got standardized in 1989, and even K&R C only specified a subset of its behaviours, which is a reason why there is so much UB, and implementation specific behaviours with YOLO C, as the Fil-C author likes to call it.
- snovymgodym 8mo ago
  > Lets not forget not having a formal spec apparently wasn't an issue for C C emerged in the 1970s when there weren't many other options, and it was rapidly propagated because US antitrust laws forced AT&T to inexpensively license Unix (and by extension C) to universities and research institutions. That was half a century ago. I am a supporter of Rust adoption, but the line of reasoning that "C didn't need X, so Rust shouldn't either" is fundamentally flawed. The time difference between today and C's public emergence is the same as the time between C's emergence and the 1920s. The field of computing is radically different today and any language that wishes to see wider adoption now must play by a completely different set of rules than those of 50 years ago.
  - pjmlp 8mo ago
    There have been options since JOVIAL in 1958, as there have been OS written in high level language a decade before C came to be invented. C has to thank its adoption by being freely available with UNIX, that AT&T was forbidden to take commercial advantage of, for many years, the only reference implementation was whatever cc does.

tancop 8mo ago

this is the right way to do it. just explain why you think rust is not a good match for your project right now. no politics and no hate for the language.

LeFantome 8mo ago

> There are many architectures that NetBSD supports where Rust is not available Are there many architectures that NetBSD supports where GCC is not available? The rustc_codegen_gcc project allows using the rustc compiler to target any architecture that GCC supports. I think it is pretty far along. https://github.com/rust-lang/rustc_codegen_gcc https://github.com/rust-lang/rustc_codegen_gcc > getting a Rust compiler running in the first place is hard > bootstrap relies on a binary package > the compiler would also have to be part of the base system The gccrs project aims to make Rust a fully supported language in GCC. It is fully self-hosting and can be built from source. Once accepted, it will be one of the languages you get when you build GCC. https://github.com/Rust-GCC/gccrs https://github.com/Rust-GCC/gccrs When gccrs is ready, it will address all the above objections. Linux faces the same issues. Today, Rust is optional in Linux and can be skipped on any platform not supported in LLVM and rustc. For Rust to become mandatory in Linux, gccrs is going to be required (for some platforms). > the release cycles of Rust are not compatible with the NetBSD ones I do not fully grasp the objection here. Obviously NetBSD would not be forced to use any Rust features they cannot compile. And using gccrs for the NetBSD kernel would not prevent them from bundling LLVM and rustc as well if they want. Anyway, I am not advocating the use of Rust in NetBSD. I am merely pointing out that solutions to the issues raised are being worked on.

LeFantome 8mo ago
I just noticed that rustc_codegen_gcc has been accepted into the mainline Rust compiler, or at least the nightly builds. https://github.com/rust-lang/rust/pull/151156 https://github.com/rust-lang/rust/pull/151156 This means that you will be able to use either GCC or LLVM to build your Rust code with rustc (the main Rust compiler). That means you can compile your Rust for any platform that GCC supports. This only addresses the first point in the blog post (architecture support) but it is an interesting development.