7 ms·

mrustc: In-progress alternative Rust compiler (to C)

80 points by my123 6y ago

coolreader18 6y ago
Oh, nice! I didn't realize that development was still happening for that, I thought it had stopped around Rust 1.20. Glad to see that it's still being developed, stuff like this will definitely be important for bootstrapping/"trusting trust".
xiphias2 6y ago
It’s strange for me why anybody would reimplement a Rust compiler in C++ instead of Rust, but people are free to do what they want :)
- monocasa 6y ago
  Part of the point is allowing bootstrapping a rust toolchain from source on systems that don't have rust yet.
  - dataflow 6y ago
    Can't you just cross compile? Wouldn't you have to do that anyway for some parts of the system? What's the point of bootstrapping?
    - monocasa 6y ago
      When I say "systems", I should have been a little more specific. There's a bunch of corporate/.gov envs that don't allow binaries to enter their system. All new code has to be compiled from source, on the target system (or at least on that side of the process firewall, it might be an airgapped cluster that can share code). They have C and C++ compilers that they've been compiling from source since the dawn of time, but they don't have a rust compiler. This gives them a mechanism to begin using rust within their process.
      - dataflow 6y ago
        Wow I see. Could they not use the initial Rust compilers (and however those were bootstrapped? I have no idea how) to get a Rust compiler going? I'm guessing it's just too painful?
        spijdar 6y ago
        Bootstrapping rust is really, really painful as is. The rust compiler itself tends to require a recent version of the rust compiler to work. So to bootstrap from the original versions of rustc (which were written in OCaml) you'd have to incrementally step up really slowly. Being able to target a pretty recent version of Rust with a compiler written in C would be so, so useful for these purposes.
        Blikkentrekker 6y ago
        Certainly that could be automated quite easily. I would assume that on such compilation farms that these systems generally use, this could be done very quickly. Perhaps there would be an interest for the Rustc team to provide a recursive automated setup that is capable of compiling the latest Rustc from OCaml, and finally from C as well since one must follow a similar process with OCaml.
        LegionMammal978 6y ago
        The automation for this is actually trickier than it would first appear: especially with the earlier snapshots, a number of patches are needed to compile the LLVM interfaces written in C++. Also, due to linking issues, some snapshots require a stage2 compiler, while others can only use a stage1 compiler. I agree that it would be trivial once you have the chain in place, but that chain can only be built through trial and error.
        Blikkentrekker 6y ago
        And no one has done this before? Methinks that building this chain by trial and error is rather trivial compared to the actual work in building a compiler.
    - dan-robertson 6y ago
      One way to trust your computer system more is to have very few binary dependencies. Ie the goal is to have some source files and hopefully if you start with (eg) a trusted C compiler and linker you can work your way up to the whole system with deterministic results. This is supposed to be a way to prove that the compiler wasn’t doing anything funky like injecting some vulnerability into the code it compiled and injecting that vulnerability injection into any compiler it compiled.
  - rowanG077 6y ago
    I thought this was already possible. The first Rust compiler is written in OCaml. It should be possible to do bootstrapping with that.
    - axelf4 6y ago
      Of course, but AFAIU this massively shortens the chain. Newer versions of rustc up the minimum compiler version required for building all the time.
    - faho 6y ago
      Unfortunately rustc uses rust features almost immediately (I think even features introduced in the immediately preceding version?), so you get a bootstrap chain, and if you start with the compiler written in OCaml you now need to build every single rustc version ever released. This takes very long and is error-prone, so having a compiler that can build even an intermediate version is already a big help. See e.g. https://guix.gnu.org/blog/2018/bootstrapping-rust/ https://guix.gnu.org/blog/2018/bootstrapping-rust/
      - kiwidrew 6y ago
        It's unfortunate that Rust has adopted development policies that are so hostile towards bootstrapping and cross-compiling rustc. The fact that most of the people cheerleading for Rust the ecosystem don't even see this as a problem is enough to convince me to steer clear of anything involving Rust the language. You don't have a real language until there is a stable specification and multiple independent implementations. Until then it's just an experimental toy.
        monocasa 6y ago
        You can't build GCC on MSVC or vis versa. And people do see the pain here as a problem, that's why the project in this very thread exists. I think most in the Rust community see mrustc and the pressure that an independent implementation brings process wise as a very good thing for the language.
- pjmlp 6y ago
  You mean just like LLVM?
- coldtea 6y ago
  Strange? The whole point of this is easy bootstrapping, so it's anything but strange, it's the main point...
lmkg 6y ago
The interesting thing to me is that this does not fully valid Rust code, e.g. not checking for borrow safety. Which would normally make no damn sense for a Rust compiler, but actually makes sense for what this particular project is trying to achieve. This is not a compiler for developing Rust programs. This is a compiler to let platforms unsupported by Rust/LLVM build rust programs from source. So it takes a Rust program which already exists, assumes it can be compiled by rustc, and translates it to C so it can be handed off to your local compiler. Skipping the burrow checker probably lets this execute in environments which are more constrained than what rustc can fit into. Of course there's a question of how much of Rust's safety survive translation into C. Certainly some. E.g. the emitted program would include bounds checks where not specifically elided in Rust, and references are statically guaranteed to be non-null before being lowered into pointers. But there could be gaps, especially in cases where Rust semantics mismatch with C in ways that LLVM hasn't exposed yet.
- rmdashrfstar 6y ago
  > But there could be gaps, especially in cases where Rust semantics mismatch with C in ways that LLVM hasn't exposed yet. Could you elaborate on what potential mismatches there currently are between the semantics of Rust and C? Where particularly do you think there might be issues/dragons hiding in the semantics of a translation?
  - deleted 6y ago
    [deleted]
  - gpm 6y ago
    Integer overflows (signed overflows defined to wrap or panic in rust, are undefined behavior in C). Float integer conversions (overflows are defined to saturate in rust, are undefined behavior in C). Maybe things like how pointer casts are treated in correct (but unsafe) rust vs C. Or similarly in transmute. Generally it wouldn't surprise me if rust was standardizing a memory model that was subtly different from C. I'm sure the list goes on, but "that sort of thing".
    - 6y ago
xucheng 6y ago
There is also a WIP rust gcc frontend: https://github.com/Rust-GCC/gccrs https://github.com/Rust-GCC/gccrs I wonder how these two projects compare.

80 points by my123 6y ago

Oh, nice! I didn't realize that development was still happening for that, I thought it had stopped around Rust 1.20. Glad to see that it's still being developed, stuff like this will definitely be important for bootstrapping/"trusting trust".

xiphias2 6y ago

It’s strange for me why anybody would reimplement a Rust compiler in C++ instead of Rust, but people are free to do what they want :)

monocasa 6y ago
Part of the point is allowing bootstrapping a rust toolchain from source on systems that don't have rust yet.
- dataflow 6y ago
  Can't you just cross compile? Wouldn't you have to do that anyway for some parts of the system? What's the point of bootstrapping?
  - monocasa 6y ago
    When I say "systems", I should have been a little more specific. There's a bunch of corporate/.gov envs that don't allow binaries to enter their system. All new code has to be compiled from source, on the target system (or at least on that side of the process firewall, it might be an airgapped cluster that can share code). They have C and C++ compilers that they've been compiling from source since the dawn of time, but they don't have a rust compiler. This gives them a mechanism to begin using rust within their process.
    - dataflow 6y ago
      Wow I see. Could they not use the initial Rust compilers (and however those were bootstrapped? I have no idea how) to get a Rust compiler going? I'm guessing it's just too painful?
      - spijdar 6y ago
        Bootstrapping rust is really, really painful as is. The rust compiler itself tends to require a recent version of the rust compiler to work. So to bootstrap from the original versions of rustc (which were written in OCaml) you'd have to incrementally step up really slowly. Being able to target a pretty recent version of Rust with a compiler written in C would be so, so useful for these purposes.
        Blikkentrekker 6y ago
        Certainly that could be automated quite easily. I would assume that on such compilation farms that these systems generally use, this could be done very quickly. Perhaps there would be an interest for the Rustc team to provide a recursive automated setup that is capable of compiling the latest Rustc from OCaml, and finally from C as well since one must follow a similar process with OCaml.
        LegionMammal978 6y ago
        The automation for this is actually trickier than it would first appear: especially with the earlier snapshots, a number of patches are needed to compile the LLVM interfaces written in C++. Also, due to linking issues, some snapshots require a stage2 compiler, while others can only use a stage1 compiler. I agree that it would be trivial once you have the chain in place, but that chain can only be built through trial and error.
        Blikkentrekker 6y ago
        And no one has done this before? Methinks that building this chain by trial and error is rather trivial compared to the actual work in building a compiler.
  - dan-robertson 6y ago
    One way to trust your computer system more is to have very few binary dependencies. Ie the goal is to have some source files and hopefully if you start with (eg) a trusted C compiler and linker you can work your way up to the whole system with deterministic results. This is supposed to be a way to prove that the compiler wasn’t doing anything funky like injecting some vulnerability into the code it compiled and injecting that vulnerability injection into any compiler it compiled.
- rowanG077 6y ago
  I thought this was already possible. The first Rust compiler is written in OCaml. It should be possible to do bootstrapping with that.
  - axelf4 6y ago
    Of course, but AFAIU this massively shortens the chain. Newer versions of rustc up the minimum compiler version required for building all the time.
  - faho 6y ago
    Unfortunately rustc uses rust features almost immediately (I think even features introduced in the immediately preceding version?), so you get a bootstrap chain, and if you start with the compiler written in OCaml you now need to build every single rustc version ever released. This takes very long and is error-prone, so having a compiler that can build even an intermediate version is already a big help. See e.g. https://guix.gnu.org/blog/2018/bootstrapping-rust/ https://guix.gnu.org/blog/2018/bootstrapping-rust/
    - kiwidrew 6y ago
      It's unfortunate that Rust has adopted development policies that are so hostile towards bootstrapping and cross-compiling rustc. The fact that most of the people cheerleading for Rust the ecosystem don't even see this as a problem is enough to convince me to steer clear of anything involving Rust the language. You don't have a real language until there is a stable specification and multiple independent implementations. Until then it's just an experimental toy.
      - monocasa 6y ago
        You can't build GCC on MSVC or vis versa. And people do see the pain here as a problem, that's why the project in this very thread exists. I think most in the Rust community see mrustc and the pressure that an independent implementation brings process wise as a very good thing for the language.
pjmlp 6y ago
You mean just like LLVM?
coldtea 6y ago
Strange? The whole point of this is easy bootstrapping, so it's anything but strange, it's the main point...

lmkg 6y ago

The interesting thing to me is that this does not fully valid Rust code, e.g. not checking for borrow safety. Which would normally make no damn sense for a Rust compiler, but actually makes sense for what this particular project is trying to achieve. This is not a compiler for developing Rust programs. This is a compiler to let platforms unsupported by Rust/LLVM build rust programs from source. So it takes a Rust program which already exists, assumes it can be compiled by rustc, and translates it to C so it can be handed off to your local compiler. Skipping the burrow checker probably lets this execute in environments which are more constrained than what rustc can fit into. Of course there's a question of how much of Rust's safety survive translation into C. Certainly some. E.g. the emitted program would include bounds checks where not specifically elided in Rust, and references are statically guaranteed to be non-null before being lowered into pointers. But there could be gaps, especially in cases where Rust semantics mismatch with C in ways that LLVM hasn't exposed yet.

rmdashrfstar 6y ago
> But there could be gaps, especially in cases where Rust semantics mismatch with C in ways that LLVM hasn't exposed yet. Could you elaborate on what potential mismatches there currently are between the semantics of Rust and C? Where particularly do you think there might be issues/dragons hiding in the semantics of a translation?
- deleted 6y ago
  [deleted]
- gpm 6y ago
  Integer overflows (signed overflows defined to wrap or panic in rust, are undefined behavior in C). Float integer conversions (overflows are defined to saturate in rust, are undefined behavior in C). Maybe things like how pointer casts are treated in correct (but unsafe) rust vs C. Or similarly in transmute. Generally it wouldn't surprise me if rust was standardizing a memory model that was subtly different from C. I'm sure the list goes on, but "that sort of thing".
  - 6y ago

xucheng 6y ago

There is also a WIP rust gcc frontend: https://github.com/Rust-GCC/gccrs https://github.com/Rust-GCC/gccrs I wonder how these two projects compare.