5 ms·

A deeper look into the GCC Rust front-end

34 points by chmaynard 4y ago

b3morales 4y ago
Having two IRs (or even having a 1st-stage IR for each frontend language) sounds like something that a compiler project would want to avoid in the long term. I wonder how the IR that's being added for Rust will end up influencing the "regular" IR.
- steveklabnik 4y ago
  Many many compilers have multiple IRs. It’s very normal. rustc has three already, last time I checked, and that’s not counting llvm-IR. Non-lexical lifetimes was gated on adding an additional IR to rustc, even.
  - MuffinFlavored 4y ago
    > rustc has three already Can you expand on this? What are they? I would've thought rustc outputs only 1 IR?
    - fzzzy 4y ago
      There are different stages of ir which allow for different optimizations.
    - steveklabnik 4y ago
      I haven't been paying close attention lately, but for the last few years, compilation looked something vaguely like this: * AST - abstract syntax tree, created from lexing/parsing * HIR - high level IR * THIR - typed high level IR * MIR - mid-level IR (this is the one that enabled non-lexical lifetimes) * LLVM IR - given to LLVM to generate the final binary. If you want to learn more, the rustc dev guide has a bunch of details https://rustc-dev-guide.rust-lang.org/hir.html https://rustc-dev-guide.rust-lang.org/hir.html > I would've thought rustc outputs only 1 IR? Yeah, I mean these are in stages. One goes to the next goes to the next, and there's only one output, but that doesn't mean there aren't a bunch in the middle. Different IRs are good at different things. MIR is based around the control-flow graph, which is also what non-lexical lifetimes are based on, so it's much easier to implement against MIR than it would have been against HIR.
      - philberty 4y ago
        Thanks @steveklabnik we basically have the same in gccrs: 1. AST 2. HIR 3. THIR (side-table lookups) 4. GCC Generic We basically skip MIR in gccrs. Its pretty sensible to have other IR's, we have many passes in gccrs simplifying things so the graph of what your working with is simpler and simpler each time. I mean in GCC for C++ for example they use GENERIC and add a bunch of custom tree-codes such as LAMBDA_EXPR or TEMPLATE stuff for example then they keep substituting etc and finally as part of handing off to GCC middle-end it triggers the gimplification of all of these custom tree codes. So even the C++ front-end you could argue has two IR's.
        steveklabnik 4y ago
        Good to know, thanks! I’m excited to see gcc-Rama develop, I just haven’t had time to learn how any of it works. Keep up the good work!
        steveklabnik 4y ago
        ...gcc-rs. thanks autocorrect.
  - b3morales 4y ago
    Those are serial, though, as I understand it -- each feeds into the next. The gccrs situation sounds like there are two parallel implementations. Please correct me if I'm wrong; I'm not trying to pretend I am any kind of expert here.
    - steveklabnik 4y ago
      I don’t know anything about how this project is architected, but > It's used for type checking and error verification; once that's done, it can be translated and handed to the GCC mid-layer. Sounds the same serial thing to me.
- pretty_dumm_guy 4y ago
  > Having two IRs (or even having a 1st-stage IR for each frontend language) sounds like something that a compiler project would want to avoid in the long term Can you please share the reason behind why it should be this way ?
  - b3morales 4y ago
    I'm not a compiler engineer, so I'm just sharing my own point of view, but to me it sounds like a heavy maintenance burden. Whenever possible I would want to avoid having multiple versions of an important subsystem in any large program that I worked on.
    - philberty 4y ago
      This is true, I tried to make gccrs only have the AST and go from that stright to GCC GENERIC. This had a lot of problems for Rust in my opinion. Many things in Rust are syntactic sugar and can be handled by desugaring the AST into another IR so you dont even have to think about it in other passes. The main issue for me is how complicate the type inference is. So if i wanted to use GCC GENERIC for type resolution for this example: ``` let a; a = 123; let b:u32 = 1; a += b; ``` How do you resolve the type of 'a' you must use inference variables and then update the TREE_TYPE as you go so this means walking the tree's over and over again as type information is gathered over time on the inference variable. Using a separate IR and using id's and side tables makes all of this much much more simple for Rust.
      - b3morales 4y ago
        Thanks for sharing those details!
puffoflogic 4y ago
> unsafe, of course, [] disables a lot of the checks that the compiler makes. This is, of course, not true. It allows only two things: (1) allow access through pointers and union fields (2) allow calling unsafe or external functions. Neither of these is really a "check", more like "some operation that just isn't present in safe rust and is present in unsafe rust". They aren't conditionally allowed in safe rust; they just aren't allowed. In particular unsafe does not alter the operation of borrowck by one jot.
- steveklabnik 4y ago
  When I’m back at my computer I’m going to reply in thread, ha. It’s very strange to see something you’ve written being presented as authoritatively saying something you didn’t say! (And it’s more than those two things, but you’re still right in general that unsafe adds, it does not remove.)
  - puffoflogic 4y ago
    I'm trying to think what "more" is. I guess I missed inline asm, although I'd probably lump it in with "external function". Static mut, technically, but I don't really consider those part of the current rust language; a historical accident which should be read as `UnsafeCell`. Therefore I'm going to lump that in with unsafe functions too, albeit an invisible one. ETA: I found your lwn comment. I forgot entirely about unsafe traits and I would say those are entirely different from my two categories. So, three things.

34 points by chmaynard 4y ago

Having two IRs (or even having a 1st-stage IR for each frontend language) sounds like something that a compiler project would want to avoid in the long term. I wonder how the IR that's being added for Rust will end up influencing the "regular" IR.

steveklabnik 4y ago
Many many compilers have multiple IRs. It’s very normal. rustc has three already, last time I checked, and that’s not counting llvm-IR. Non-lexical lifetimes was gated on adding an additional IR to rustc, even.
- MuffinFlavored 4y ago
  > rustc has three already Can you expand on this? What are they? I would've thought rustc outputs only 1 IR?
  - fzzzy 4y ago
    There are different stages of ir which allow for different optimizations.
  - steveklabnik 4y ago
    I haven't been paying close attention lately, but for the last few years, compilation looked something vaguely like this: * AST - abstract syntax tree, created from lexing/parsing * HIR - high level IR * THIR - typed high level IR * MIR - mid-level IR (this is the one that enabled non-lexical lifetimes) * LLVM IR - given to LLVM to generate the final binary. If you want to learn more, the rustc dev guide has a bunch of details https://rustc-dev-guide.rust-lang.org/hir.html https://rustc-dev-guide.rust-lang.org/hir.html > I would've thought rustc outputs only 1 IR? Yeah, I mean these are in stages. One goes to the next goes to the next, and there's only one output, but that doesn't mean there aren't a bunch in the middle. Different IRs are good at different things. MIR is based around the control-flow graph, which is also what non-lexical lifetimes are based on, so it's much easier to implement against MIR than it would have been against HIR.
    - philberty 4y ago
      Thanks @steveklabnik we basically have the same in gccrs: 1. AST 2. HIR 3. THIR (side-table lookups) 4. GCC Generic We basically skip MIR in gccrs. Its pretty sensible to have other IR's, we have many passes in gccrs simplifying things so the graph of what your working with is simpler and simpler each time. I mean in GCC for C++ for example they use GENERIC and add a bunch of custom tree-codes such as LAMBDA_EXPR or TEMPLATE stuff for example then they keep substituting etc and finally as part of handing off to GCC middle-end it triggers the gimplification of all of these custom tree codes. So even the C++ front-end you could argue has two IR's.
      - steveklabnik 4y ago
        Good to know, thanks! I’m excited to see gcc-Rama develop, I just haven’t had time to learn how any of it works. Keep up the good work!
        steveklabnik 4y ago
        ...gcc-rs. thanks autocorrect.
- b3morales 4y ago
  Those are serial, though, as I understand it -- each feeds into the next. The gccrs situation sounds like there are two parallel implementations. Please correct me if I'm wrong; I'm not trying to pretend I am any kind of expert here.
  - steveklabnik 4y ago
    I don’t know anything about how this project is architected, but > It's used for type checking and error verification; once that's done, it can be translated and handed to the GCC mid-layer. Sounds the same serial thing to me.
pretty_dumm_guy 4y ago
> Having two IRs (or even having a 1st-stage IR for each frontend language) sounds like something that a compiler project would want to avoid in the long term Can you please share the reason behind why it should be this way ?
- b3morales 4y ago
  I'm not a compiler engineer, so I'm just sharing my own point of view, but to me it sounds like a heavy maintenance burden. Whenever possible I would want to avoid having multiple versions of an important subsystem in any large program that I worked on.
  - philberty 4y ago
    This is true, I tried to make gccrs only have the AST and go from that stright to GCC GENERIC. This had a lot of problems for Rust in my opinion. Many things in Rust are syntactic sugar and can be handled by desugaring the AST into another IR so you dont even have to think about it in other passes. The main issue for me is how complicate the type inference is. So if i wanted to use GCC GENERIC for type resolution for this example: ``` let a; a = 123; let b:u32 = 1; a += b; ``` How do you resolve the type of 'a' you must use inference variables and then update the TREE_TYPE as you go so this means walking the tree's over and over again as type information is gathered over time on the inference variable. Using a separate IR and using id's and side tables makes all of this much much more simple for Rust.
    - b3morales 4y ago
      Thanks for sharing those details!

puffoflogic 4y ago

> unsafe, of course, [] disables a lot of the checks that the compiler makes. This is, of course, not true. It allows only two things: (1) allow access through pointers and union fields (2) allow calling unsafe or external functions. Neither of these is really a "check", more like "some operation that just isn't present in safe rust and is present in unsafe rust". They aren't conditionally allowed in safe rust; they just aren't allowed. In particular unsafe does not alter the operation of borrowck by one jot.

steveklabnik 4y ago
When I’m back at my computer I’m going to reply in thread, ha. It’s very strange to see something you’ve written being presented as authoritatively saying something you didn’t say! (And it’s more than those two things, but you’re still right in general that unsafe adds, it does not remove.)
- puffoflogic 4y ago
  I'm trying to think what "more" is. I guess I missed inline asm, although I'd probably lump it in with "external function". Static mut, technically, but I don't really consider those part of the current rust language; a historical accident which should be read as `UnsafeCell`. Therefore I'm going to lump that in with unsafe functions too, albeit an invisible one. ETA: I found your lwn comment. I forgot entirely about unsafe traits and I would say those are entirely different from my two categories. So, three things.