6 ms·

I read the projects web page and github page, but nowhere did I see any detailing of the "why". Why do we need another IR for LLVM? I am working on the fronten

I read the projects web page and github page, but nowhere did I see any detailing of the "why". Why do we need another IR for LLVM? I am working on the frontend for a toy lang, and was planning to transpile to LLVM IR. This is of interest to me. But its frustrating how opaque the dev docs for such projects are.

erichocean 4y ago
> Why do we need another IR for LLVM? Maybe because it's not "another IR for LLVM", but an IR for Clang? Anyway, if you actually care: https://discourse.llvm.org/t/rfc-an-mlir-based-clang-ir-cir/63319 https://discourse.llvm.org/t/rfc-an-mlir-based-clang-ir-cir/...
- rishav_sharan 4y ago
  Thanks for the Discourse link. It has much more information. Quoting here for benefit of others; "Motivation In general, Clang’s AST is not an appropriate representation for dataflow analysis and reasoning about control flow. On the other hand, LLVM IR is too low level — it exists at a point in which we have already lost vital language information (e.g. scope information, loop forms and type hierarchies are invisible at the LLVM level), forcing a pass writer to attempt reconstruction of the original semantics. This leads to inaccurate results and inefficient analysis - not to mention the Sisyphean maintenance work given how fast LLVM changes. Clang’s CFG is supposed to bridge this gap but isn’t ideal either: a parallel lowering path for dataflow diagnostics that (a) is discarded after analysis, (b) has lots of known problems (checkout Kristóf Uman’s great survey 31 regarding “dataflowness”) and (c) has testing coverage for CFG pieces not quite up to LLVM’s standards. We also have the prominent recent success stories of Swift’s SIL and Rust’s HIR and MIR. These two projects have leveraged high level IRs to improve their performance and safety. We believe CIR could provide the same improvements for C++." > Maybe because it's not "another IR for LLVM", but an IR for Clang? AFAIK, Clang also uses LLVM IR.
  - astrange 4y ago
    > AFAIK, Clang also uses LLVM IR. Clang doesn't use an IR for the frontend stuff; it emits AST straight to LLVM. It's built for speed and not very flexible. It can be difficult to see what some features like ObjC ARC are doing since there's no way to see an "intermediate" representation without debugging the compiler.
flohofwoe 4y ago
Probably for the same reasons that Rust has MIR [0] and Zig has ZIR [1], LLVM IR may be too lowlevel and awkward to do some language-specific analysis passes. As I understand it, those language-specific IRs are closer to the language, why LLVM IR is closer to a cross-platform assembly dialect (or at least doesn't preserve much information from the original language). [0] https://blog.rust-lang.org/2016/04/19/MIR.html https://blog.rust-lang.org/2016/04/19/MIR.html [1] https://mitchellh.com/zig/astgen#what-does-zir-look-like https://mitchellh.com/zig/astgen#what-does-zir-look-like
- amelius 4y ago
  Do these IRs all build upon LLVM? Or do they intend to totally reinvent the wheel for every new target?
  - conradludgate 4y ago
    At least for Rust, I know that it flows through the whole pipeline and piggy backs off of llvmir for multi platform support and strong optimisations Rust -> HIR (higher intermediate representation) -> MIR (middle intermediate representation) -> LLVMIR Different stages are differing levels of complexity and do different checks
  - pdpi 4y ago
    The optimisation opportunities available to the compiler author changes depending on where you are along the process. As you go from the original source code to the final binary, you progressively lose the ability to reason about high-level constructs and gain the ability to reason about low-level concerns. What these extra IRs afford you is more granularity. More points to choose from along that high-level/low-level gradient mean you also have a better chance at the right balance of high-level and low-level information for your optimisation pass to work.
danuker 4y ago
The second sentence is as follows: The LLVM’s discourse RFC goes in depth about the project motivation Link to the discourse RFC: https://discourse.llvm.org/t/rfc-an-mlir-based-clang-ir-cir/63319 https://discourse.llvm.org/t/rfc-an-mlir-based-clang-ir-cir/... Motivation: > In general, Clang’s AST is not an appropriate representation for dataflow analysis and reasoning about control flow. On the other hand, LLVM IR is too low level — it exists at a point in which we have already lost vital language information (e.g. scope information, loop forms and type hierarchies are invisible at the LLVM level), forcing a pass writer to attempt reconstruction of the original semantics. This leads to inaccurate results and inefficient analysis - not to mention the Sisyphean maintenance work given how fast LLVM changes. Clang’s CFG is supposed to bridge this gap but isn’t ideal either: a parallel lowering path for dataflow diagnostics that (a) is discarded after analysis, (b) has lots of known problems (checkout Kristóf Uman’s great survey 32 regarding “dataflowness”) and (c) has testing coverage for CFG pieces not quite up to LLVM’s standards.
CalChris 4y ago
Why? Well, LLVM IR is pretty low level. Lattner and company learned when doing Swift that analyses and optimizations were more easily done with a higher level IR. So they created the higher level Swift IR and analyzed and optimized in that higher level IR and then lowered to LLVM IR. Generalizing from that, they created MLIR as a toolkit for higher level IRs which can lower to LLVM IR and indeed even other targets. However, Clang remained unchanged. So what Facebook is doing now is applying this IR refactoring that had been successful with Swift to Clang. The result, CIR, will allow a better structured, faster and more secure Clang front end [1]. This is not some magical nonsense and you're not going to get magical results. This is just solid engineering and is based on an already successful pattern. [1] https://www.phoronix.com/scan.php?page=news_item&px=Meta-Developing-Clang-IR-CIR https://www.phoronix.com/scan.php?page=news_item&px=Meta-Dev...

erichocean 4y ago

> Why do we need another IR for LLVM? Maybe because it's not "another IR for LLVM", but an IR for Clang? Anyway, if you actually care: https://discourse.llvm.org/t/rfc-an-mlir-based-clang-ir-cir/63319 https://discourse.llvm.org/t/rfc-an-mlir-based-clang-ir-cir/...

rishav_sharan 4y ago
Thanks for the Discourse link. It has much more information. Quoting here for benefit of others; "Motivation In general, Clang’s AST is not an appropriate representation for dataflow analysis and reasoning about control flow. On the other hand, LLVM IR is too low level — it exists at a point in which we have already lost vital language information (e.g. scope information, loop forms and type hierarchies are invisible at the LLVM level), forcing a pass writer to attempt reconstruction of the original semantics. This leads to inaccurate results and inefficient analysis - not to mention the Sisyphean maintenance work given how fast LLVM changes. Clang’s CFG is supposed to bridge this gap but isn’t ideal either: a parallel lowering path for dataflow diagnostics that (a) is discarded after analysis, (b) has lots of known problems (checkout Kristóf Uman’s great survey 31 regarding “dataflowness”) and (c) has testing coverage for CFG pieces not quite up to LLVM’s standards. We also have the prominent recent success stories of Swift’s SIL and Rust’s HIR and MIR. These two projects have leveraged high level IRs to improve their performance and safety. We believe CIR could provide the same improvements for C++." > Maybe because it's not "another IR for LLVM", but an IR for Clang? AFAIK, Clang also uses LLVM IR.
- astrange 4y ago
  > AFAIK, Clang also uses LLVM IR. Clang doesn't use an IR for the frontend stuff; it emits AST straight to LLVM. It's built for speed and not very flexible. It can be difficult to see what some features like ObjC ARC are doing since there's no way to see an "intermediate" representation without debugging the compiler.

flohofwoe 4y ago

Probably for the same reasons that Rust has MIR [0] and Zig has ZIR [1], LLVM IR may be too lowlevel and awkward to do some language-specific analysis passes. As I understand it, those language-specific IRs are closer to the language, why LLVM IR is closer to a cross-platform assembly dialect (or at least doesn't preserve much information from the original language). [0] https://blog.rust-lang.org/2016/04/19/MIR.html https://blog.rust-lang.org/2016/04/19/MIR.html [1] https://mitchellh.com/zig/astgen#what-does-zir-look-like https://mitchellh.com/zig/astgen#what-does-zir-look-like

amelius 4y ago
Do these IRs all build upon LLVM? Or do they intend to totally reinvent the wheel for every new target?
- conradludgate 4y ago
  At least for Rust, I know that it flows through the whole pipeline and piggy backs off of llvmir for multi platform support and strong optimisations Rust -> HIR (higher intermediate representation) -> MIR (middle intermediate representation) -> LLVMIR Different stages are differing levels of complexity and do different checks
- pdpi 4y ago
  The optimisation opportunities available to the compiler author changes depending on where you are along the process. As you go from the original source code to the final binary, you progressively lose the ability to reason about high-level constructs and gain the ability to reason about low-level concerns. What these extra IRs afford you is more granularity. More points to choose from along that high-level/low-level gradient mean you also have a better chance at the right balance of high-level and low-level information for your optimisation pass to work.

danuker 4y ago

The second sentence is as follows: The LLVM’s discourse RFC goes in depth about the project motivation Link to the discourse RFC: https://discourse.llvm.org/t/rfc-an-mlir-based-clang-ir-cir/63319 https://discourse.llvm.org/t/rfc-an-mlir-based-clang-ir-cir/... Motivation: > In general, Clang’s AST is not an appropriate representation for dataflow analysis and reasoning about control flow. On the other hand, LLVM IR is too low level — it exists at a point in which we have already lost vital language information (e.g. scope information, loop forms and type hierarchies are invisible at the LLVM level), forcing a pass writer to attempt reconstruction of the original semantics. This leads to inaccurate results and inefficient analysis - not to mention the Sisyphean maintenance work given how fast LLVM changes. Clang’s CFG is supposed to bridge this gap but isn’t ideal either: a parallel lowering path for dataflow diagnostics that (a) is discarded after analysis, (b) has lots of known problems (checkout Kristóf Uman’s great survey 32 regarding “dataflowness”) and (c) has testing coverage for CFG pieces not quite up to LLVM’s standards.

CalChris 4y ago

Why? Well, LLVM IR is pretty low level. Lattner and company learned when doing Swift that analyses and optimizations were more easily done with a higher level IR. So they created the higher level Swift IR and analyzed and optimized in that higher level IR and then lowered to LLVM IR. Generalizing from that, they created MLIR as a toolkit for higher level IRs which can lower to LLVM IR and indeed even other targets. However, Clang remained unchanged. So what Facebook is doing now is applying this IR refactoring that had been successful with Swift to Clang. The result, CIR, will allow a better structured, faster and more secure Clang front end [1]. This is not some magical nonsense and you're not going to get magical results. This is just solid engineering and is based on an already successful pattern. [1] https://www.phoronix.com/scan.php?page=news_item&px=Meta-Developing-Clang-IR-CIR https://www.phoronix.com/scan.php?page=news_item&px=Meta-Dev...