4 ms·

A New Back End for Cranelift

172 points by weinzierl 6y ago

isaacimagine 6y ago
Cranelift looks really promising, and this article is a great peek into how it generates machine code. +1
- pjmlp 6y ago
  It is also faster to compile than using LLVM, ideally it will eventually become another cargo option.
  - jayflux 6y ago
    In case anyone is wondering what’s happening with this, there was a progress report in Sept https://bjorn3.github.io/2020/09/28/progress-report-sep-2020.html https://bjorn3.github.io/2020/09/28/progress-report-sep-2020...
fanf2 6y ago
The earlier version of this article on Chris Fallin’s blog was discussed at https://news.ycombinator.com/item?id=24526088 https://news.ycombinator.com/item?id=24526088
mitchtbaum 6y ago
I have a very powerful Rust transpiler back end that uses semantic programming. My profile has links to the programming engine and the details are put together across many semantically organized issues. The software still needs work and the code generator still needs Rust's lexemes to get to the next stage. The #wg-grammar issue and Discord channel has those plans
Ericson2314 6y ago
First of all, this is good, cranelift is good. Second of all, it's really funny how LLVM started focused on JIT with emphasis on compilation speed, and now is quite slow. Cranelift started in the same way, and now is gaining more passes and IRs, etc. I think more passes and IRs is good for clarity, correctness, and programming productivity, and single-pass designs are too brittle since the purpose of all software (the requirements) changes over time. The solution is we redouble our focus on fusion techniques to try to allow algorithms specified in multiple passes to be fused into a single pass. Then libraries like cranelift and LLVM can prevent a variety of passes, and one has the option to either go for compiler performance, picking only those that fuse or speed of compilation code, picking additional ones that don't fuse.
- chubot 6y ago
  Yeah this is a good observation. Some related papers I saved: Miniphases: compilation using modular and efficient tree transformations, Odersky et al. https://scholar.google.com/scholar?cluster=7946863864036706551&hl=en&as_sdt=2005&sciodt=0,5 https://scholar.google.com/scholar?cluster=79468638640367065... (I'd be interested if anyone has any knowledge of how this actually worked out in the Scala compiler; I don't have experience with it) TreeFuser: a framework for analyzing and fusing general recursive tree traversals. This one said they prototyped it in Clang. https://dl.acm.org/doi/abs/10.1145/3133900 https://dl.acm.org/doi/abs/10.1145/3133900 Deforestation: Transforming programs to eliminate trees, Wadler https://scholar.google.com/scholar?cluster=13826873296507613488&hl=en&as_sdt=0,5&sciodt=0,5 https://scholar.google.com/scholar?cluster=13826873296507613... I think the key point is that some of the frameworks introduce restrictions that make it hard to write a big program like a compiler. There is a hard tradeoff between the amount of fusing that can be done and how expressive the metalanguage is. --- related: https://github.com/oilshell/oil/wiki/Compact-AST-Representation https://github.com/oilshell/oil/wiki/Compact-AST-Representat... (compiling programs that operate on trees to operate on packed representations)
  - still_grokking 6y ago
    > Miniphases To my knowledge the (to be released) Scala 3 compiler[1] implements this concept. Interesting and related is also Odersky's "compilers are databases"[2] concept. [1] https://dotty.epfl.ch/ https://dotty.epfl.ch/ [2] https://www.slideshare.net/Odersky/compilers-are-databases https://www.slideshare.net/Odersky/compilers-are-databases
    - pjmlp 6y ago
      His database concept can be seen already being discussed for C++, a couple of decades ago. https://www.dreamsongs.com/Cadillac.html https://www.dreamsongs.com/Cadillac.html Ironically modern C++ are only now catching up.
- cjv 6y ago
  Chez Scheme is a good example on how many compiler passes can be extremely effective. https://github.com/nanopass/nanopass-framework-scheme/ https://github.com/nanopass/nanopass-framework-scheme/

172 points by weinzierl 6y ago

isaacimagine 6y ago

Cranelift looks really promising, and this article is a great peek into how it generates machine code. +1

pjmlp 6y ago
It is also faster to compile than using LLVM, ideally it will eventually become another cargo option.
- jayflux 6y ago
  In case anyone is wondering what’s happening with this, there was a progress report in Sept https://bjorn3.github.io/2020/09/28/progress-report-sep-2020.html https://bjorn3.github.io/2020/09/28/progress-report-sep-2020...

fanf2 6y ago

The earlier version of this article on Chris Fallin’s blog was discussed at https://news.ycombinator.com/item?id=24526088 https://news.ycombinator.com/item?id=24526088

mitchtbaum 6y ago

I have a very powerful Rust transpiler back end that uses semantic programming. My profile has links to the programming engine and the details are put together across many semantically organized issues. The software still needs work and the code generator still needs Rust's lexemes to get to the next stage. The #wg-grammar issue and Discord channel has those plans

Ericson2314 6y ago

First of all, this is good, cranelift is good. Second of all, it's really funny how LLVM started focused on JIT with emphasis on compilation speed, and now is quite slow. Cranelift started in the same way, and now is gaining more passes and IRs, etc. I think more passes and IRs is good for clarity, correctness, and programming productivity, and single-pass designs are too brittle since the purpose of all software (the requirements) changes over time. The solution is we redouble our focus on fusion techniques to try to allow algorithms specified in multiple passes to be fused into a single pass. Then libraries like cranelift and LLVM can prevent a variety of passes, and one has the option to either go for compiler performance, picking only those that fuse or speed of compilation code, picking additional ones that don't fuse.

chubot 6y ago
Yeah this is a good observation. Some related papers I saved: Miniphases: compilation using modular and efficient tree transformations, Odersky et al. https://scholar.google.com/scholar?cluster=7946863864036706551&hl=en&as_sdt=2005&sciodt=0,5 https://scholar.google.com/scholar?cluster=79468638640367065... (I'd be interested if anyone has any knowledge of how this actually worked out in the Scala compiler; I don't have experience with it) TreeFuser: a framework for analyzing and fusing general recursive tree traversals. This one said they prototyped it in Clang. https://dl.acm.org/doi/abs/10.1145/3133900 https://dl.acm.org/doi/abs/10.1145/3133900 Deforestation: Transforming programs to eliminate trees, Wadler https://scholar.google.com/scholar?cluster=13826873296507613488&hl=en&as_sdt=0,5&sciodt=0,5 https://scholar.google.com/scholar?cluster=13826873296507613... I think the key point is that some of the frameworks introduce restrictions that make it hard to write a big program like a compiler. There is a hard tradeoff between the amount of fusing that can be done and how expressive the metalanguage is. --- related: https://github.com/oilshell/oil/wiki/Compact-AST-Representation https://github.com/oilshell/oil/wiki/Compact-AST-Representat... (compiling programs that operate on trees to operate on packed representations)
- still_grokking 6y ago
  > Miniphases To my knowledge the (to be released) Scala 3 compiler[1] implements this concept. Interesting and related is also Odersky's "compilers are databases"[2] concept. [1] https://dotty.epfl.ch/ https://dotty.epfl.ch/ [2] https://www.slideshare.net/Odersky/compilers-are-databases https://www.slideshare.net/Odersky/compilers-are-databases
  - pjmlp 6y ago
    His database concept can be seen already being discussed for C++, a couple of decades ago. https://www.dreamsongs.com/Cadillac.html https://www.dreamsongs.com/Cadillac.html Ironically modern C++ are only now catching up.
cjv 6y ago
Chez Scheme is a good example on how many compiler passes can be extremely effective. https://github.com/nanopass/nanopass-framework-scheme/ https://github.com/nanopass/nanopass-framework-scheme/