4 ms·

Rust’s Incremental Compiler Architecture

119 points by signa11 2y ago

gdiamos 2y ago
This pattern is used in many places. Build systems. Now compiler passes. Container layers. Also in ML systems, eg dvc. Where else is it missing?
- rbalicki 2y ago
  Language servers
fulafel 2y ago
It seems that in https://perf.rust-lang.org/dashboard.html https://perf.rust-lang.org/dashboard.html the improvements have essentially plateaued, I wonder how they decide how much work to invest there. (Although as the language complexity continues to grow maybe that question is missing the opposite force of compiler slowdowns from added work?)
- geewee 2y ago
  I wonder if these graphs capture the new multi-threaded frontend that's on nightly.
  - n42 2y ago
    genuine question: how much can the frontend really impact compile times at the end of the day? I would guess most of the time spent compiling is on the backend, but IANA compiler engineer
    - saagarjha 2y ago
      One obvious example of this would be C++, where a smarter frontend that doesn't do textual inclusion of a million lines would significantly improve compile times.
      - n42 2y ago
        that makes sense. are Rust's macros handled in the frontend? if so, I could see how multithreading could have a dramatic impact
      - menaerus 2y ago
        But that's low-hanging fruit "optimization", no? Once you get around it, and this has been forever, the bottleneck is in the backend generation. So if Rust has already solved this, the area where they can improve most is the backend generation? Most C++ builds I have worked with, and all of them being MMLoC codebases, were actually bottlenecked by the high memory pressure. In part due to heavy templates and symbol visibility.
        fulafel 2y ago
        I think the history of C++ implementations shows that it's not low hanging fruit, it's a huge effort to implement and the payoffs aren't game changing.
        menaerus 2y ago
        Modules? Yeah, I don't expect them to be a game changer either. Exactly because of the reasons I mentioned.
    - nindalf 2y ago
      > How much can the frontend really impact compile times? Your question is addressed in this blog post - Faster compilation with the parallel front-end in nightly (https://blog.rust-lang.org/2023/11/09/parallel-rustc.html https://blog.rust-lang.org/2023/11/09/parallel-rustc.html). One the example shown there: - Serial took 10.2 seconds in the frontend and 6.2 seconds in the backend. - Parallel tool 5.9 seconds (-42%) and 5.3 seconds (-14.5%). So to answer your question, parallelising the frontend can have a substantial impact. You are right that frontend is only a subset of the total compilation time - backend and linking time matter too. Happily, that's something that's being worked on too! There is an alternate backend called cranelift (https://bjorn3.github.io/2024/11/14/progress-report-nov-2024.html https://bjorn3.github.io/2024/11/14/progress-report-nov-2024...) that compiles faster than LLVM, at the cost of producing slower binaries. That's a good trade-off for debug builds. And there's work on changing the default linker to lld (https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-linux.html https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-l...). We may see all 3 efforts land in the next year :)
  - nindalf 2y ago
    These graphs only include stable releases, not nightly ones. Since the multi-threaded frontend hasn't landed on stable yet, we can't see it's effect on these graphs.
- lifthrasiir 2y ago
  The log-scale plot hasn't plateaued yet, so there are still rooms for improvement. (Remind that rustc was initially quite slow at the very beginning.)
- epage 2y ago
  For some insight into what more can be done, see - https://internals.rust-lang.org/t/slow-incremental-compilation-when-changing-small-things-or-comments/22007 https://internals.rust-lang.org/t/slow-incremental-compilati... - https://internals.rust-lang.org/t/feedback-request-performance-improvements-from-reducing-debug-info/21825?u=epage https://internals.rust-lang.org/t/feedback-request-performan... (there is more context to that being published today in a blog post) There are several more ideas that are in even earlier stages of discussion.
sjrd 2y ago
The Scala.js compiler toolchain has been using similar techniques since 2015 by default. It features parallel and incremental whole-program optimizations, so it is also used for release builds. In fact it's so fast we run most advanced optimizations in development builds as well. For those interested in more details, we published a paper about it in OOPSLA'16 [1]. [1] https://dl.acm.org/doi/10.1145/2983990.2984013 https://dl.acm.org/doi/10.1145/2983990.2984013
osa1 2y ago
The article doesn't mention rust-analyzer (the language server), I wonder how much of this infrastructure is also used by rust-analyzer?
- thramp 2y ago
  (I work on rust-analyzer) We don't share the query infrastructure in rust-analyzer, but rust-analyzer is a query-driven system. We use Salsa (https://github.com/salsa-rs/salsa/ https://github.com/salsa-rs/salsa/), which is strongly inspired by rustc's query system, down to the underlying red/green algorithm (https://rustc-dev-guide.rust-lang.org/queries/incremental-compilation.html#the-basic-algorithm https://rustc-dev-guide.rust-lang.org/queries/incremental-co...), hence the name: Salsa.
  - pabs3 2y ago
    Will rust-analyzer ever be safe to run on untrusted code?
phplovesong 2y ago
rust is super slow to compile. Its a showstopper for me using it more. when i compare to say, ocaml to build speeds are like night and day.
- dboreham 2y ago
  And it generates enormous piles of files on disk. Sometimes so much that it won't fit on a github runner.
rbalicki 2y ago
This [1] is a (somewhat outdated!) design doc for how we're approaching incremental compilation in the Isograph compiler! If anyone if interested in helping build an incremental compiler, the project is being actively developed and open to new contributors! See also my recent talk at GraphQL conf [2] In particular, we're actively implementing a simpler Salsa [3] and transform the currently-batch-mode compiler into an incremental, pull compiler. [1] https://isograph.dev/docs/design-docs/incremental-compilation/ https://isograph.dev/docs/design-docs/incremental-compilatio... [2] https://www.youtube.com/watch?v=sf8ac2NtwPY https://www.youtube.com/watch?v=sf8ac2NtwPY [3] https://www.youtube.com/watch?v=_muY4HjSqVw https://www.youtube.com/watch?v=_muY4HjSqVw
bfrog 2y ago
I’m interested to see how things progress! rustc itself is still reliant on llvm and ld, but there’s some intriguing alternatives coming in cranelift and wild both written in rust themselves, and I think done in a way that would allow incremental compilation and linking to be done in a way that better suits rust.
deleted 2y ago
[deleted]

119 points by signa11 2y ago

This pattern is used in many places. Build systems. Now compiler passes. Container layers. Also in ML systems, eg dvc. Where else is it missing?

rbalicki 2y ago
Language servers

fulafel 2y ago

It seems that in https://perf.rust-lang.org/dashboard.html https://perf.rust-lang.org/dashboard.html the improvements have essentially plateaued, I wonder how they decide how much work to invest there. (Although as the language complexity continues to grow maybe that question is missing the opposite force of compiler slowdowns from added work?)

geewee 2y ago
I wonder if these graphs capture the new multi-threaded frontend that's on nightly.
- n42 2y ago
  genuine question: how much can the frontend really impact compile times at the end of the day? I would guess most of the time spent compiling is on the backend, but IANA compiler engineer
  - saagarjha 2y ago
    One obvious example of this would be C++, where a smarter frontend that doesn't do textual inclusion of a million lines would significantly improve compile times.
    - n42 2y ago
      that makes sense. are Rust's macros handled in the frontend? if so, I could see how multithreading could have a dramatic impact
    - menaerus 2y ago
      But that's low-hanging fruit "optimization", no? Once you get around it, and this has been forever, the bottleneck is in the backend generation. So if Rust has already solved this, the area where they can improve most is the backend generation? Most C++ builds I have worked with, and all of them being MMLoC codebases, were actually bottlenecked by the high memory pressure. In part due to heavy templates and symbol visibility.
      - fulafel 2y ago
        I think the history of C++ implementations shows that it's not low hanging fruit, it's a huge effort to implement and the payoffs aren't game changing.
        menaerus 2y ago
        Modules? Yeah, I don't expect them to be a game changer either. Exactly because of the reasons I mentioned.
  - nindalf 2y ago
    > How much can the frontend really impact compile times? Your question is addressed in this blog post - Faster compilation with the parallel front-end in nightly (https://blog.rust-lang.org/2023/11/09/parallel-rustc.html https://blog.rust-lang.org/2023/11/09/parallel-rustc.html). One the example shown there: - Serial took 10.2 seconds in the frontend and 6.2 seconds in the backend. - Parallel tool 5.9 seconds (-42%) and 5.3 seconds (-14.5%). So to answer your question, parallelising the frontend can have a substantial impact. You are right that frontend is only a subset of the total compilation time - backend and linking time matter too. Happily, that's something that's being worked on too! There is an alternate backend called cranelift (https://bjorn3.github.io/2024/11/14/progress-report-nov-2024.html https://bjorn3.github.io/2024/11/14/progress-report-nov-2024...) that compiles faster than LLVM, at the cost of producing slower binaries. That's a good trade-off for debug builds. And there's work on changing the default linker to lld (https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-linux.html https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-l...). We may see all 3 efforts land in the next year :)
- nindalf 2y ago
  These graphs only include stable releases, not nightly ones. Since the multi-threaded frontend hasn't landed on stable yet, we can't see it's effect on these graphs.
lifthrasiir 2y ago
The log-scale plot hasn't plateaued yet, so there are still rooms for improvement. (Remind that rustc was initially quite slow at the very beginning.)
epage 2y ago
For some insight into what more can be done, see - https://internals.rust-lang.org/t/slow-incremental-compilation-when-changing-small-things-or-comments/22007 https://internals.rust-lang.org/t/slow-incremental-compilati... - https://internals.rust-lang.org/t/feedback-request-performance-improvements-from-reducing-debug-info/21825?u=epage https://internals.rust-lang.org/t/feedback-request-performan... (there is more context to that being published today in a blog post) There are several more ideas that are in even earlier stages of discussion.

sjrd 2y ago

The Scala.js compiler toolchain has been using similar techniques since 2015 by default. It features parallel and incremental whole-program optimizations, so it is also used for release builds. In fact it's so fast we run most advanced optimizations in development builds as well. For those interested in more details, we published a paper about it in OOPSLA'16 [1]. [1] https://dl.acm.org/doi/10.1145/2983990.2984013 https://dl.acm.org/doi/10.1145/2983990.2984013

osa1 2y ago

The article doesn't mention rust-analyzer (the language server), I wonder how much of this infrastructure is also used by rust-analyzer?

thramp 2y ago
(I work on rust-analyzer) We don't share the query infrastructure in rust-analyzer, but rust-analyzer is a query-driven system. We use Salsa (https://github.com/salsa-rs/salsa/ https://github.com/salsa-rs/salsa/), which is strongly inspired by rustc's query system, down to the underlying red/green algorithm (https://rustc-dev-guide.rust-lang.org/queries/incremental-compilation.html#the-basic-algorithm https://rustc-dev-guide.rust-lang.org/queries/incremental-co...), hence the name: Salsa.
- pabs3 2y ago
  Will rust-analyzer ever be safe to run on untrusted code?

phplovesong 2y ago

rust is super slow to compile. Its a showstopper for me using it more. when i compare to say, ocaml to build speeds are like night and day.

dboreham 2y ago
And it generates enormous piles of files on disk. Sometimes so much that it won't fit on a github runner.

rbalicki 2y ago

This [1] is a (somewhat outdated!) design doc for how we're approaching incremental compilation in the Isograph compiler! If anyone if interested in helping build an incremental compiler, the project is being actively developed and open to new contributors! See also my recent talk at GraphQL conf [2] In particular, we're actively implementing a simpler Salsa [3] and transform the currently-batch-mode compiler into an incremental, pull compiler. [1] https://isograph.dev/docs/design-docs/incremental-compilation/ https://isograph.dev/docs/design-docs/incremental-compilatio... [2] https://www.youtube.com/watch?v=sf8ac2NtwPY https://www.youtube.com/watch?v=sf8ac2NtwPY [3] https://www.youtube.com/watch?v=_muY4HjSqVw https://www.youtube.com/watch?v=_muY4HjSqVw

bfrog 2y ago

I’m interested to see how things progress! rustc itself is still reliant on llvm and ld, but there’s some intriguing alternatives coming in cranelift and wild both written in rust themselves, and I think done in a way that would allow incremental compilation and linking to be done in a way that better suits rust.

deleted 2y ago

[deleted]