6 ms·

This is pure speculation, but I suspect egregious use of proc macros in Rust also inflate compilation times a bit. Just speaking anecdotally, but removing proc

This is pure speculation, but I suspect egregious use of proc macros in Rust also inflate compilation times a bit. Just speaking anecdotally, but removing proc macros in one of my hobby projects decreased clean build by about ten seconds. Definitely not a couple magnitudes like the author saw, but not unnoticeable either.

strager 4y ago
Were those proc macros that you wrote, or proc macros in third-party crates?
- ChubbyGlasses 4y ago
  Mine were mostly third party crates like bitflags and thiserror. Not saying that's what was effecting your build times, I'm not really familiar with your repo (my comment was only directed at my proj), but also it's pretty hard to write a significantly large project without macros; it's just the devil you kinda have to live with.
dist1ll 4y ago
Proc macros are used too often in the Rust ecosystem, even for very simple crates. Library authors and maintainers care too little about compilation times.
adwn 4y ago
I always wonder: If the proc macros were executed JIT-like with the Cranelift backend instead of compiling them with the LLVM backend, wouldn't that drastically reduce build times of proc-macro-using code? The Rust compiler spends most of its time in LLVM, if I recall correctly.
- strager 4y ago
  I have heard talk of compiling proc macro crates with different rustc flags than other code. But I didn't go down this rabbit hole.
  - epage 4y ago
    Someone recently looked into doing this by default. The problem was the results varied depending on how often your "host dependencies" (proc macro, build.rs) were shared with your "target dependencies" (`dependencies`, `dev-dependencies`). When there was a lot of sharing, it hurt, but otherwise it sped up build times.
- __jem 4y ago
  The brightest minds in the Rust community are investigating! https://github.com/dtolnay/watt https://github.com/dtolnay/watt
- cormacrelf 4y ago
  The most promising proposal so far is to compile proc macros to WASM. Then you precompile them to a single cross-platform WASM binary, publish those to crates.io alongside the source code, and then execute them in a sandbox. You wouldn’t see much benefit until all your chosen proc macros were available in this form, but after a while this would be great, basically no downsides. In fact you can publish proc macros this way today, IIRC with some caveats about compiler versions (?), but it’s manual and a proof of concept only so nobody has done it. The only thing missing was effort into the idea to make it the default and able to be supported long term. You need a stable WASM ABI that won’t change across rustc versions and for someone to add a runtime to rustc. Dtolnay (the author of the Watt concept) recently put out a message looking for help putting together an RFC. https://twitter.com/davidtolnay/status/1574913813400330241 https://twitter.com/davidtolnay/status/1574913813400330241
- vegerot 4y ago
  The article breaks down how much time is spent by each part of the compiler (borrow checking, LLVM, macro expansion, etc.) check it out!

strager 4y ago

Were those proc macros that you wrote, or proc macros in third-party crates?

ChubbyGlasses 4y ago
Mine were mostly third party crates like bitflags and thiserror. Not saying that's what was effecting your build times, I'm not really familiar with your repo (my comment was only directed at my proj), but also it's pretty hard to write a significantly large project without macros; it's just the devil you kinda have to live with.

dist1ll 4y ago

Proc macros are used too often in the Rust ecosystem, even for very simple crates. Library authors and maintainers care too little about compilation times.

adwn 4y ago

I always wonder: If the proc macros were executed JIT-like with the Cranelift backend instead of compiling them with the LLVM backend, wouldn't that drastically reduce build times of proc-macro-using code? The Rust compiler spends most of its time in LLVM, if I recall correctly.

strager 4y ago
I have heard talk of compiling proc macro crates with different rustc flags than other code. But I didn't go down this rabbit hole.
- epage 4y ago
  Someone recently looked into doing this by default. The problem was the results varied depending on how often your "host dependencies" (proc macro, build.rs) were shared with your "target dependencies" (`dependencies`, `dev-dependencies`). When there was a lot of sharing, it hurt, but otherwise it sped up build times.
__jem 4y ago
The brightest minds in the Rust community are investigating! https://github.com/dtolnay/watt https://github.com/dtolnay/watt
cormacrelf 4y ago
The most promising proposal so far is to compile proc macros to WASM. Then you precompile them to a single cross-platform WASM binary, publish those to crates.io alongside the source code, and then execute them in a sandbox. You wouldn’t see much benefit until all your chosen proc macros were available in this form, but after a while this would be great, basically no downsides. In fact you can publish proc macros this way today, IIRC with some caveats about compiler versions (?), but it’s manual and a proof of concept only so nobody has done it. The only thing missing was effort into the idea to make it the default and able to be supported long term. You need a stable WASM ABI that won’t change across rustc versions and for someone to add a runtime to rustc. Dtolnay (the author of the Watt concept) recently put out a message looking for help putting together an RFC. https://twitter.com/davidtolnay/status/1574913813400330241 https://twitter.com/davidtolnay/status/1574913813400330241
vegerot 4y ago
The article breaks down how much time is spent by each part of the compiler (borrow checking, LLVM, macro expansion, etc.) check it out!