5 ms·

I once had a similar project to OP that took 30 seconds to compile a one-line change (albeit on a 10-year-old CPU). I split it into about 7 crates, and got comp

I once had a similar project to OP that took 30 seconds to compile a one-line change (albeit on a 10-year-old CPU). I split it into about 7 crates, and got compiles down to 3 seconds. Since then I'm always vigilant about keeping crates nice and small. I think as long as you're keeping an eye on your crate sizes, compile times won't get away from you. This is different from C++ where each individual source file can be compiled in parallel, so it's something I had to re-learn.

heavyset_go 5y ago
What was your heuristic for deciding if a crate is too big or not?
- howdydoo 5y ago
  Going by feel, same as when I have a giant function I decide to split up. IIRC the crates ended up between 1k-10k LOC
  - yawaramin 5y ago
    In OP the biggest crate after the split-up is 1,513 loc, with most of them a few hundred loc.
amelius 5y ago
Can this be automated? Could a script divide code into small crates upon compilation?
- howdydoo 5y ago
  I did it by hand. It's a mostly mechanical transformation, so in theory it could be automated, but I've never seen an automated refactor for any language that can e.g. take a giant file and decide how to split it up into 5 smaller files. But that's something I'd love to see exist
brabel 5y ago
That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went. OTOH there might be a benefit too as more code becomes re-usable in independent crates.
- masklinn 5y ago
  > That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went. That's... not the same thing at all. Rust lets you have multiple crate in a single workspace, which allows parallelising the build (because crates are the concurrency unit of building rust). That's got nothing whatsoever to do with pulling random crates, which is a separate issue.
  - yawaramin 5y ago
    Why doesn't Rust default to one crate per source file then? Wouldn't that make the build massively parallelized :-)
    - howinteresting 5y ago
      Because you can have circular dependencies within a compilation unit, but not between units.
      - yawaramin 5y ago
        That actually sounds more appealing to me. In fact it's exactly how OCaml works. Of course in OCaml compilation units are just files already.
        howinteresting 5y ago
        I don't think there is a moral justification for circular dependency scopes to always be individual files, especially given that import statements count as circular. The same issue in other languages where circular scopes are individual files leads to some horrible workarounds: * Python TYPE_CHECKING https://adamj.eu/tech/2021/05/13/python-type-hints-how-to-fix-circular-imports/ https://adamj.eu/tech/2021/05/13/python-type-hints-how-to-fi... * C/C++ header files But regardless, there is too much Rust code in existence to change this now.
dragonelite 5y ago
Sounds like the community still has to decide/write about what are best practises to make the compiler go fast?

heavyset_go 5y ago

What was your heuristic for deciding if a crate is too big or not?

howdydoo 5y ago
Going by feel, same as when I have a giant function I decide to split up. IIRC the crates ended up between 1k-10k LOC
- yawaramin 5y ago
  In OP the biggest crate after the split-up is 1,513 loc, with most of them a few hundred loc.

amelius 5y ago

Can this be automated? Could a script divide code into small crates upon compilation?

howdydoo 5y ago
I did it by hand. It's a mostly mechanical transformation, so in theory it could be automated, but I've never seen an automated refactor for any language that can e.g. take a giant file and decide how to split it up into 5 smaller files. But that's something I'd love to see exist

brabel 5y ago

That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went. OTOH there might be a benefit too as more code becomes re-usable in independent crates.

masklinn 5y ago
> That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went. That's... not the same thing at all. Rust lets you have multiple crate in a single workspace, which allows parallelising the build (because crates are the concurrency unit of building rust). That's got nothing whatsoever to do with pulling random crates, which is a separate issue.
- yawaramin 5y ago
  Why doesn't Rust default to one crate per source file then? Wouldn't that make the build massively parallelized :-)
  - howinteresting 5y ago
    Because you can have circular dependencies within a compilation unit, but not between units.
    - yawaramin 5y ago
      That actually sounds more appealing to me. In fact it's exactly how OCaml works. Of course in OCaml compilation units are just files already.
      - howinteresting 5y ago
        I don't think there is a moral justification for circular dependency scopes to always be individual files, especially given that import statements count as circular. The same issue in other languages where circular scopes are individual files leads to some horrible workarounds: * Python TYPE_CHECKING https://adamj.eu/tech/2021/05/13/python-type-hints-how-to-fix-circular-imports/ https://adamj.eu/tech/2021/05/13/python-type-hints-how-to-fi... * C/C++ header files But regardless, there is too much Rust code in existence to change this now.

dragonelite 5y ago

Sounds like the community still has to decide/write about what are best practises to make the compiler go fast?