4 ms·

As someone who has only done small projects in Rust, I'm curious how many LoC are we're talking? And were you splitting your project into crates where it made s

As someone who has only done small projects in Rust, I'm curious how many LoC are we're talking? And were you splitting your project into crates where it made sense?

eminence32 3y ago
One thing I've observed is that every smallish projects (~10k LoC) can take a while to build if you're working on slow network filesystems. When using local disks, a 10k LoC project takes ~5 seconds to build When using network FS, the same project takes ~23 seconds to build
- the8472 3y ago
  That doesn't seem particularly surprising. Lots of software is slow when your storage is slow.
  - eminence32 3y ago
    You're right that some slowdown is expected, but for me personally I hadn't realized how bad this particular FS was, nor had I expected how much it impacted build times
- lolinder 3y ago
  Wouldn't this be true of any compiler? Do other compilers have optimizations that avoid excess disk usage when on a slow file system?
  - eminence32 3y ago
    Yes, I think that's right, but I don't have any data to compare rust to other compilers.
  - ragnese 3y ago
    I wouldn't be surprised if Rust/Cargo does more disk IO than other build tools, though. Rust does a lot of compile time code gen and caches a lot of stuff on disk.
- insanitybit 3y ago
  FWIW we did this to significantly reduce storage usage strip = "debuginfo" This was a ~20x reduction in size, 279M -> 19M. If you have slow storage it could really help performance (don't recall, personally).
  - eminence32 3y ago
    Good hint, thank you. I'm trying this out and I do see an improvement in performance. The ~23 second build now takes about ~14 seconds
    - insanitybit 3y ago
      Hell yeah! I'm so glad I could help, that's a big win. FWIW it does break debuggers since you won't have the symbols. Just comment it out when you need a build that has all of that info.
insanitybit 3y ago
We split each service into crates, plus some libraries. There were some native dependencies as well, which could really impact compile times, as well as some codegen for things like protobuf. From a quick tokei, looks like 49kloc of Rust.

As someone who has only done small projects in Rust, I'm curious how many LoC are we're talking? And were you splitting your project into crates where it made sense?

eminence32 3y ago

One thing I've observed is that every smallish projects (~10k LoC) can take a while to build if you're working on slow network filesystems. When using local disks, a 10k LoC project takes ~5 seconds to build When using network FS, the same project takes ~23 seconds to build

the8472 3y ago
That doesn't seem particularly surprising. Lots of software is slow when your storage is slow.
- eminence32 3y ago
  You're right that some slowdown is expected, but for me personally I hadn't realized how bad this particular FS was, nor had I expected how much it impacted build times
lolinder 3y ago
Wouldn't this be true of any compiler? Do other compilers have optimizations that avoid excess disk usage when on a slow file system?
- eminence32 3y ago
  Yes, I think that's right, but I don't have any data to compare rust to other compilers.
- ragnese 3y ago
  I wouldn't be surprised if Rust/Cargo does more disk IO than other build tools, though. Rust does a lot of compile time code gen and caches a lot of stuff on disk.
insanitybit 3y ago
FWIW we did this to significantly reduce storage usage strip = "debuginfo" This was a ~20x reduction in size, 279M -> 19M. If you have slow storage it could really help performance (don't recall, personally).
- eminence32 3y ago
  Good hint, thank you. I'm trying this out and I do see an improvement in performance. The ~23 second build now takes about ~14 seconds
  - insanitybit 3y ago
    Hell yeah! I'm so glad I could help, that's a big win. FWIW it does break debuggers since you won't have the symbols. Just comment it out when you need a build that has all of that info.

insanitybit 3y ago

We split each service into crates, plus some libraries. There were some native dependencies as well, which could really impact compile times, as well as some codegen for things like protobuf. From a quick tokei, looks like 49kloc of Rust.