4 ms·

Not related to async in Rust, but a 26 sec clean build time for a project with 69 dependencies and no code of its own just to get to the point where we can "do

Not related to async in Rust, but a 26 sec clean build time for a project with 69 dependencies and no code of its own just to get to the point where we can "do something useful" feels a little wasteful.

kungito 5y ago
I'd say it's pretty unfair to take clean build time to consider doing something useful. 98% of my rust builds are incremental builds which take 10-15 seconds for a project I have been working on for 9 months full time and I think my dependency tree has easily 1000 dependencies.
- DixieDev 5y ago
  True, it's not a real problem in most scenarios if incremental compile times are good. I still feel uneasy depending on so many other crates, but this seems to be a level of paranoia that others in the community don't share. Having 1000 dependencies sounds crazy to me! If there's a bug in even one of them that affects you then there's gonna be a lot of digging to figure out the cause, and possibly multiple pull requests to get it fixed. I think my CPU would spend a bit longer than 10-15 secs on the linker step, too.
  - fasterthanlime 5y ago
    Compile/link times are definitely a discussion topic. Here's some quick tips: * Incremental compilation helps locally, not so much in CI, unless you save/restore the whole cache which gets large quickly and evicting the out-of-date objects is non-trivial. * In CI, sccache helps, but not as much as I'd like: there's a bunch of things that are non-cacheable, notably crates that pull in & compile C/C++ code (I'd trade 2 c-bindings crates against 12 Rust-only crates any day for that reason alone) * Splitting stuff across different crates helps parallelizing the build (and caching it better), which may be one of the reasons some projects end up having "1000 dependencies" (although I've rarely seen upwards of 600) * For incremental builds, linking does become the bottleneck. Full LTO is especially slow, Thin LTO is better. Switching to LLD improves thing. I'm hopeful that mold will improve things some more. * Re multiple PRs: thankfully crate families tend to live in the same repository, so fixing something across both tracing / tracing-subscriber could be a single PR, for example. I wish the Rust community invested more in build caching, but even with the current state of things, there's often steps you can take to make things better.
    - styluss 5y ago
      Have you tried sccache? https://github.com/mozilla/sccache https://github.com/mozilla/sccache
      - riquito 5y ago
        It's his second bullet point
    - snovv_crash 5y ago
      Can you combine ccache with a sccache to make the C/C++ parts fast too?
  - myrrlyn 5y ago
    the reason for > I still feel uneasy depending on so many other crates, but this seems to be a level of paranoia that others in the community don't share. Having 1000 dependencies sounds crazy to me! If there's a bug in even one of them that affects you then there's gonna be a lot of digging to figure out the cause is that there is far, far more likely to be a bug in the version you write on your own to achieve the same goal than there is in a widely-observed library written by somebody who's chosen to specialize in that specific thing
    - hnlmorg 5y ago
      In theory yes. But in practice that isn't always true. People often don't audit other modules on the assumption someone else had. Which means nobody ends up doing it. And if you end up with an ecosystem that favours more modules over fewer, you can end up with more modules than a given developer or team are willing to audit (a bit like "alarm fatigue" where if you have too many objects to check then people will inevitably just get lazy). Just look at how many C and C++ libraries are maintained by 1 individual and have almost no 3rd party oversight to see that Rust can't automatically make the claim you made. That all said, for anything complicated and/or directly security related, one should always check if there is a module first.
      - db48x 5y ago
        I look at it the other way around. You own any bug in your product whether it comes from a dependency or from code of your own; you have to fix the bug either way. Using a dependency doesn’t reduce your responsibility, but it does reduce the amount of code that you have to write yourself.
        hnlmorg 5y ago
        But if you are willing to own that responsibility then you should read the code you're importing to begin with. I know I do but I also know most people don't bother. I do acknowledge that there will always be bugs that are identified by your users but equally if you're not auditing your dependencies first then it's hard to argue that you're not just passing off that responsibility wholesale to your users.
  - volta83 5y ago
    I rather have 1000 dependencies than re-implement all those 1000 dependencies myself, chase all the bugs that have already been ironed out in those, etc. My time is worth more than my computer's time.
  - kelnos 5y ago
    > Having 1000 dependencies sounds crazy to me! I was thinking that too, but then I did a quick check of a few of my Java services for work, and many of them have more than 400 dependencies (most transitive, of course), with a few getting up to 600. Obviously comparing Java and Rust is not exactly apples to apples, but I rarely care much about the number of dependencies in my Java projects, so I guess I shouldn't be too worried about Rust projects either (aside from link time, I guess, which javac doesn't have to do).
pornel 5y ago
Async runtime, channels, tracing, backtraces. It does compile a lot of useful stuff ready for you to use. It's all cached, so consider it more of a library install time.

kungito 5y ago

I'd say it's pretty unfair to take clean build time to consider doing something useful. 98% of my rust builds are incremental builds which take 10-15 seconds for a project I have been working on for 9 months full time and I think my dependency tree has easily 1000 dependencies.

DixieDev 5y ago
True, it's not a real problem in most scenarios if incremental compile times are good. I still feel uneasy depending on so many other crates, but this seems to be a level of paranoia that others in the community don't share. Having 1000 dependencies sounds crazy to me! If there's a bug in even one of them that affects you then there's gonna be a lot of digging to figure out the cause, and possibly multiple pull requests to get it fixed. I think my CPU would spend a bit longer than 10-15 secs on the linker step, too.
- fasterthanlime 5y ago
  Compile/link times are definitely a discussion topic. Here's some quick tips: * Incremental compilation helps locally, not so much in CI, unless you save/restore the whole cache which gets large quickly and evicting the out-of-date objects is non-trivial. * In CI, sccache helps, but not as much as I'd like: there's a bunch of things that are non-cacheable, notably crates that pull in & compile C/C++ code (I'd trade 2 c-bindings crates against 12 Rust-only crates any day for that reason alone) * Splitting stuff across different crates helps parallelizing the build (and caching it better), which may be one of the reasons some projects end up having "1000 dependencies" (although I've rarely seen upwards of 600) * For incremental builds, linking does become the bottleneck. Full LTO is especially slow, Thin LTO is better. Switching to LLD improves thing. I'm hopeful that mold will improve things some more. * Re multiple PRs: thankfully crate families tend to live in the same repository, so fixing something across both tracing / tracing-subscriber could be a single PR, for example. I wish the Rust community invested more in build caching, but even with the current state of things, there's often steps you can take to make things better.
  - styluss 5y ago
    Have you tried sccache? https://github.com/mozilla/sccache https://github.com/mozilla/sccache
    - riquito 5y ago
      It's his second bullet point
  - snovv_crash 5y ago
    Can you combine ccache with a sccache to make the C/C++ parts fast too?
- myrrlyn 5y ago
  the reason for > I still feel uneasy depending on so many other crates, but this seems to be a level of paranoia that others in the community don't share. Having 1000 dependencies sounds crazy to me! If there's a bug in even one of them that affects you then there's gonna be a lot of digging to figure out the cause is that there is far, far more likely to be a bug in the version you write on your own to achieve the same goal than there is in a widely-observed library written by somebody who's chosen to specialize in that specific thing
  - hnlmorg 5y ago
    In theory yes. But in practice that isn't always true. People often don't audit other modules on the assumption someone else had. Which means nobody ends up doing it. And if you end up with an ecosystem that favours more modules over fewer, you can end up with more modules than a given developer or team are willing to audit (a bit like "alarm fatigue" where if you have too many objects to check then people will inevitably just get lazy). Just look at how many C and C++ libraries are maintained by 1 individual and have almost no 3rd party oversight to see that Rust can't automatically make the claim you made. That all said, for anything complicated and/or directly security related, one should always check if there is a module first.
    - db48x 5y ago
      I look at it the other way around. You own any bug in your product whether it comes from a dependency or from code of your own; you have to fix the bug either way. Using a dependency doesn’t reduce your responsibility, but it does reduce the amount of code that you have to write yourself.
      - hnlmorg 5y ago
        But if you are willing to own that responsibility then you should read the code you're importing to begin with. I know I do but I also know most people don't bother. I do acknowledge that there will always be bugs that are identified by your users but equally if you're not auditing your dependencies first then it's hard to argue that you're not just passing off that responsibility wholesale to your users.
- volta83 5y ago
  I rather have 1000 dependencies than re-implement all those 1000 dependencies myself, chase all the bugs that have already been ironed out in those, etc. My time is worth more than my computer's time.
- kelnos 5y ago
  > Having 1000 dependencies sounds crazy to me! I was thinking that too, but then I did a quick check of a few of my Java services for work, and many of them have more than 400 dependencies (most transitive, of course), with a few getting up to 600. Obviously comparing Java and Rust is not exactly apples to apples, but I rarely care much about the number of dependencies in my Java projects, so I guess I shouldn't be too worried about Rust projects either (aside from link time, I guess, which javac doesn't have to do).

pornel 5y ago

Async runtime, channels, tracing, backtraces. It does compile a lot of useful stuff ready for you to use. It's all cached, so consider it more of a library install time.