6 ms·
As a relative neophyte in Rust (have gone through about half the chapters in the Rust Book), I recently deployed a small Rust server in DigitalOcean, and was su
by ajtjp 6y ago
As a relative neophyte in Rust (have gone through about half the chapters in the Rust Book), I recently deployed a small Rust server in DigitalOcean, and was surprised by the compilation speed. The server's code was about 27 KB in size, producing a binary of about 160 KB in size. But it had 92 dependencies, including transitive dependencies, and took 5 minutes and 38 seconds to build. Which was quite impressive relative to the size of the code, even allowing for the not-super-fast CPU.
While I was watching its output, I realized that in Rust, all the dependencies are compiled; I'm used to Java+Maven or JavaScript+NPM where compiled dependencies are used instead, and that tends to be pretty quick (provided your network pipe is wide enough). I'd be curious to learn why Cargo re-compiles from scratch instead of offering pre-compiled binaries as well. I guess part of it is related to different target platforms, but it seems like if the top 5 platforms were targeted and had compiled resources available, you could reduce compile times for those platforms by a significant amount.
On the other hand, the error messages I ran into along the way were quite good at pointing me in the right direction about how to fix them, which saved more time than the extra compile time cost, relative to the Python-based alternative I had been trying to set up before that.
- Elinvynia 6y agoWith Rust you get safety at compile time while avoiding memory safety issues at runtime. The compile times are objectively longer compared to most other languages. But whether this trade-off is worth it for you depends a lot on your company's CI pipeline and the skill of the developers.
- Gibbon1 6y agoI feel like the long compilation times is going to be a show stopper for smaller teams/companies where you can't afford the loss in productivity.
- efnx 6y agoAnecdotally I can tell you it does not amount to a loss in productivity. Cargo caches built deps so the cost is really only paid once in a while and the advantages of rusts type system are a boon to productivity.
- lmm 6y agoIf productivity is important, you really can't afford memory corruption bugs. Those can easily take weeks to hunt down.
- pdimitar 6y agoIt can be slightly annoying -- compared to f.ex. OCaml with its lightning-fast compiler -- but in practice incremental recompilation is much faster than a compilation from scratch so it rarely irks me that badly.
- verdagon 6y agoI'm not sure safety has anything to do with long compile times, can you enlighten?
- efnx 6y agoThere’s a lot happening at compile time that less safe languages don’t do. Type checking and type erasure, etc
- steveklabnik 6y agoIt's not just target platforms, it's also build flags; you could do the default for debug and release, but set any sort of custom option and you're back to square 1. We would also need to pay for the cost and such of building, hosting, and distributing all of that... There are other middle grounds too. There's certainly interest, it's just not trivial. If it was, we'd do it!
- pjmlp 6y agoHave you watched Swift related announcements at WWDC? Package manager support for binary dependencies is coming with Big Sur.
- steveklabnik 6y agoNot yet. The most valuable company in the world has significantly more resources than we do.
- jka 6y agoWith hope for the presence of positive-minded folks with contribution time to spare: any suggestions of areas that could use help towards binary builds?
- steveklabnik 6y agoI would reach out to the Cargo team: https://www.rust-lang.org/governance/teams/dev-tools https://www.rust-lang.org/governance/teams/dev-tools
- littlestymaar 6y agoApple controlling the hardware makes it way easier though. At home, I compile Rust with CPU-specific features on about as many different hardware as Apple for their whole ecosystem!
- masklinn 6y ago
- pjmlp 6y agoThat is the approach taken by most compiled languages, specially in commercial environments. I think it is only a matter of time until such cargo gets its "Maven". Incidently one of Swift announcements at WWDC was the support of binary packages in Swift Package Manager.
- masklinn 6y ago> I'd be curious to learn why Cargo re-compiles from scratch instead of offering pre-compiled binaries as well. I guess part of it is related to different target platforms There’s that but there’s also issues like ABI stability (and the lack thereof), compilation flags, hosting (infrastructure and its cost), distribution mechanisms, … There’s an issue on cargo dating back to 2015 (#1139) but there’s a lot of efforts needed to think about the problem, then actually solve it.
- Cyph0n 6y agoJust to add to other comments: Cargo feature flags are another reason why you need to build your dependencies. Of course, if Cargo supported pre-compiled dependencies, I am guessing that it would be smart enough to only recompile the dependencies that are using non-default features.
- Matthias247 6y agoIn addition to just feature flags the fact that a lot of Rust code is generic and would only be compiled into binary form in the final application would likely prevent excessive use of precompiled artifacts.
- sk0g 6y agoGo would have compiled an order of magnitude faster still. The compiler is less safe, and thorough than Rust's, but still.
- efnx 6y agoYou’re right, though after an initial compilation cargo does a good job caching and subsequent builds will often beat Go’s compile times.
- sk0g 6y agoThe backend service I'm using doesn't have cached CI yet, so the entire build takes ~20 seconds. The CircleCI free tier is good enough for now, and more time is spent pulling dependencies than building, anyway. The build pulls in heaps of dependencies, and the main codebase itself being around 20kloc. Subsequent local builds take around .2 seconds locally, so I'm very happy with it. Even working on a tiny Java/ Kotlin codebase recently made me miss the good compile times! How hard is the caching to set up, especially in a CI setting?
- efnx 6y agoIt depends - at my work we use buildkite which is a “bring your own runners” build service, so caching is available by default, so long as the cache is outside of the project directory. For personal stuff I use GitHub and Gitlab - they both offer caching. GitHub’s offering is easier but Gitlabs is just as effective. Cargo caches by default and you can specify the path that it stores artifacts. Most of the cache story is about what your chosen provider uses to specify build steps, etc
- rmdashrfstar 6y agoUse sccache and an S3 bucket
- alkonaut 6y ago> it had 92 dependencies, including transitive dependencies, and took 5 minutes and 38 seconds to build. What is it that triggers such a "full" build though? Obviously in some CI scenarios you might start from scratch, but in an edit/compile cycle, you'd never be hitting those 5 minutes, correct?
- Measter 6y agoTypically in an edit/compile cycle you'd only compile your dependencies once for each build type (check, debug, release). Unless you change a dependency's feature flag, the compiler will just re-use what's already been compiled. If you clean your build folder, things will need to be rebuilt from scratch. Likewise if you change your compiler version.
- alkonaut 6y agoIt seems a CI system should be able to work the same way and cache the dependencies just like a local build. If it does, then the long compile times are almost never encountered for neither developers nor CI. So are they really problematic?