9 ms·

I had rare occasion to compile a Rust application recently because a prebuilt binary didn’t exist for my platform. It took just over two hours! Coming from prim

by donatj 5y ago

I had rare occasion to compile a Rust application recently because a prebuilt binary didn’t exist for my platform. It took just over two hours! Coming from primarily Go development recently, I was astonished. How do you get anything done when the cost of testing an idea is several hours?! I’d feel like the sunk cost fallacy would kick in for even the most lackluster of changes just because you took the time to try it.

swiftcoder 5y ago
What on earth did you compile? I can't recall ever having compiled a rust project that took more than 10 minutes from scratch. I do have memories of the days when compiling Chromium or AOSP was a 4 hour battle, though :)
- deleted 5y ago
  [deleted]
- jynelson 5y ago
  The rust compiler itself can take as long as 2 hours on a modern laptop if you do a full stage 2 build. I don't think that's what the top poster was talking about though, it sounded like they had a rust toolchain already.
  - steveklabnik 5y ago
    (You know this, but for anyone else reading, don't forget that that doing this compiles LLVM, which is in of itself a massive C++ project, as well as compiling the compiler itself multiple times. That's part of why it would be such an outlier.)
    - swiftcoder 5y ago
      Right. Nested C/C++ dependencies are the only case I can think of where drastically-long compile times are common.
      - jandrese 5y ago
        Or maybe someone included Boost? Sometimes it is just that someone drank the Kool-aid and made everything a template. Then split their code across hundreds of tiny files and did the one big massive include at the top of every file that pulls in everything.
JD557 5y ago
> How do you get anything done when the cost of testing an idea is several hours?! I haven't used rust in production, but usually you can just use `cargo check` to only run the typecheck. Using `cargo build` is also usually much faster than `cargo build --release`. Having said that, at least in my toy projects it was not uncommon to have to compile using `--release` to run the tests (since otherwise the tests would take forever).
- littlestymaar 5y ago
  > Having said that, at least in my toy projects it was not uncommon to have to compile using `--release` to run the tests (since otherwise the tests would take forever). Maybe you're already aware of that, but if the reason why your tests are slow is not “your code not being optimized” but “your dependencies not being optimized” then Cargo profile override[1] can save your life. You can have one specific dependency (or all of them) being build in release mode even when you're building you own code in debug mode. The first development build is going to be quite slow, but after that, you're going to have the best of both worlds. [1]: https://doc.rust-lang.org/cargo/reference/profiles.html#overrides https://doc.rust-lang.org/cargo/reference/profiles.html#over...
yakubin 5y ago
Although Rust compile times are rather bad, IME such astronomical compile times are rather a result of terrible code quality, a lot of code doing essentially nothing, just gluing glue, to the point where it's normal to get a 50-60 frames deep stack trace with most functions being named such that it would suggest that all of them do basically the same. Then you look at their bodies and... yep... doing the same, aka nothing. With Rust, procedural macros add a lot of overhead when compiling, so I try to avoid them. At work, we have a C++ project which, without using distributed builds with distributed ccache, running on powerful cloud computers, takes 3h+. Code quality is adequate. Debugging anything is a nightmare in such a codebase.
- spoiler 5y ago
  While I agree that the compile times are bizarre, I don't think we can jump to conclusions such as "such astronomical compile times are rather a result of terrible code quality" without knowing more about what's being compiled and under what conditions it was being compiled!
wongarsu 5y ago
In my experience, Rust compilation times aren't that different from what I'm used to in the JavaScript world. Initial compilation takes a minute or two (comparable to the runtime of `npm install` on Windows), when you make changes you usually get incremental compilation within a few seconds. I guess you can have projects that are just huge and/or run into some pathological case that increases compile time a lot (just like with C++), but for any subsequent compilations you should get very fast incremental builds.
IshKebab 5y ago
Were you compiling Servo on a Raspberry Pi or something? 2 hours is ridiculous even for Rust. > How do you get anything done when the cost of testing an idea is several hours? Incremental compilation. Depending on the project you can get the edit compile cycle down to somewhere between 1 second and 5 minutes. It's nowhere near as fast as Go but it's not like anyone is actually making edits then waiting 2 hours for them to compile.
yosefk 5y ago
I doubt your compile times were due to pattern matching which TFA demonstrates to be NP-complete, any more than C++'s compile-times are due to the undecidability of template metaprogramming. (Though I guess one might argue that slow compile times on typical code and rarely-used computationally hard features have the same root cause of not treating fast compile times as a top priority.)
- tialaramex 5y ago
  Right, and C++ got a lot of its compiler performance for large projects by making this an embarrassingly parallel problem even though the consequences are negative for the main consumers of the code (people, not compilers). One cost of being embarrassingly parallel is the One Definition Rule. If we can compile N different code units in parallel, but they're all allowed to define things in the same namespace, obviously those definitions might contradict each other and we wouldn't notice. So, the C++ language explicitly forbids this, knowing you'll probably do it anyway, at least by mistake. If (when) you do, that isn't a valid C++ program, but the compiler isn't expected to produce any diagnostic (warning or error). So, you get a binary, but the language doesn't care what that binary does. Maybe it does exactly what you expected. Maybe it does almost exactly what you expected. If not too bad, the One Definition Rule means your compiler was fast and that's what matters.
  - sakex 5y ago
    I usually fails at link time doesn't it?
    - bluGill 5y ago
      No. I have intentionally violated the one definition rule and nothing broke at all. In my case I wrote a second timer, which had hooks my unit test system could use to advance time without having to inject timers. It will fail at link time if you link everything into the same library. Even here there is an escape: there are ways to mark something as a weak symbol and than the linker won't complain about more than one definition. See your OS documentation for how this works on your implementation. (though don't be surprised if the documentation is wrong...)
ekidd 5y ago
> How do you get anything done when the cost of testing an idea is several hours?! My Rust compile times are usually between 10 and 30 seconds. Some tricks that help: - Get a good development machine. I prefer a Dell Precision laptop from the last couple of years, with plenty of cores. A 5-year old laptop with 2 cores will be a lot slower. - Use Visual Studio Code and the rust-analyzer plugin. This will give you feedback in the editor while you're working. - Rely on the type system to identify most problems before ever generating code. - Minimize the use of slow-to-compile libraries that rely on complex generic types. Diesel, for example, is a great library but it's slow to compile. - Install cargo-watch, and use it to automatically re-run the tests. Also, remember that incrementally re-compiling a single crate in debug mode will be much faster than building an optimized application and all its dependencies from scratch. TL;Dr: Rust compilation times are less than ideal, but with a fast laptop and rust-analyzer, it's possible to spend very little time actually waiting for the compiler.
- remix2000 5y ago
  Just get a new machine for a couple zillion bucks, why haven’t we thought of it earlier? :^)
  - ekidd 5y ago
    Yes. If you are a professional developer who gets paid to work on large Rust projects all day long, then a US$2500 high-end laptop will pay for itself many times over. (I'm assuming US or European salaries here.) I've gotten by with much less for personal projects. But Rust will make very efficient use of (say) an 8-core i9 if you have one.
  - isaacimagine 5y ago
    As a developer who uses Rust a bunch (work & personal), this is what I ended up having to do. I'm joking, but this one simple trick can cut compile times in half.
  - edflsafoiewq 5y ago
    Yeah, can really see that "empowering everyone" motto coming into play.
tester34 5y ago
jesus christ rust's compilation times are as terrible as c++'s?
- kzrdude 5y ago
  It's the same ballpark
  - tester34 5y ago
    I think we must do better AAA games need less resources than those tree walkers
  - spoiler 5y ago
    In my experience (C++ dev at work) C++ generally feels slower than Rust. Edit: I quickly realised, this might only because of Rust's—in my opinion—superior type system and tooling, so maybe it just requires less waiting for compilations than when working C++. Edit 2: I've never actually measured Rust compilation times, but even for large-ish graphical codebases (wezterm, alacritty, bevy games) I've compiled from scratch, it felt noticeably faster even then.
- okamiueru 5y ago
  I don't know about Him, but at least when working with C++ I feel the compile time is entirely up to deliberate tradeoffs. With better planning and more effort made with forward headers, better implementation boundaries w/pimpl like constructs, and otherwise linking, and avoiding meta-programming entirely, I've not found it to be an issue. Incremental C++ builds can be lightning fast, or it can be ridiculously slow. I've worked on large projects where change + incremental compile + test was less than 5 seconds. And I've worked with project where no effort was made where the same could take 10 minutes.
  - gpderetta 5y ago
    Keeping your C++ compilation times reasonable requires eternal vigilance (or a distributed system with aggressive caching).
tinco 5y ago
Go is geared towards productivity. If you have a problem you can solve adequately in Go, why not just do it in Go? Because they gained popularity roughly around the same time many people think they are competing languages, but they're really not. Ruby is my main programming language, if I need something to be fast and/or light weight I switch to Go, sacrificing some comfort. And if I need absolute lowest possible overhead, I switch to Rust, sacrificing a whole lot more comfort. Rust is more expressive than Go, but if your Go project is so large that it needs a stronger typesystem in my opinion you should switch to a language like C# or Scala.
- sesm 5y ago
  Go is geared towards productivity of specific people at specific environment: fresh grads at Google.
  - tinco 5y ago
    First time I used it, I had not written a line of Go prior, I rewrote our data ingestion gateway from Ruby to Go. It took me 1 week, and it had full feature parity. I don't think I had the GPA at university to land a fresh grad job at Google. Sure the tooling around it (besides the compiler itself) was the dumbest 21st century tooling I've seen (this was ~6 years ago), but it was quick to learn, quick to write, quick to compile and quick to run so who am I to complain.
    - sim_card_map 5y ago
      How was the tooling dumb?
      - tinco 5y ago
        It had this weird system where all your projects were supposed to be located in one directory structure that mapped to their git namespaces, it made no sense at all and was a pain to work around.
spoiler 5y ago
I'm really curious what you were compiling, and if you remember the machine specs (and was the machine under other load at the time). Two hours seems beyond ridiculous. Also, I'm assuming it was a full (as opposed to an incremental) release build? But even then, I've never had to wait for that long.
- lmilcin 5y ago
  Two hours is not beyond ridiculous. A lot of projects in the past compiled for way longer than that, days even. In 90s and early 2000s if you worked on a large desktop application (the size of Office or Photoshop) you could expect to spend a business day waiting for compilation results.
  - spoiler 5y ago
    Sorry, I should've clarified that I meant specifically meant in recent versions of Rust. I wasn't speaking in the general sense. I've experienced slow compilation times in C++ projects, although not of the same magnitude you describe (ie having to wait days)!
  - gpderetta 5y ago
    Around the turn of the millennium I remember recompiling, for some reason I don't remember, the whole of KDE. That took a bit...
eptcyka 5y ago
I've only had 2 hour build times when I was compiling a rust toolchain on an emulated ARM system running on x86.
spockz 5y ago
Compiling linkerd-proxy took me also north of an hour on an Intel 6850k. Most of the time was spent compiling dependencies though. I also compiled envoy proxy which also took more than an hour. Subsequent rust builds for linkerd-proxy were quite fast. For envoy most time was spent in Bazel itself. Probably because I just did a Bazel build instead of specifying a more specific target.
- mkj 5y ago
  Out of curiousity I tried linkerd2-proxy here on a i5-1145G7 (I guess similar total CPU? Less cores, laptop model, newer). Took 5 minutes but used ~6GB resident. Maybe yours was swapping a lot?
  - spockz 5y ago
    My previous comment didn’t get saved apparently. I can hardly imagine that it was swapping. The machine has 32g ram. Perhaps compiling on macOS is slower. Most of the time was spent on compiling dependencies though. I can imagine that would be faster if you already have those builds lying around.
adwn 5y ago
Compiling Servo, a web browser engine, takes 5-6 minutes for an optimized build from scratch on my 6C/12T machine. So no, 2 hour build times are not normal for all but the largest software projects.
- genghizkhan 5y ago
  Compiling paru, an AUR helper for Arch Linux, on the other hand, takes me around 3 - 4 minutes on a 4C/8T i5 for an optimised build. I think Rust compile times might just fall within a narrow range.
lmilcin 5y ago
I have worked for 3 years on a project where it took a whole week to get the code compiled, signed by an external company and deployed to the device so that I could see the results. I just learned to work without compiling for a long time. Over time my productivity increased and the number of bugs fell dramatically. Working this way requires you to really think about what you are doing, which is always a good idea. This was over a decade ago and now I work mostly on Java backends and I am happy that I typically spend days or even weeks without ever compiling the code and that it usually works the first time I run it. I can't think how I could get back. It looks really strange to me to observe other developers constantly compiling and running their code just to see if it works. It kinda looks as if they did not exactly understand what they are doing because if they did, they would be confident the implementation works. The only time when I actually run a lot of compile/execute iterations is when I actually don't know how something works. I typically do this to learn, and I typically use a separate toy project for this.
- random_kris 5y ago
  I am one of those devs that is always compiling and checking if their program is working. I am mainly working with java and still doing this. Less so than when working with JavaScript. But please teach me your ways hahha
  - lmilcin 5y ago
    Start by understanding that this compile/run process is a crutch. Rather than use your knowledge, experience and intelligence to predict if it works you resign yourself to just waiting to see. This is a problem for many reasons. One is that this may help you get something working, but with lack of deep understanding the outcome will likely be subpar if only because not all problems that you could have predicted will show themselves on execution. Another is that it basically shrinks the part of brain that is necessary for understanding and predicting behavior of your code (figuratively). Sort of like driving with GPS makes me helpless without it. Try to write larger stretches of code without compilation. Try to focus on modularizing your application so that you can reason about modules separately. This is always a good idea, but it is even more important when you need to be able to predict how something works without trying it. When you have finally compiled your code and it failed, do not immediately go to fix the problem. Try to spend a moment to learn from the failure and improve your process so that you minimize chance of this happening in the future. Ask yourself, what you could have done to prevent this problem from happening? Could you have specified some function or module better? Could you have simplified your code to be able to better reason about it? Would it help if you have spent a bit more time getting acquainted with this internal or external library before you decided to use it? From my experience, most of this comes down to following things: - defining your modules and APIs correctly -- badly defined modules make it difficult to predict the behavior, - finding simple solutions to problems -- complex solutions tend to make it difficult to predict behavior, - using only tools you understand, - only interacting with existing code after you have understood how it works (I typically at least look over the code that I plan to use), - thinking hygiene (make sure you base your work on hard facts and not beliefs), - refactoring, refactoring, refactoring -- first solution to the problem is rarely optimal. I write something that works and then immediately keep refactoring it removing any unnecessary complexity until I am satisfied. Don't leave refactoring for later -- when you have just written a piece of code it is easiest to change it. - as much as possible, writing your code in a way that it is not even allowed to produce wrong result. This is very large topic so I won't explain. There is a lot of techniques that you can research.
weavie 5y ago
Incremental compilation. Once you get the first build done, subsequent builds are much faster. Also a project is likely to be split up over multiple crates, so a lot of the changes you make will require building and testing just that one crate.
- runevault 5y ago
  Isn't incremental compile disabled currently? I thought they found a significant bug and so turned it off until they could validate a fix.
  - kibwen 5y ago
    The grandparent was building a binary utility, which means they were (hopefully!) building in release mode, which doesn't use incremental compilation by default in any version of the compiler. For debug builds, where incremental is usually on by default, it was disabled for 1.52.1 due to bugs, and then kept off in 1.53 out of an abundance of caution. It should be back on in 1.54.
    - runevault 5y ago
      okay that's what I thought (and the not turned back on yet officially was what I was trying to hint at with the abundance of caution).
stonemetal12 5y ago
If I had to guess it is because all of the dependencies had to be compiled too. In a normal Rust application you break it up into crates. Crates are the unit of compilation, and are only recompiled when something in the crate changes. In a "normal" developer flow you only touch 1 or two crates at a time so normal developer compile time would be a couple of minutes at most. Even for a new build on a machine most developers would never see the two hour compile time because they would have gotten precompiled dependencies.
- Nullabillity 5y ago
  > In a "normal" developer flow you only touch 1 or two crates at a time so normal developer compile time would be a couple of minutes at most. Obviously this depends on your computer, but generally incremental Rust builds should typically be on the order of seconds, not minutes. > Even for a new build on a machine most developers would never see the two hour compile time because they would have gotten precompiled dependencies. Cargo doesn't do precompiled dependencies (even between projects on the same machine).
Kaze404 5y ago
Compiling for production and compiling for development are two very different processes, with the latter benefiting from less optimizations and incremental compilation. It wouldn't take two hours to test an idea.
glennpratt 5y ago
I worked on a Rust project at work for several months. Compile times were not a significant issues for developers where it was generally cached. For example, repeatedly running a test only compiled a small amount of code. It was a bit slow in CI/CD build environments with no cache, but so are the k8s Go projects I work on (go mod pulling in the world). The only thing approaching 2 hours I've ever seen is building the compiler from scratch.
gameswithgo 5y ago
Short answer is it doesn’t normally take that long. You also don’t have the problem of cgo being slow or ever having to wait for the data race checker to run. initial compile has to download and compile all dependencies. after that compiling is incremental. still slower than go though
kibwen 5y ago
I suspect the machine you were using was swapping itself to death, because I've never experienced anything resembling that in Rust. I also presume you were compiling in release mode, which is good for producing extremely fast programs but not something I bother with in regular development (in contrast to Go, which has no distinction between debug and release optimization levels). > How do you get anything done. The vast majority of my workflow just involves seeing if my code typechecks, for which I don't even need to build the program (so no codegen, no linking). This I do very frequently, as a sanity check on my work. The command for this is `cargo check`. This takes less than a second for small projects, one to five seconds for medium projects, and one to twenty seconds for large projects.
- adwn 5y ago
  > I suspect the machine you were using was swapping itself to death Good point. I recently upgraded my machine to 64 GiB RAM, because 16 GiB filled up pretty quickly with parallel compilation, several VS Code/rust-analyzer instances and a couple of browser tabs open.
nicetryguy 5y ago
I'm pretty excited for the Linux kernel to take three weeks to build.
jackcviers3 5y ago
I get this question about large scala codebases too - clean build and test cycles for things on underpowered containers in cloud build pipelines are in the tens of minutes sometimes. One: my local dev machine has 16 gigs of memory and 16 cores. What takes the tiny docker container with 1 gig and 2 vcpus 30 minutes takes my computer about 1. Two: Incremental compilation and testOnly make build/test cycles maybe a second / twenty seconds max, and most of that is test runtime on complex property based tests. You just get by without a clean compile most of the time after you build the project the first time. And really, a lot of builds spend an inordinate amount of time just pulling in external dependencies (which are also cached on my local machine, but not on the containerized builds a lot of the time).

by donatj 5y ago

swiftcoder 5y ago

What on earth did you compile? I can't recall ever having compiled a rust project that took more than 10 minutes from scratch. I do have memories of the days when compiling Chromium or AOSP was a 4 hour battle, though :)

deleted 5y ago
[deleted]
jynelson 5y ago
The rust compiler itself can take as long as 2 hours on a modern laptop if you do a full stage 2 build. I don't think that's what the top poster was talking about though, it sounded like they had a rust toolchain already.
- steveklabnik 5y ago
  (You know this, but for anyone else reading, don't forget that that doing this compiles LLVM, which is in of itself a massive C++ project, as well as compiling the compiler itself multiple times. That's part of why it would be such an outlier.)
  - swiftcoder 5y ago
    Right. Nested C/C++ dependencies are the only case I can think of where drastically-long compile times are common.
    - jandrese 5y ago
      Or maybe someone included Boost? Sometimes it is just that someone drank the Kool-aid and made everything a template. Then split their code across hundreds of tiny files and did the one big massive include at the top of every file that pulls in everything.

JD557 5y ago

> How do you get anything done when the cost of testing an idea is several hours?! I haven't used rust in production, but usually you can just use `cargo check` to only run the typecheck. Using `cargo build` is also usually much faster than `cargo build --release`. Having said that, at least in my toy projects it was not uncommon to have to compile using `--release` to run the tests (since otherwise the tests would take forever).

littlestymaar 5y ago
> Having said that, at least in my toy projects it was not uncommon to have to compile using `--release` to run the tests (since otherwise the tests would take forever). Maybe you're already aware of that, but if the reason why your tests are slow is not “your code not being optimized” but “your dependencies not being optimized” then Cargo profile override[1] can save your life. You can have one specific dependency (or all of them) being build in release mode even when you're building you own code in debug mode. The first development build is going to be quite slow, but after that, you're going to have the best of both worlds. [1]: https://doc.rust-lang.org/cargo/reference/profiles.html#overrides https://doc.rust-lang.org/cargo/reference/profiles.html#over...

yakubin 5y ago

Although Rust compile times are rather bad, IME such astronomical compile times are rather a result of terrible code quality, a lot of code doing essentially nothing, just gluing glue, to the point where it's normal to get a 50-60 frames deep stack trace with most functions being named such that it would suggest that all of them do basically the same. Then you look at their bodies and... yep... doing the same, aka nothing. With Rust, procedural macros add a lot of overhead when compiling, so I try to avoid them. At work, we have a C++ project which, without using distributed builds with distributed ccache, running on powerful cloud computers, takes 3h+. Code quality is adequate. Debugging anything is a nightmare in such a codebase.

spoiler 5y ago
While I agree that the compile times are bizarre, I don't think we can jump to conclusions such as "such astronomical compile times are rather a result of terrible code quality" without knowing more about what's being compiled and under what conditions it was being compiled!

wongarsu 5y ago

In my experience, Rust compilation times aren't that different from what I'm used to in the JavaScript world. Initial compilation takes a minute or two (comparable to the runtime of `npm install` on Windows), when you make changes you usually get incremental compilation within a few seconds. I guess you can have projects that are just huge and/or run into some pathological case that increases compile time a lot (just like with C++), but for any subsequent compilations you should get very fast incremental builds.

IshKebab 5y ago

Were you compiling Servo on a Raspberry Pi or something? 2 hours is ridiculous even for Rust. > How do you get anything done when the cost of testing an idea is several hours? Incremental compilation. Depending on the project you can get the edit compile cycle down to somewhere between 1 second and 5 minutes. It's nowhere near as fast as Go but it's not like anyone is actually making edits then waiting 2 hours for them to compile.

yosefk 5y ago

I doubt your compile times were due to pattern matching which TFA demonstrates to be NP-complete, any more than C++'s compile-times are due to the undecidability of template metaprogramming. (Though I guess one might argue that slow compile times on typical code and rarely-used computationally hard features have the same root cause of not treating fast compile times as a top priority.)

tialaramex 5y ago
Right, and C++ got a lot of its compiler performance for large projects by making this an embarrassingly parallel problem even though the consequences are negative for the main consumers of the code (people, not compilers). One cost of being embarrassingly parallel is the One Definition Rule. If we can compile N different code units in parallel, but they're all allowed to define things in the same namespace, obviously those definitions might contradict each other and we wouldn't notice. So, the C++ language explicitly forbids this, knowing you'll probably do it anyway, at least by mistake. If (when) you do, that isn't a valid C++ program, but the compiler isn't expected to produce any diagnostic (warning or error). So, you get a binary, but the language doesn't care what that binary does. Maybe it does exactly what you expected. Maybe it does almost exactly what you expected. If not too bad, the One Definition Rule means your compiler was fast and that's what matters.
- sakex 5y ago
  I usually fails at link time doesn't it?
  - bluGill 5y ago
    No. I have intentionally violated the one definition rule and nothing broke at all. In my case I wrote a second timer, which had hooks my unit test system could use to advance time without having to inject timers. It will fail at link time if you link everything into the same library. Even here there is an escape: there are ways to mark something as a weak symbol and than the linker won't complain about more than one definition. See your OS documentation for how this works on your implementation. (though don't be surprised if the documentation is wrong...)

ekidd 5y ago

> How do you get anything done when the cost of testing an idea is several hours?! My Rust compile times are usually between 10 and 30 seconds. Some tricks that help: - Get a good development machine. I prefer a Dell Precision laptop from the last couple of years, with plenty of cores. A 5-year old laptop with 2 cores will be a lot slower. - Use Visual Studio Code and the rust-analyzer plugin. This will give you feedback in the editor while you're working. - Rely on the type system to identify most problems before ever generating code. - Minimize the use of slow-to-compile libraries that rely on complex generic types. Diesel, for example, is a great library but it's slow to compile. - Install cargo-watch, and use it to automatically re-run the tests. Also, remember that incrementally re-compiling a single crate in debug mode will be much faster than building an optimized application and all its dependencies from scratch. TL;Dr: Rust compilation times are less than ideal, but with a fast laptop and rust-analyzer, it's possible to spend very little time actually waiting for the compiler.

remix2000 5y ago
Just get a new machine for a couple zillion bucks, why haven’t we thought of it earlier? :^)
- ekidd 5y ago
  Yes. If you are a professional developer who gets paid to work on large Rust projects all day long, then a US$2500 high-end laptop will pay for itself many times over. (I'm assuming US or European salaries here.) I've gotten by with much less for personal projects. But Rust will make very efficient use of (say) an 8-core i9 if you have one.
- isaacimagine 5y ago
  As a developer who uses Rust a bunch (work & personal), this is what I ended up having to do. I'm joking, but this one simple trick can cut compile times in half.
- edflsafoiewq 5y ago
  Yeah, can really see that "empowering everyone" motto coming into play.

tester34 5y ago

jesus christ rust's compilation times are as terrible as c++'s?

kzrdude 5y ago
It's the same ballpark
- tester34 5y ago
  I think we must do better AAA games need less resources than those tree walkers
- spoiler 5y ago
  In my experience (C++ dev at work) C++ generally feels slower than Rust. Edit: I quickly realised, this might only because of Rust's—in my opinion—superior type system and tooling, so maybe it just requires less waiting for compilations than when working C++. Edit 2: I've never actually measured Rust compilation times, but even for large-ish graphical codebases (wezterm, alacritty, bevy games) I've compiled from scratch, it felt noticeably faster even then.
okamiueru 5y ago
I don't know about Him, but at least when working with C++ I feel the compile time is entirely up to deliberate tradeoffs. With better planning and more effort made with forward headers, better implementation boundaries w/pimpl like constructs, and otherwise linking, and avoiding meta-programming entirely, I've not found it to be an issue. Incremental C++ builds can be lightning fast, or it can be ridiculously slow. I've worked on large projects where change + incremental compile + test was less than 5 seconds. And I've worked with project where no effort was made where the same could take 10 minutes.
- gpderetta 5y ago
  Keeping your C++ compilation times reasonable requires eternal vigilance (or a distributed system with aggressive caching).

tinco 5y ago

Go is geared towards productivity. If you have a problem you can solve adequately in Go, why not just do it in Go? Because they gained popularity roughly around the same time many people think they are competing languages, but they're really not. Ruby is my main programming language, if I need something to be fast and/or light weight I switch to Go, sacrificing some comfort. And if I need absolute lowest possible overhead, I switch to Rust, sacrificing a whole lot more comfort. Rust is more expressive than Go, but if your Go project is so large that it needs a stronger typesystem in my opinion you should switch to a language like C# or Scala.

sesm 5y ago
Go is geared towards productivity of specific people at specific environment: fresh grads at Google.
- tinco 5y ago
  First time I used it, I had not written a line of Go prior, I rewrote our data ingestion gateway from Ruby to Go. It took me 1 week, and it had full feature parity. I don't think I had the GPA at university to land a fresh grad job at Google. Sure the tooling around it (besides the compiler itself) was the dumbest 21st century tooling I've seen (this was ~6 years ago), but it was quick to learn, quick to write, quick to compile and quick to run so who am I to complain.
  - sim_card_map 5y ago
    How was the tooling dumb?
    - tinco 5y ago
      It had this weird system where all your projects were supposed to be located in one directory structure that mapped to their git namespaces, it made no sense at all and was a pain to work around.

spoiler 5y ago

I'm really curious what you were compiling, and if you remember the machine specs (and was the machine under other load at the time). Two hours seems beyond ridiculous. Also, I'm assuming it was a full (as opposed to an incremental) release build? But even then, I've never had to wait for that long.

lmilcin 5y ago
Two hours is not beyond ridiculous. A lot of projects in the past compiled for way longer than that, days even. In 90s and early 2000s if you worked on a large desktop application (the size of Office or Photoshop) you could expect to spend a business day waiting for compilation results.
- spoiler 5y ago
  Sorry, I should've clarified that I meant specifically meant in recent versions of Rust. I wasn't speaking in the general sense. I've experienced slow compilation times in C++ projects, although not of the same magnitude you describe (ie having to wait days)!
- gpderetta 5y ago
  Around the turn of the millennium I remember recompiling, for some reason I don't remember, the whole of KDE. That took a bit...

eptcyka 5y ago

I've only had 2 hour build times when I was compiling a rust toolchain on an emulated ARM system running on x86.

spockz 5y ago

Compiling linkerd-proxy took me also north of an hour on an Intel 6850k. Most of the time was spent compiling dependencies though. I also compiled envoy proxy which also took more than an hour. Subsequent rust builds for linkerd-proxy were quite fast. For envoy most time was spent in Bazel itself. Probably because I just did a Bazel build instead of specifying a more specific target.

mkj 5y ago
Out of curiousity I tried linkerd2-proxy here on a i5-1145G7 (I guess similar total CPU? Less cores, laptop model, newer). Took 5 minutes but used ~6GB resident. Maybe yours was swapping a lot?
- spockz 5y ago
  My previous comment didn’t get saved apparently. I can hardly imagine that it was swapping. The machine has 32g ram. Perhaps compiling on macOS is slower. Most of the time was spent on compiling dependencies though. I can imagine that would be faster if you already have those builds lying around.

adwn 5y ago

Compiling Servo, a web browser engine, takes 5-6 minutes for an optimized build from scratch on my 6C/12T machine. So no, 2 hour build times are not normal for all but the largest software projects.

genghizkhan 5y ago
Compiling paru, an AUR helper for Arch Linux, on the other hand, takes me around 3 - 4 minutes on a 4C/8T i5 for an optimised build. I think Rust compile times might just fall within a narrow range.

lmilcin 5y ago

I have worked for 3 years on a project where it took a whole week to get the code compiled, signed by an external company and deployed to the device so that I could see the results. I just learned to work without compiling for a long time. Over time my productivity increased and the number of bugs fell dramatically. Working this way requires you to really think about what you are doing, which is always a good idea. This was over a decade ago and now I work mostly on Java backends and I am happy that I typically spend days or even weeks without ever compiling the code and that it usually works the first time I run it. I can't think how I could get back. It looks really strange to me to observe other developers constantly compiling and running their code just to see if it works. It kinda looks as if they did not exactly understand what they are doing because if they did, they would be confident the implementation works. The only time when I actually run a lot of compile/execute iterations is when I actually don't know how something works. I typically do this to learn, and I typically use a separate toy project for this.

random_kris 5y ago
I am one of those devs that is always compiling and checking if their program is working. I am mainly working with java and still doing this. Less so than when working with JavaScript. But please teach me your ways hahha
- lmilcin 5y ago
  Start by understanding that this compile/run process is a crutch. Rather than use your knowledge, experience and intelligence to predict if it works you resign yourself to just waiting to see. This is a problem for many reasons. One is that this may help you get something working, but with lack of deep understanding the outcome will likely be subpar if only because not all problems that you could have predicted will show themselves on execution. Another is that it basically shrinks the part of brain that is necessary for understanding and predicting behavior of your code (figuratively). Sort of like driving with GPS makes me helpless without it. Try to write larger stretches of code without compilation. Try to focus on modularizing your application so that you can reason about modules separately. This is always a good idea, but it is even more important when you need to be able to predict how something works without trying it. When you have finally compiled your code and it failed, do not immediately go to fix the problem. Try to spend a moment to learn from the failure and improve your process so that you minimize chance of this happening in the future. Ask yourself, what you could have done to prevent this problem from happening? Could you have specified some function or module better? Could you have simplified your code to be able to better reason about it? Would it help if you have spent a bit more time getting acquainted with this internal or external library before you decided to use it? From my experience, most of this comes down to following things: - defining your modules and APIs correctly -- badly defined modules make it difficult to predict the behavior, - finding simple solutions to problems -- complex solutions tend to make it difficult to predict behavior, - using only tools you understand, - only interacting with existing code after you have understood how it works (I typically at least look over the code that I plan to use), - thinking hygiene (make sure you base your work on hard facts and not beliefs), - refactoring, refactoring, refactoring -- first solution to the problem is rarely optimal. I write something that works and then immediately keep refactoring it removing any unnecessary complexity until I am satisfied. Don't leave refactoring for later -- when you have just written a piece of code it is easiest to change it. - as much as possible, writing your code in a way that it is not even allowed to produce wrong result. This is very large topic so I won't explain. There is a lot of techniques that you can research.

weavie 5y ago

Incremental compilation. Once you get the first build done, subsequent builds are much faster. Also a project is likely to be split up over multiple crates, so a lot of the changes you make will require building and testing just that one crate.

runevault 5y ago
Isn't incremental compile disabled currently? I thought they found a significant bug and so turned it off until they could validate a fix.
- kibwen 5y ago
  The grandparent was building a binary utility, which means they were (hopefully!) building in release mode, which doesn't use incremental compilation by default in any version of the compiler. For debug builds, where incremental is usually on by default, it was disabled for 1.52.1 due to bugs, and then kept off in 1.53 out of an abundance of caution. It should be back on in 1.54.
  - runevault 5y ago
    okay that's what I thought (and the not turned back on yet officially was what I was trying to hint at with the abundance of caution).

stonemetal12 5y ago

If I had to guess it is because all of the dependencies had to be compiled too. In a normal Rust application you break it up into crates. Crates are the unit of compilation, and are only recompiled when something in the crate changes. In a "normal" developer flow you only touch 1 or two crates at a time so normal developer compile time would be a couple of minutes at most. Even for a new build on a machine most developers would never see the two hour compile time because they would have gotten precompiled dependencies.

Nullabillity 5y ago
> In a "normal" developer flow you only touch 1 or two crates at a time so normal developer compile time would be a couple of minutes at most. Obviously this depends on your computer, but generally incremental Rust builds should typically be on the order of seconds, not minutes. > Even for a new build on a machine most developers would never see the two hour compile time because they would have gotten precompiled dependencies. Cargo doesn't do precompiled dependencies (even between projects on the same machine).

Kaze404 5y ago

Compiling for production and compiling for development are two very different processes, with the latter benefiting from less optimizations and incremental compilation. It wouldn't take two hours to test an idea.

glennpratt 5y ago

I worked on a Rust project at work for several months. Compile times were not a significant issues for developers where it was generally cached. For example, repeatedly running a test only compiled a small amount of code. It was a bit slow in CI/CD build environments with no cache, but so are the k8s Go projects I work on (go mod pulling in the world). The only thing approaching 2 hours I've ever seen is building the compiler from scratch.

gameswithgo 5y ago

Short answer is it doesn’t normally take that long. You also don’t have the problem of cgo being slow or ever having to wait for the data race checker to run. initial compile has to download and compile all dependencies. after that compiling is incremental. still slower than go though

kibwen 5y ago

I suspect the machine you were using was swapping itself to death, because I've never experienced anything resembling that in Rust. I also presume you were compiling in release mode, which is good for producing extremely fast programs but not something I bother with in regular development (in contrast to Go, which has no distinction between debug and release optimization levels). > How do you get anything done. The vast majority of my workflow just involves seeing if my code typechecks, for which I don't even need to build the program (so no codegen, no linking). This I do very frequently, as a sanity check on my work. The command for this is `cargo check`. This takes less than a second for small projects, one to five seconds for medium projects, and one to twenty seconds for large projects.

adwn 5y ago
> I suspect the machine you were using was swapping itself to death Good point. I recently upgraded my machine to 64 GiB RAM, because 16 GiB filled up pretty quickly with parallel compilation, several VS Code/rust-analyzer instances and a couple of browser tabs open.

nicetryguy 5y ago

I'm pretty excited for the Linux kernel to take three weeks to build.

jackcviers3 5y ago

I get this question about large scala codebases too - clean build and test cycles for things on underpowered containers in cloud build pipelines are in the tens of minutes sometimes. One: my local dev machine has 16 gigs of memory and 16 cores. What takes the tiny docker container with 1 gig and 2 vcpus 30 minutes takes my computer about 1. Two: Incremental compilation and testOnly make build/test cycles maybe a second / twenty seconds max, and most of that is test runtime on complex property based tests. You just get by without a clean compile most of the time after you build the project the first time. And really, a lot of builds spend an inordinate amount of time just pulling in external dependencies (which are also cached on my local machine, but not on the containerized builds a lot of the time).