23 ms·

A practical comparison of build and test speed between C++ and Rust

314 points by cod1r 4y ago

deleted 4y ago
[deleted]
vegerot 4y ago
Thank you for finally talking about this! I appreciate it, thanks
taspeotis 4y ago
http://webcache.googleusercontent.com/search?q=cache:qTP19KUQXF0J:https://quick-lint-js.com/blog/cpp-vs-rust-build-times/&hl=en&gl=au&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:qTP19KU...
neonate 4y ago
https://archive.ph/f5aiI https://archive.ph/f5aiI
azakai 4y ago
Very well written, and I like that it ends on this note: > Looking at my hypotheses, I was wrong on all counts The author expected some things, and had good reasons to, then did a lot of serious work, and ended up finding out somewhat different things. Impressive and honest exploration of a complex topic that is quite hard to measure.
- vasvir 4y ago
  Totally agree. I would like to stress the "serious work" point. He tried a lot of configurations and kept track of the results. This is not easy as it sounds since most people lack the discipline (me included). I wish more papers were written like this with clear separation of hypothesis, experimental results and conclusions. Also the clarity of presentation was superb again not an easy task. Extra bonus points for the intellectual honesty.
  - brabel 4y ago
    Writing a blog post like this and running benchmarks on several combinations of OS, language and compiler options and different tools takes so much effort and time. We should be very thankful to the people who do this. I've done it a couple of times myself... but it was so much more effort than I thought, literally weeks non-stop, that I don't think I will be doing it again any time soon.
    - strager 4y ago
      Thank you for your kind words! I appreciate your empathy. This project took about 7 weeks (including trimming, porting, benchmarking, optimizing, and finally writing the article).
- deleted 4y ago
  [deleted]
oslac 4y ago
Does this article account for the fact that a rustc compilation does more things than simple C++ default compilation, where you need to run address sanitizers etc. to do the same as rustc?
- strager 4y ago
  No. If I was going to run sanitizers with the C++ code, for a fair comparison, I would also run the Rust code with Miri. (C++ sanitizers check unsafe code, but without Miri, rustc mostly checks only safe code.)
  - vegerot 4y ago
    What about the other way around? Trying to cripple rustc until its features are closer to clang's? For example, removing some of the sanitizers or borrow checker in rustc?
    - strager 4y ago
      That's a great idea! Unfortunately, I think there's no way to disable rustc's borrow checking or other safety features.
  - insanitybit 4y ago
    Just fwiw you can run rust code with sanitizers, no need for miri, which works differently.
    - strager 4y ago
      Oh, I did not know this.
- jupp0r 4y ago
  Compilation doesn't necessarily need strict checks, it's really just translating source code into machine code. It happens to be the case that we mostly do both using the same tools because compilers need to have a good understanding of type systems to do their job well, but it's not a strict requirement. A great counter example is esbuild, which will compile TypeScript but not perform actual type checks for performance reasons. This is possible because TypeScript types have no representation at runtime. and are really only a development tool.
pjmlp 4y ago
Thing is, not every C++ project is full of template metaprogramming, and at least on the corporate world it is quite common to use binary libraries, which NuGET, conan, vcpkg, and OS package managers support. Also at least in VC++, modules are already a thing, no need to keep parsing template code all the time. Doing a C++23 import std (which imports the whole standard library) takes a fraction of the time of #include <iostream> in VC++. So a clean build out from repo means compiling only our own code, not the whole world as in Rust. Still looking forward to binary libraries support in cargo (well one can dream).
- forrestthewoods 4y ago
  > Doing a C++23 import std It’ll be nice when this can be reliably used. According to cppreference MSVC has partial support and other compilers have zero support. I’m not sure modules will be in widespread use before 2029. https://en.cppreference.com/w/cpp/compiler_support https://en.cppreference.com/w/cpp/compiler_support
  - pjmlp 4y ago
    VC++ support is good enough that all my C++ hobby coding is done with modules, not everything we code must be cross-platform from the get go. GCC is getting there. clang, well they seem stuck with their own initial modules implementation (module maps based), and now with Apple and Google out of the picture, the various compiler vendors that benefit from MIT license don't seem that eager to contribute to upstream other than LLVM. Unfortunely everyone else seem to still be catching up with C++14 and C++17, depending on the ecosystem.
    - forrestthewoods 4y ago
      > not everything we code must be cross-platform from the get go. No, but the code I write for my job does have to be cross-platform from day 0. And that's the code base that is large enough where module improvements to compile time would be most useful! Hobby projects are small enough it's not super important. So C++20 modules don't help me and probably won't until 2029! To my great dismay.
habibur 4y ago
The site's going up and down. Here's Conclusion : //// Are compilation times a problem with Rust? Yes. Are build times as bad with Rust as with C++? Yes. Looking at my hypotheses, I was wrong on all counts: The Rust port had more lines than the C++ version, not fewer. For full builds, compared to Rust, C++ builds took about the same amount of time (17k SLOC) or took less time (100k+ SLOC), not longer. For incremental builds, compared to C++, Rust builds were sometimes shorter and sometimes longer (17k SLOC) or much longer (100k+ SLOC), not always longer.
- deleted 4y ago
  [deleted]
- vegerot 4y ago
  or just https://smmry.com/https://quick-lint-js.com/blog/cpp-vs-rust-build-times/#&SM_LENGTH=14 https://smmry.com/https://quick-lint-js.com/blog/cpp-vs-rust... like I did
berkut 4y ago
In my experience of building large C++ projects on machines with >= 64 threads (especially if using ccache or something), linking is often the bottleneck to incremental compilation as opposed to cpu-bound compilation. Some of our dev builds actually are split up into separate .so files which is a bit slower at execution time, but means linking is a lot faster, especially when you want full debug info...
- jupp0r 4y ago
  Chrome is doing the exact same thing for the same reason. Dev builds are dynamically linked while release builds are statically linked. Works quite well, it's a great technique.
  - pjmlp 4y ago
    It is quite common in Windows, either produce lib or dlls for each set of main modules. In the old days it used to be quite common on UNIX world as well, no idea why "modern" Linux seems to have forgotten about this approach and always builds everything as a single project unit.
    - bombolo 4y ago
      Because google decided we all want one big binary that contains the go runtime and weights at least 100MB
      - mwcampbell 4y ago
        I have several Go-based binaries in my $HOME/bin, and not a single one is 100 MB.
        vegerot 4y ago
        And all of them combined compile faster than either of these implementations!
        4y ago
29athrowaway 4y ago
C++20 modules should help with the build speed... But 3 years later, many systems don't support them. (This is mentioned in the article)
- fooker 4y ago
  Modules help with build speed of small projects where compiling standard library headers are a significant part of the compilation time. For well organized large projects, modules would be marginally better at best.
  - college_physics 4y ago
    This could still be a dramatic improvement for numerical work that requires a lot of exploratory compilations and iterative style (i.e. steal back some market share from python)
- saurik 4y ago
  Is there any new reason to believe that? What I had seen--and which I would have thought would be obvious--is that the C++ module design only helps at lower levels of parallelism and actually slows down the build dramatically at the higher levels of parallelism used by people who actually care about build performance; this is apparently because the inherent design of modules causes the same kind of pervasive coupling seen with Rust, destroying the inherent scalability of the classic C++ separate compilation model. https://vector-of-bool.github.io/2019/01/27/modules-doa.html https://vector-of-bool.github.io/2019/01/27/modules-doa.html https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1441r1.pdf https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p14... > Expectedly, as hardware parallelism increases, headers’ lead over modules becomes more and more pronounced. There is also a relationship between the DAG-depth (i.e. The length of the chain of modules that import each other). As this depth increases, modules grow slower and slower, while headers remain fairly constant for even “extreme” depths approaching 300).
  - pjmlp 4y ago
    Old outdated news. In C++23 it takes less time to import the whole standard library than a single major include like iostreams. Currently only available in VC++ preview, though.
choeger 4y ago
I have a hunch that rusts global compilation model (as opposed to C++ separate compilation, as crippled as it is in practice due to header-only libs and link times) should always lead to C++ scaling better with the size of the project. Simply put: Rust doesn't offer an abstraction past type checking. Rust could potentially introduce its equivalent of PCH, or maybe it already does so in some internal cache, but as long as it uses monomorphization and no PhD student comes up with a clever trick to combine that with separate compilation it will scale worse. That being said, maybe a more fair comparison wouldn't involve polymorphism in the rust code? C++ doesn't really have polymorphism, it uses templates for a similar feature. So what happens if someone replaces rusts polymorphic functions with macros?
- jimbob45 4y ago
  What do you mean by abstraction beyond type checking? I’m asking to educate myself.
  - choeger 4y ago
    "Beyond" in the sense of "after". Languages that support separate compilation usually have some sort of artifacts that do not need to be type-checked anymore. Say, you compile an Ocaml module. The result is a compiled (bytecode or native code) file that represents the implementation of the module and doesn't need to be generated ever again. Essentially, the compiler abstracts over a module after it dealt with it. You could the If you do the same in rust, you need access to the module's implementation for monomorphization. You might skip the type checking of a function of the implementation or even cache the generated code for a monomorphization as optimizations, but generally you have to treat every function as often as it is used. So the best rust could do is akin to C++ precompiled headers whereas a language like OCaml can simply compile every module exactly once. Note: OCaml pays for this elegance. Its uniform object representation is less efficient than what rust does. It's a trade-off that the rust designers consciously made, I think.
    - scaredginger 4y ago
      What you've said here is mostly correct. The key part you've omitted is that monomorphization is opt-in. One could write Rust like C if they wanted to; they would have to be insane, but they could
fooker 4y ago
Interesting results, I was expecting incremental Rust to be significantly faster than incremental C++ builds. Apparently not. This is something Rust folks should strive to solve as a differentiator, because there is no realistic way of making incremental C++ builds faster.
- pjmlp 4y ago
  The incremental compiler and linker in VC++ alongside modules seems realistic enough to me.
- strager 4y ago
  > This is something Rust folks should strive to solve as a differentiator, because there is no realistic way of making incremental C++ builds faster. I think C++20 modules are the promised golden ticket. It's 2023 and I still can't use them on Linux, though.
fouronnes3 4y ago
An interesting thought is that we really ought to not compare rust with just GCC, but GCC + clang-tidy, since you get so much more static analysis by default in rust compilation than C++.
- strager 4y ago
  > An interesting thought is that we really ought to not compare rust with just GCC, but GCC + clang-tidy Nitpick: The article compares Clang, not GCC. (GCC is mentioned only to show how much faster Clang is.) > you get so much more static analysis by default in rust compilation than C++. I care about the build-test cycle, so adding static analysis is against my goals. I would minimize the analyses done by rustc if I could.
misja111 4y ago
As a Scala developer, I can only dream about these build times. 1.8s to build a 17K LOC project .. This would take minutes for my similarly sized Scala project.
- strager 4y ago
  I'm sorry to hear that! Did you try upgrading your hardware? It really helps.
- odersky 4y ago
  This looks completely out of the ordinary. With a warm compiler, on a 4 year old Macbook Pro, I get 2000-4000 lines/sec. I.e. 4-8 seconds for your project. Unless you do some very involved typelevel or meta-programming stuff that's what you should expect to see.
  - misja111 4y ago
    Yes my project does contain a bit of typelevel stuff (Cats Effect). And it's Scala 2.13, so not the latest 3.x which might compile a bit faster. Also to be fair, I was referring to the time taken by sbt compile, which I suppose does more than just invoking scalac. But still the time reported by sbt's Compile/ Compile incremental is over 200s.
    - gpderetta 4y ago
      to be fair, C+ header only libraries full of advanced template metaprogramming will also slow down compilation by a few order of magnitude.
      - bluGill 4y ago
        I'm hopeful modules will come fast, they seem to fix that issue. Modules are already in C++20, but most compilers don't support them well enough yet.
        4y ago
insanitybit 4y ago
I guess my question is - what is to be done? Even C++ is quite slow to compile. If all rustc development stopped and there was a years-long effort to write it "fast", what sort of improvements could we see?
- vegerot 4y ago
  They should rewrite it in Rust
  - xnickb 4y ago
    It is already in rust. Which is a major problem for anyone trying to bootstrap rust for packages.
- strager 4y ago
  Good question! Based on my profiling, it seems like the code gen backend (LLVM) is a big offender. A code generator optimized for build times would be nice. (Cranelift seems to be a failure in that regard.) C++ has an explicit template instantiation feature which helps reduce backend time. I wonder if this can be done in Rust too. Maybe this is what -Zshare-generics=y is for?
  - Bedon292 4y ago
    I don't know enough C++ or Rust to judge but is it possible that they are optimized around Rust that isn't C++ style? I know I have seen articles about other languages where some very minor changes make a huge difference because the optimizations in place didn't understand what was happening and had to work harder. And did you try any of the optimization strategies again at 24x? I would be curious if there are differences there.
    - strager 4y ago
      > I don't know enough C++ or Rust to judge but is it possible that they are optimized around Rust that isn't C++ style? I know I have seen articles about other languages where some very minor changes make a huge difference because the optimizations in place didn't understand what was happening and had to work harder. Are you talking about run-time optimizations? My article is focused on build times. I don't think what you mentioned applies to build times. I could be wrong, though. (There are certainly compile time traps you can fall into, but I wouldn't know what those are in Rust.) > And did you try any of the optimization strategies again at 24x? I would be curious if there are differences there. No, I did not. That's a good suggestion. But I was pretty tired of this project by the time I made the scaling benchmarks. xD
ycui1986 4y ago
the author should talk to an FPGA engineer about build/compile time. That will make any computer programming language look like lightning speed.
- Gigachad 4y ago
  Or talk to a web developer.
- strager 4y ago
  No, I'll stay far away from them, thanks. ;P
charcircuit 4y ago
This isn't fair. The Rust version was broken up into multiple crates, but the C++ version was not broken up into multiple libraries.
- strager 4y ago
  I also benchmarked the Rust code with a monolithic crate. It built slower. https://quick-lint-js.com/blog/cpp-vs-rust-build-times/#optimizing-rust-with-different-layouts https://quick-lint-js.com/blog/cpp-vs-rust-build-times/#opti...
  - charcircuit 4y ago
    I was more curious about if there was a speed up on the C++ side. Especially if you use Bazel it's common in C++ to compile a lot of small libraries.
    - strager 4y ago
      Ah, good point. I hadn't considered this.
eska 4y ago
I don’t think the Rust code could be shorter than the C++ code when converting pretty much line by line, since that leads to C++-style Rust code. For example idiomatic Rust code can do with less defensive programming due to language features.
- strager 4y ago
  > I don’t think the Rust code could be shorter than the C++ code when converting pretty much line by line, since that leads to C++-style Rust code. I thought the opposite: header files (and CMakeLists.txt, which was included in the subtotal SLOC) artificially increase C++ SLOC. > For example idiomatic Rust code can do with less defensive programming due to language features. Can you show an example in the Rust code where this might be possible? https://github.com/quick-lint/cpp-vs-rust/tree/master/rust https://github.com/quick-lint/cpp-vs-rust/tree/master/rust
  - batmansmk 4y ago
    Just wanted to say thank you for sharing your code! Im not familiar with C++, so Im discovering the union, UnsafeCell, ManuallyDrop and all those concepts that are rare in the Rust projects I see everyday. Im learning a lot.
    - strager 4y ago
      You're welcome! But maybe my code (written by a Rust newbie) isn't the best to learn from... ;P
  - xiphias2 4y ago
    I just randomly looked at one part: linked vector. I know that the comparision here is a line by line comparison, but Rust gives a more stable interface for reusing libraries than C++. It would be a great experiment to use lots of external libraries. You might find that the code doesn't get slower (build times would probably go up though )
    - 4y ago
junon 4y ago
I don't really like these comparisons looking at these languages through the lens of commonality. The only thing the two have in common is that they're systems languages. C++ is a bloated mess that does exactly zero checking beyond type checking. Rust is a language that favors safety over build times and its serious users understand that. Rust is also developed completely in the open. C++'s Standards committee is rife with resignation letters, accusations of misconduct, and walled gardens - not to mention years between standards updates, each costing a ton of money to even look at. Yes, I understand. Rust has no standard. It has no stable ABI yet. Depending on who you are or what you're doing these may or may not be issues for you. Alas, comparing these two as though one should outperform the other is nonsense to me. Of course Rust is slower than C++. C++ has had decades of maturity and has enjoyed many contributors to its tooling. Rust has not, and its feature set (as in, the things the compiler needs to do) is much broader than any C++ compiler is specified to do. The end result here is this will only work to fuel more language wars and be used as a cheap weapon in further language debates. In my opinion, anyone who argued about which language is better than which other language has completely missed the point of computer science and programming.
- charcircuit 4y ago
  >only work to fuel more language wars Iteration time is an important part of the developer experience. The trade off of slower iteration time for improved safety is a real point that needs to be considered when choosing between C++ or Rust. This isn't something silly like curly braces vs whitespace.
  - vegerot 4y ago
    Agreed! …but while we’re at it curly gang all day
- strager 4y ago
  > I don't really like these comparisons looking at these languages through the lens of commonality. The only thing the two have in common is that they're systems languages. > > [...] In my opinion, anyone who argued about which language is better than which other language has completely missed the point of computer science and programming. When I start a new programming project, I need to choose a language. For this project, three years ago, I had three serious choices: * C * C++ * Rust I rejected C because I wanted the productivity boost from C++ or Rust. I rejected Rust because Rust was new to me (thus risky) but I was very familiar with C++. I don't see why it's a bad idea to compare languages—especially along specific dimensions, like run-time perf or build time or safety—when that's what us software engineers do whenever we start a new project.
t43562 4y ago
I think it's hard to draw absolute conclusions because small things can make a huge difference to build times. I worked on optimising lots of builds from Symbian to Android and other proprietary ones. C++ compiler choice is very important - gcc might seem slow but it is a speed demon compared to the old ARM compilers. There are a lot of tricks based on trying not to do the same work twice like pre-compiled headers which might help massively (since headers can be much bigger than your code) but quite often these are also very fragile because changing one #define in one central header file might invalidate everything. It's not that easy to be sure that you are truly handling all dependencies. Hence one tends to have to regularly build from scratch to be certain that everything is done properly. If we really could handle it all perfectly then nobody would ever bother to build from clean. It's also very important to be able to see how your build tool is scheduling tasks and making use of your CPUs - often some odd dependencies force most of your cores to sit idle while something gets done. There aren't good visualisation tools for this that aren't proprietary imo. I have cobbled together something for gmake which can output something you can visualise in chrome's profiler but it's crap compared to the best commercial tool I've used. You can remember the previous build times of various objects and use them to try to schedule large tasks as early as possible so they don't extend the build time by randomly getting started towards the end and then leaving the build going on and on just to get them finished. The structure of your code matters too - a simple optimisation for C/C++ is just to have much larger source files rather than splitting everything up into many small ones. This acts to reduce the number of repeated includes. If I was thinking about how to get fast builds it would force the language to change - header files would be banned and that means macros and the whole idea of preprocessing. That means a different approach to multi-platform support. Android benefits from having a java layer where you can pretty much build once and run anywhere and that is very good. Eventually, on one big project, we got to the point where it was the packaging that was the slowest component and that was being done by the CI system .... for no good reason other than organisational inertia. Another team was responsible and didn't want to take advice. Since we couldn't bring it into the build and use all our tricks to optimise and redesign it we hit a wall where all our efforts to improve build performance had little impact. The best tricks for improvement, of course, are the ones which always work and don't need a lot of maintenance to keep them functioning properly. I think we need builds to be integrated with source control. Getting new versions of source and new binaries is all far too "separate" a process. Checking in 5 files should result in a very accurate rebuild of only the dependencies and a subsequent checkout should be able to get me those binaries. Developers should never really have to build all of Android just to work on some small part and they shouldn't really have to do more than one checkout to get everything they need including binaries.
varajelle 4y ago
Regarding the sloc count, the default automated Rust formating tool is very eager to adds lot of lines by basically keeping only one word per line. Something I'm not a fan of, I must say.
- sph 4y ago
  It usually does that on iterator chains, which AFAIK do not exist as such in C++, so multiple operations would be expressed as multiple imperative statements. My C++ is rusty (no pun intended) but I struggle to imagine their variant of `vector.iter().map().collect()` to be as concise and fit in fewer than 4 lines. I wonder if OP's C++ port doesn't use iterators that much, and how idiomatic it is. EDIT: the code is not idiomatic at all.
  - strager 4y ago
    > I wonder if OP's C++ port doesn't use iterators that much, and how idiomatic it is. I think I only used iterators in places where there's no built-in function on slices like C++'s strchr and strspn. (I think Rust's str has these, but not [u8].) For example: C++: https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac5d0e676ead553a3610284d47a3e/cpp/src/quick-lint-js/i18n/locale.cpp#L38-L43 https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac... std::size_t length = std::strcspn(c, separators); if (c[length] == '\0') { return found_separator{.length = length, .which_separator = static_cast<std::size_t>(-1)}; } const char* separator = std::strchr(separators, c[length]); Rust: https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac5d0e676ead553a3610284d47a3e/rust/libs/i18n/src/locale.rs#L53-L64 https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac... match s .as_bytes() .iter() .position(|c: &u8| separators.contains(c)) { None => FoundSeparator { length: s.len(), which_separator: INVALID_WHICH_SEPARATOR, }, Some(length) => { let found_separator: u8 = unsafe { *s.as_bytes().get_unchecked(length) }; match separators.iter().position(|c: &u8| *c == found_separator) {
  - mgaunard 4y ago
    Of course it exists in C++, and has done since before Rust even existed. Syntax is usually `vector | map | collect`.
Dowwie 4y ago
Note how the OP uses an M1 laptop w/64MB of ram. If you want to compile a monolithic web application written in Rust within a reasonable amount of time, you need that much top-of-the-line hardware for your work. You're still waiting several minutes to compile, but compile times get even worse with older hardware running debian and mold/lld. CI test/build runs also take a long time to complete. Edits: 64GB, not 64MB. 32GB is really the minimum recommended
- zozbot234 4y ago
  > w/64MB of RAM. So, an Apple Newton in a laptop form factor?
  - Dowwie 4y ago
    Oh, I did just say MB. It's early here.
  - brabel 4y ago
    No one has corrected this yet, so I'll do it: it's 64GB of RAM! The Mac M1 Max the author used is an absolute beast of a machine. I have a M1 Pro (which is not as fast) and it's ridiculously faster (and quieter - it barely needs a fan) than my Linux Dell XPS13 and older Macbook Pro.
    - strager 4y ago
      It is a beast. I love my MacBook Pro!
- josephg 4y ago
  I don’t think 64mb of ram is necessary at all. And minutes? What are you compiling? How big is your codebase? Are you doing everything in docker or something? Rustc is slow but I’ve never seen it be that bad. I’ve been working on a rust project for a couple of years, currently sitting at about 40k loc. Incremental debug builds still only take a second or two. Full compiles take about 10 seconds.
sph 4y ago
Re: the SLOC count: the code is here https://github.com/quick-lint/cpp-vs-rust/tree/master/rust https://github.com/quick-lint/cpp-vs-rust/tree/master/rust That... doesn't seem to be idiomatic Rust at all. unsafe and passing pointers everywhere? Reimplementing vectors and linked lists. Also, makes use of a ton of macros. That's basically writing C++ in Rust.
- throw827474737 4y ago
  Read also the article about what he did? Could have said that in advance without looking at any code ;) (And further hint, that shouldn't be the significant hook here..)
- strager 4y ago
  > unsafe and passing pointers everywhere? Do you think this would impact Rust build times? > Reimplementing vectors and linked lists. Someone has to implement them. Note that they're also implemented in the C++ code, so the comparison is fair. > Also, makes use of a ton of macros. Can you show an example where the Rust code uses "a ton of macros"? The only place I can think of is the tests, but I thought macros were what you're supposed to use in Rust for assertions. If you're referring to proc macros, I thought those were very popular in Rust.
  - scaredginger 4y ago
    I gave up only a little beyond where they revealed that they ported line-by-line. I think this is an interesting experiment for those considering a Rust rewrite, but has no utility for a greenfield project. The scales are stacked strongly in favour of C++ here, without a doubt. Software designed for C++ has essentially zero chance of becoming idiomatic Rust without a huge amount of work, and tooling tends to be designed for idiomatic use
    - sebastianz 4y ago
      > I gave up only a little beyond where they revealed that they ported line-by-line. The commenter you are so dismissively replying to is the author. He did not "give up only a little beyond something" like you did, he made a serious effort that probably took hundreds of hours, and documented it. I am not sure why you are posting such a comment full of disrespect for his work, after reading a couple of paragraphs of his article.
pharmakom 4y ago
I think it’s acceptable for Rust compilation to end up slower than C++. After all, the compiler does a lot more work for you with the borrow checker and all. That doesn’t mean we shouldn’t strive to make it better though.
- strager 4y ago
  I'd wager that the C++ compiler does more work churning through #include-s than the Rust compiler does tracking borrows.
  - bluGill 4y ago
    C++20 modules is supposed to fix that. I wish I could get compilers that implemented them.
    - pjmlp 4y ago
      There you go, https://visualstudio.microsoft.com/vs/community/ https://visualstudio.microsoft.com/vs/community/ One example to get you started, https://github.com/pjmlp/RaytracingWeekend-CPP https://github.com/pjmlp/RaytracingWeekend-CPP
      - bluGill 4y ago
        I target linux with my C++.
        pjmlp 4y ago
        That wasn't part of the comment.
- Narishma 4y ago
  What makes you think it is slower because of the borrow checker?
  - 4y ago
_448 4y ago
Very thorough write-up, and an objective comparison. I would like to mention my subjective concerns for Rust. I hope the Rust community can think me as a canary who is from the C++ world. I am desperately looking for an alternative to C++, and I am a mid-weight C++ programmer. And here is what Rust could face down the line (remember, in early days C++ too was a grockable language and hence its popularity): 0. Rust's C++ problem: Learning curve. From the get go if there is murmur that Rust has a learning curve then think what will happen when Rust reaches the maturity of C++. 1. Rust's NPM problem: supply-chain. If I decide to use npm, and use the npm toolchain to download a project then to my horror it downloads god knows what. Lots of dependency hierarchies, some of them have not even reached 1.0. This raises the concern for supply-chain attacks. Rust with the aim of "boosting developer productivity" just decided to follow the path of all those who are susceptible to supply-chain attacks. 2. Rust's Haskell problem. Haskell is a very beautiful language. Even then it has not reached the level of adoption of other popular languages. It has nothing to do with the language, but some in the community who have a tendency to project a coolness level by saying something like "A monad is just a monoid in the category of endofunctors". What that ultimately does is alienate lot of software developers. Early Rust evangelist tried very hard to project Rust as having a very welcoming community, but if acronyms and terms from category theory and lambda calculus, that are alien to most developers, are thrown around without taking into consideration the larger development community then I fear that Rust will go the Haskell or OCaml way i.e it will become a niche language with small community. Just 2cents from an average everyday software developer.
- trenchgun 4y ago
  I can understand that category theory is alien to most developers, but lambda calculus? Isn't it similarly basic knowledge in CS as Turing machines, finite state automata etc?
  - donatzsky 4y ago
    Why do you assume that all developers have studied CS?
  - alltheworlds 4y ago
the8472 4y ago
A recent change[0] on nightly rustc might help with incremental builds. And for repeated clean + full build cycles there's sccache[1]. [0] https://github.com/rust-lang/rust/pull/84762 https://github.com/rust-lang/rust/pull/84762 [1] https://github.com/mozilla/sccache https://github.com/mozilla/sccache
- strager 4y ago
  > A recent change[0] on nightly rustc might help with incremental builds. I tested with rustc Git commit c7572670a1302f5c7e245d069200e22da9df0316, which (I think) includes that change. > And for repeated clean + full build cycles there's sccache[1]. You're right. I included full builds in the article because almost-full builds happen a lot in C++ (after common certain header files, or if you think the build system broke something). I imagine almost-full builds rarely happen when working in Rust though, so maybe I should have deemphasized my full-build benchmarks.
  - nicoburns 4y ago
    Yeah, I pretty much only do a clean build when I update the compiler. Or when I’m tweaking the build configuration itself.
  - laund 4y ago
    Rust by default only does a full build really rarely. Ive gone through days, even weeks, of working on a project without a full build. Partial builds are of course way faster, especially if you use many dependencies (i know you don't). I mainly work with the bevy game engine in Rust, which has a lot of dependencies. Even if i don't use its dylib feature, i get 2-3s compiles. And that's on a project with multiple hundreds of thousands LoC when you include dependencies. With dylib, it goes down to 0.5-1 second builds. If your main conclusion is based on full builds, i would urge you to re-evaluate. The normal experience is just "cargo run" which rarely does a full build.
    - berkut 4y ago
      I've got a ~9k loc Bevy "game" I'm writing that doesn't use dylib and takes > 20 secs every update, purely because of the linking (I'm not using mold yet, but am using lld)... i.e., I type 'cargo build', it compiles the single .rs I changed almost instantly, but then I'm staring at: Building [=======================> ] 314/315: landscape(bin) for the next 20 seconds.
diimdeep 4y ago
Kudos. Are there languages on the horizon that could also be included as possible serious choices? Nim, zig, jai, or we all will be eternally waiting for future version of C++ ?
- vegerot 4y ago
  The author said he liked Rust and will switch to Rust if/when it builds faster than C++. So for the author at least, his answer is Rust!
- strager 4y ago
  I'd love to try Zig and Jai for this purpose, but they are not stable yet. If either becomes stable, I'll happily re-run this experiment. Nim doesn't look like the language for me.
  - henry_viii 4y ago
    Would love to hear your opinion about V too especially since one of its main selling points is fast build times: https://vlang.io/#:~:text=Small%20and%20easy%20to%20build%20compiler https://vlang.io/#:~:text=Small%20and%20easy%20to%20build%20...
scns 4y ago
An OT hint to the author: I noticed that your RAM runs at 3800, that leads to a mismatch with the Infinity Fabrics' clock of 1800, i.e. increased latency. There are two solutions to this: Raise the FCLK (Fabric Clock) to 1900. This should be well tolerated by a Ryzen from the 5000 line. Lower the RAM clock to 3600 and set sharp sub timings. This will take a little more time but this tool [0] will help. Runs on Windows though, i have an old partition around for stuff like this, dunno if a VM will do. RAM OC became a time consuming hobby for some people, the rabbit hole... [0] https://www.techpowerup.com/download/ryzen-dram-calculator/ https://www.techpowerup.com/download/ryzen-dram-calculator/ (edit) add sentence bout windows
- strager 4y ago
  > Raise the FCLK (Fabric Clock) to 1900. Hmm, I think I did this, or something like this. I'llcheck tomorrow. (If I did, then my comment about not overclocking the CPU is misleading!)
- Arch-TK 4y ago
  >[0] https://www.techpowerup.com/download/ryzen-dram-calculator/ https://www.techpowerup.com/download/ryzen-dram-calculator/ Why on earth is this a binary program? Is there a document online which explains the maths or just implements it in javascript? I don't feel like spinning up a windows VM just to do this kind of calculation.
  - scns 4y ago
    The author links a google sheet somewhere where he draws the values from. It is several miles long ;)
- strager 4y ago
  Current settings according to BIOS: Target CPU Speed: 3400MHz Target DRAM Frequency: 800MHz Target FCLK Frequency: 1800MHz DRAM timings: 19/20-19-19-19-39 (The DRAM CAS# Latency setting is set to 19, but "CHA" and "CHB" show 20. I don't know what this means exactly.) So I think you're right, scns. I was not getting the best performance. --- I changed FCLK to 1900MHz and re-ran the Linux C++-vs-Rust benchmarks. Some results: build+test w/o deps: 1847ms Rust, 1874ms C++ --> 1801ms Rust, 1837ms C++ incremental test-utf-8: 288ms Rust, 358ms C++ --> 280ms Rust, 350ms C++ So I did get a performance win for both C++ and Rust builds by fixing my FCLK. Thanks!
minigamedev 4y ago
A lot of times I felt like deleting the cargo.lock file should have been done inbetween optimising the rust build. There were changes to the used crates, and sometimes those changes don't propogate before a `cargo clean`, `cargo update` or something like that.
- strager 4y ago
  I'm not sure what you mean. Are you talking about the benchmarks? I run `cargo clean` before each benchmark. That should clean up changes to third-party crates, right?
Jyaif 4y ago
Very impressive investigation. Unfortunately matches my experience. There's a chance that Rust is the programming language with the longest build times ever.
- Narishma 4y ago
  Scala has it easily beaten in that regard.
kibwen 4y ago
I'm curious if the results at the very end for the copy-pasted code benchmarks are a result of linking. They seem to have chosen to use mold for C++ and not for Rust after seeing that it gave little benefit for small projects, but I would expect that to change as the project scales. In addition, I'd be interested in seeing how `cargo check` fares. In practice I do `cargo build` quite rarely, in favor of `cargo check` and `cargo test`.
- chrismorgan 4y ago
  I think https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac5d0e676ead553a3610284d47a3e/tools/bench-build-charts.py#L873 https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac... indicates Rust is using Mold too. And I get the impression that the thing unit being benchmarked is a single crate—that is, one crate 8×, 16×, 24× the size (which puts the burden on the compiler, in areas that don’t parallelise well), rather than 8×, 16×, 24× as many crates (which puts the weight on the linker and on well-parallelised parts of the compiler). On the hardware in question, which has loads of cores, I’m very confident that Rust would fare considerably better with 24 17.1k-line crates (410k lines, larger due to duplicating the entire thing rather than just the lexer) than with the one 104.4k-line crate apparently tested.
  - strager 4y ago
    > I’m very confident that Rust would fare considerably better with 24 17.1k-line crates (410k lines, larger due to duplicating the entire thing rather than just the lexer) than with the one 104.4k-line crate apparently tested. I didn't consider this in my scaling benchmark. You make a good point.
- eximius 4y ago
  Yea, comparing C++ build times and Rust build times seems to ignore the practical workflow differences between writing the two. Also, I think it's a little silly to say that Rust build times scale poorly (even if it isn't quite as good as C++). 100k LOC in, what, 6s at the end there is not so shabby if you're typically never building all of it.
- 4y ago
deleted 4y ago
[deleted]
sylware 4y ago
I would be more interesting to know how easier it is to write a naive rust compiler than a c++ naive compiler, aka actual syntax complexity comparison. To put that in perspective, how easier it is to write a naive rust compiler than a naive C compiler.
epage 4y ago
A surprising source of slow compile times can be declarative macros in Rust [0]. I believe the core of the problem is that it has to reparse the code to pattern match for the macro. One egregious patter is tt-munchers [1] where your macro is implemented recursively, requiring it to reparse the source on each call [2]. In one of my projects, someone decided to wrap a lot of core functions in simple macros (ie nt tt-munchers) to simplify the signatures. Unlike most macros which are used occasionally and have small inputs, this was a lot of input. When I refactored the code, I suspect dropping the macros is the reason CI times were cut in half and a clean `cargo check` went from 3s to 0.5s. [0]: https://nnethercote.github.io/2022/04/12/how-to-speed-up-the-rust-compiler-in-april-2022.html https://nnethercote.github.io/2022/04/12/how-to-speed-up-the... [1]: https://veykril.github.io/tlborm/decl-macros/patterns/tt-muncher.html https://veykril.github.io/tlborm/decl-macros/patterns/tt-mun... [2]: https://github.com/dtolnay/quote/blob/31c3be473d0457e29c4f47ab9cff73498ac804a7/src/lib.rs#L664-L746 https://github.com/dtolnay/quote/blob/31c3be473d0457e29c4f47...
- deleted 4y ago
  [deleted]
- vegerot 4y ago
  cool! Does quick-lint use this pattern? https://github.com/quick-lint/cpp-vs-rust https://github.com/quick-lint/cpp-vs-rust
bjt2n3904 4y ago
This author has a very meticulous experimental approach. I'm not a rust developer, but I enjoyed the article none the less.
tulio_ribeiro 4y ago
You should definitely keep an eye on Val and Carbon. They're being designed to be interoperable with C++. Both are designed to match C++'s performance while still being able to work with your existing C++ code. Both of these languages offer a more modern developer experience and are built with software and language evolution in mind. They have practical safety and testing mechanisms, etc. Definitely check them out if you're looking for a C++ replacement! https://github.com/val-lang https://github.com/val-lang https://github.com/carbon-language https://github.com/carbon-language
- vegerot 4y ago
  Do they compile fast? If not, why not just use Rust? The author said he’ll switch to Rust when the compilation speed improves
- strager 4y ago
  > Definitely check [Val and Carbon] out if you're looking for a C++ replacement! I don't want to base my project on an experimental language like Carbon or Val. I want a C++ replacement which compiles quickly. Do Carbon or Val compile quickly? If not, I don't know why you mention these languages.
throw10920 4y ago
> In theory, if you split your code into multiple crates, Cargo can parallelize rustc invocations. Because I have a 32-thread CPU on my Linux machine, and a 10-thread CPU on my macOS machine, I expect unlocking parallelization to reduce build times. Why can't Cargo parallelize rustc invocations per-file, like C++ can?
- lights0123 4y ago
  rustc needs to look at the entire crate because there's no header files that provide information about other items like C++. rustc does parallelize internally though: https://doc.rust-lang.org/rustc/codegen-options/index.html#codegen-units https://doc.rust-lang.org/rustc/codegen-options/index.html#c...
  - throw10920 4y ago
    > Increasing parallelism may speed up compile times, but may also produce slower code Why do I have to make the choice between faster compiles and faster binaries?
    - lights0123 4y ago
      The exact same thing happens in C++—when you're not using LTO, you can pass multiple source files to the compiler at a time to allow these files to be optimized together, but keeping them separated at a file-level prevents optimizations such as inlining functions between them. Compiling with and without optimizations is another example of that choice.
- steveklabnik 4y ago
  Because the unit of compilation is a crate, not a file. You don't pass each file to rustc, you pass the crate root only.
dcow 4y ago
This is a nice article. But, I don't really understand the conclusion that Rust compile times are a problem (and that C++ times are too). Rust is not a tool I pick up because 3s compile times are a nonstarter and I need sub 1s to feed my developer ADAH. I spend orders of magnitude more time writing Rust than I do compiling it so a 2s difference is utterly marginal. I don't know.. this obsession with compile times in the last 5 years seems rather irrelevant to me, not that I don't appreciate lower ones. But "problematic" is not an adjective I'd use.
- epolanski 4y ago
  I think the article points a good reason why: even with an incredibly powerful machine it takes many, many minutes to compile projects like chromium. On a "normal" machine, it takes several hours. This is non-ideal and chromium is probably and already heavily optimized and well written project with lots of documentation.
  - dcow 4y ago
    In what language or toolset can you compile a project like Chromium faster? Big projects take a while to compile. Nobody has broken this rule yet so why are we fixated on it being problematic? Obviously, we could not compile the project and eschew all the optimizations we get, but then the end user would be unhappy. I don't know, I've always found the tradeoff to be quite understandable and inoffensive: if I want faster development (among other things), I use an interpreted scripting language. If I want a fast end product (among other things) I compile and optimize to machine code beforehand (obviously it's not that simple, but you get the point).
    - vegerot 4y ago
      Something to stress is that the article is not just about compile times. It’s about compile+test times. If Chrome were written in Python it’d be hella slower iteration times
    - netr0ute 4y ago
      > In what language or toolset can you compile a project like Chromium faster? Go is super duper fast, so there's that.
ho_schi 4y ago
For full builds, C++ will take longer to compile than Rust (i.e. Rust wins). This is because of C++'s #include feature and C++ templates, which need to be compiled once per .cpp file. This compilation is done in parallel, but parallelism is imperfect. The author gives later on within the article a hint upon "and externing template instantiations". This allows to compile a template just once with a given set of template arguments. Shall reduce reduce compile-time. I assume it also reduces size of created object-files? Explicit instantiation of templates is supported since C++11. Therefore templates with one set of template arguments can be are declared in multiple files and defined just once - in one file. The keyword for this is extern upon each declaration. Please note, that all member of a template will be instantiated whether used or not. The user "strager" mentioned that briefly. see: https://en.cppreference.com/w/cpp/language/class_template#Class_template_instantiation https://en.cppreference.com/w/cpp/language/class_template#Cl... https://en.cppreference.com/w/cpp/language/function_template#Function_template_instantiation https://en.cppreference.com/w/cpp/language/function_template...
- strager 4y ago
  > I assume it also reduces size of created object-files? Normal template instantiations are deduplicated by the linker. Total object file size is smaller with explicit template instantiations, but the size of the final executable is the same with both styles.
  - ho_schi 4y ago
    Wow. Thanks :)
Night_Thastus 4y ago
C++ build systems and compilation time in general is basically my most significant complaint as a C++ developer. There are so many disparate build systems, none of them are intuitive, none of them do much in the way of error checking or give good user feedback, and the resulting build is always slow. And as for the code itself, there's no type of linter or checker that will tell you if there's a way to save on compile time. "Hey, this could be a forward decl instead of a include!", or "Hey, this include is completely unused!" or some such. (There's include-what-you-use but I could not for the life of me figure out how to make it work) I know the problems are hard, I just wish more effort was expended to make the whole process less painful and slow.
- strager 4y ago
  I have found Clang's -ftime-trace flag helpful in finding bloated #include-s and templates. Also, I have analyzed the .ninja_log file (for CMake+Ninja) to find slow-to-compile .cpp files.
hacksoi 4y ago
Rust's only selling point is the way it tries to solve memory errors, but it's based on a style of memory management that is inferior. There are other styles that prevent such errors from occurring in the first place.
- roeles 4y ago
  Please elaborate. I have zero knowledge about rust, so I am wondering what you're referring to.
teknico 4y ago
> (Unless I become enchanted by Zig first.) You probably will. :-)

314 points by cod1r 4y ago

deleted 4y ago

[deleted]

vegerot 4y ago

Thank you for finally talking about this! I appreciate it, thanks

taspeotis 4y ago

http://webcache.googleusercontent.com/search?q=cache:qTP19KUQXF0J:https://quick-lint-js.com/blog/cpp-vs-rust-build-times/&hl=en&gl=au&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:qTP19KU...

neonate 4y ago

https://archive.ph/f5aiI https://archive.ph/f5aiI

azakai 4y ago

Very well written, and I like that it ends on this note: > Looking at my hypotheses, I was wrong on all counts The author expected some things, and had good reasons to, then did a lot of serious work, and ended up finding out somewhat different things. Impressive and honest exploration of a complex topic that is quite hard to measure.

vasvir 4y ago
Totally agree. I would like to stress the "serious work" point. He tried a lot of configurations and kept track of the results. This is not easy as it sounds since most people lack the discipline (me included). I wish more papers were written like this with clear separation of hypothesis, experimental results and conclusions. Also the clarity of presentation was superb again not an easy task. Extra bonus points for the intellectual honesty.
- brabel 4y ago
  Writing a blog post like this and running benchmarks on several combinations of OS, language and compiler options and different tools takes so much effort and time. We should be very thankful to the people who do this. I've done it a couple of times myself... but it was so much more effort than I thought, literally weeks non-stop, that I don't think I will be doing it again any time soon.
  - strager 4y ago
    Thank you for your kind words! I appreciate your empathy. This project took about 7 weeks (including trimming, porting, benchmarking, optimizing, and finally writing the article).
deleted 4y ago
[deleted]

oslac 4y ago

Does this article account for the fact that a rustc compilation does more things than simple C++ default compilation, where you need to run address sanitizers etc. to do the same as rustc?

strager 4y ago
No. If I was going to run sanitizers with the C++ code, for a fair comparison, I would also run the Rust code with Miri. (C++ sanitizers check unsafe code, but without Miri, rustc mostly checks only safe code.)
- vegerot 4y ago
  What about the other way around? Trying to cripple rustc until its features are closer to clang's? For example, removing some of the sanitizers or borrow checker in rustc?
  - strager 4y ago
    That's a great idea! Unfortunately, I think there's no way to disable rustc's borrow checking or other safety features.
- insanitybit 4y ago
  Just fwiw you can run rust code with sanitizers, no need for miri, which works differently.
  - strager 4y ago
    Oh, I did not know this.
jupp0r 4y ago
Compilation doesn't necessarily need strict checks, it's really just translating source code into machine code. It happens to be the case that we mostly do both using the same tools because compilers need to have a good understanding of type systems to do their job well, but it's not a strict requirement. A great counter example is esbuild, which will compile TypeScript but not perform actual type checks for performance reasons. This is possible because TypeScript types have no representation at runtime. and are really only a development tool.

pjmlp 4y ago

Thing is, not every C++ project is full of template metaprogramming, and at least on the corporate world it is quite common to use binary libraries, which NuGET, conan, vcpkg, and OS package managers support. Also at least in VC++, modules are already a thing, no need to keep parsing template code all the time. Doing a C++23 import std (which imports the whole standard library) takes a fraction of the time of #include <iostream> in VC++. So a clean build out from repo means compiling only our own code, not the whole world as in Rust. Still looking forward to binary libraries support in cargo (well one can dream).

forrestthewoods 4y ago
> Doing a C++23 import std It’ll be nice when this can be reliably used. According to cppreference MSVC has partial support and other compilers have zero support. I’m not sure modules will be in widespread use before 2029. https://en.cppreference.com/w/cpp/compiler_support https://en.cppreference.com/w/cpp/compiler_support
- pjmlp 4y ago
  VC++ support is good enough that all my C++ hobby coding is done with modules, not everything we code must be cross-platform from the get go. GCC is getting there. clang, well they seem stuck with their own initial modules implementation (module maps based), and now with Apple and Google out of the picture, the various compiler vendors that benefit from MIT license don't seem that eager to contribute to upstream other than LLVM. Unfortunely everyone else seem to still be catching up with C++14 and C++17, depending on the ecosystem.
  - forrestthewoods 4y ago
    > not everything we code must be cross-platform from the get go. No, but the code I write for my job does have to be cross-platform from day 0. And that's the code base that is large enough where module improvements to compile time would be most useful! Hobby projects are small enough it's not super important. So C++20 modules don't help me and probably won't until 2029! To my great dismay.

habibur 4y ago

The site's going up and down. Here's Conclusion : //// Are compilation times a problem with Rust? Yes. Are build times as bad with Rust as with C++? Yes. Looking at my hypotheses, I was wrong on all counts: The Rust port had more lines than the C++ version, not fewer. For full builds, compared to Rust, C++ builds took about the same amount of time (17k SLOC) or took less time (100k+ SLOC), not longer. For incremental builds, compared to C++, Rust builds were sometimes shorter and sometimes longer (17k SLOC) or much longer (100k+ SLOC), not always longer.

deleted 4y ago
[deleted]
vegerot 4y ago
or just https://smmry.com/https://quick-lint-js.com/blog/cpp-vs-rust-build-times/#&SM_LENGTH=14 https://smmry.com/https://quick-lint-js.com/blog/cpp-vs-rust... like I did

berkut 4y ago

In my experience of building large C++ projects on machines with >= 64 threads (especially if using ccache or something), linking is often the bottleneck to incremental compilation as opposed to cpu-bound compilation. Some of our dev builds actually are split up into separate .so files which is a bit slower at execution time, but means linking is a lot faster, especially when you want full debug info...

jupp0r 4y ago
Chrome is doing the exact same thing for the same reason. Dev builds are dynamically linked while release builds are statically linked. Works quite well, it's a great technique.
- pjmlp 4y ago
  It is quite common in Windows, either produce lib or dlls for each set of main modules. In the old days it used to be quite common on UNIX world as well, no idea why "modern" Linux seems to have forgotten about this approach and always builds everything as a single project unit.
  - bombolo 4y ago
    Because google decided we all want one big binary that contains the go runtime and weights at least 100MB
    - mwcampbell 4y ago
      I have several Go-based binaries in my $HOME/bin, and not a single one is 100 MB.
      - vegerot 4y ago
        And all of them combined compile faster than either of these implementations!
      - 4y ago

29athrowaway 4y ago

C++20 modules should help with the build speed... But 3 years later, many systems don't support them. (This is mentioned in the article)

fooker 4y ago
Modules help with build speed of small projects where compiling standard library headers are a significant part of the compilation time. For well organized large projects, modules would be marginally better at best.
- college_physics 4y ago
  This could still be a dramatic improvement for numerical work that requires a lot of exploratory compilations and iterative style (i.e. steal back some market share from python)
saurik 4y ago
Is there any new reason to believe that? What I had seen--and which I would have thought would be obvious--is that the C++ module design only helps at lower levels of parallelism and actually slows down the build dramatically at the higher levels of parallelism used by people who actually care about build performance; this is apparently because the inherent design of modules causes the same kind of pervasive coupling seen with Rust, destroying the inherent scalability of the classic C++ separate compilation model. https://vector-of-bool.github.io/2019/01/27/modules-doa.html https://vector-of-bool.github.io/2019/01/27/modules-doa.html https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1441r1.pdf https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p14... > Expectedly, as hardware parallelism increases, headers’ lead over modules becomes more and more pronounced. There is also a relationship between the DAG-depth (i.e. The length of the chain of modules that import each other). As this depth increases, modules grow slower and slower, while headers remain fairly constant for even “extreme” depths approaching 300).
- pjmlp 4y ago
  Old outdated news. In C++23 it takes less time to import the whole standard library than a single major include like iostreams. Currently only available in VC++ preview, though.

choeger 4y ago

I have a hunch that rusts global compilation model (as opposed to C++ separate compilation, as crippled as it is in practice due to header-only libs and link times) should always lead to C++ scaling better with the size of the project. Simply put: Rust doesn't offer an abstraction past type checking. Rust could potentially introduce its equivalent of PCH, or maybe it already does so in some internal cache, but as long as it uses monomorphization and no PhD student comes up with a clever trick to combine that with separate compilation it will scale worse. That being said, maybe a more fair comparison wouldn't involve polymorphism in the rust code? C++ doesn't really have polymorphism, it uses templates for a similar feature. So what happens if someone replaces rusts polymorphic functions with macros?

jimbob45 4y ago
What do you mean by abstraction beyond type checking? I’m asking to educate myself.
- choeger 4y ago
  "Beyond" in the sense of "after". Languages that support separate compilation usually have some sort of artifacts that do not need to be type-checked anymore. Say, you compile an Ocaml module. The result is a compiled (bytecode or native code) file that represents the implementation of the module and doesn't need to be generated ever again. Essentially, the compiler abstracts over a module after it dealt with it. You could the If you do the same in rust, you need access to the module's implementation for monomorphization. You might skip the type checking of a function of the implementation or even cache the generated code for a monomorphization as optimizations, but generally you have to treat every function as often as it is used. So the best rust could do is akin to C++ precompiled headers whereas a language like OCaml can simply compile every module exactly once. Note: OCaml pays for this elegance. Its uniform object representation is less efficient than what rust does. It's a trade-off that the rust designers consciously made, I think.
  - scaredginger 4y ago
    What you've said here is mostly correct. The key part you've omitted is that monomorphization is opt-in. One could write Rust like C if they wanted to; they would have to be insane, but they could

fooker 4y ago

Interesting results, I was expecting incremental Rust to be significantly faster than incremental C++ builds. Apparently not. This is something Rust folks should strive to solve as a differentiator, because there is no realistic way of making incremental C++ builds faster.

pjmlp 4y ago
The incremental compiler and linker in VC++ alongside modules seems realistic enough to me.
strager 4y ago
> This is something Rust folks should strive to solve as a differentiator, because there is no realistic way of making incremental C++ builds faster. I think C++20 modules are the promised golden ticket. It's 2023 and I still can't use them on Linux, though.

fouronnes3 4y ago

An interesting thought is that we really ought to not compare rust with just GCC, but GCC + clang-tidy, since you get so much more static analysis by default in rust compilation than C++.

strager 4y ago
> An interesting thought is that we really ought to not compare rust with just GCC, but GCC + clang-tidy Nitpick: The article compares Clang, not GCC. (GCC is mentioned only to show how much faster Clang is.) > you get so much more static analysis by default in rust compilation than C++. I care about the build-test cycle, so adding static analysis is against my goals. I would minimize the analyses done by rustc if I could.

misja111 4y ago

As a Scala developer, I can only dream about these build times. 1.8s to build a 17K LOC project .. This would take minutes for my similarly sized Scala project.

strager 4y ago
I'm sorry to hear that! Did you try upgrading your hardware? It really helps.
odersky 4y ago
This looks completely out of the ordinary. With a warm compiler, on a 4 year old Macbook Pro, I get 2000-4000 lines/sec. I.e. 4-8 seconds for your project. Unless you do some very involved typelevel or meta-programming stuff that's what you should expect to see.
- misja111 4y ago
  Yes my project does contain a bit of typelevel stuff (Cats Effect). And it's Scala 2.13, so not the latest 3.x which might compile a bit faster. Also to be fair, I was referring to the time taken by sbt compile, which I suppose does more than just invoking scalac. But still the time reported by sbt's Compile/ Compile incremental is over 200s.
  - gpderetta 4y ago
    to be fair, C+ header only libraries full of advanced template metaprogramming will also slow down compilation by a few order of magnitude.
    - bluGill 4y ago
      I'm hopeful modules will come fast, they seem to fix that issue. Modules are already in C++20, but most compilers don't support them well enough yet.
      - 4y ago

insanitybit 4y ago

I guess my question is - what is to be done? Even C++ is quite slow to compile. If all rustc development stopped and there was a years-long effort to write it "fast", what sort of improvements could we see?

vegerot 4y ago
They should rewrite it in Rust
- xnickb 4y ago
  It is already in rust. Which is a major problem for anyone trying to bootstrap rust for packages.
strager 4y ago
Good question! Based on my profiling, it seems like the code gen backend (LLVM) is a big offender. A code generator optimized for build times would be nice. (Cranelift seems to be a failure in that regard.) C++ has an explicit template instantiation feature which helps reduce backend time. I wonder if this can be done in Rust too. Maybe this is what -Zshare-generics=y is for?
- Bedon292 4y ago
  I don't know enough C++ or Rust to judge but is it possible that they are optimized around Rust that isn't C++ style? I know I have seen articles about other languages where some very minor changes make a huge difference because the optimizations in place didn't understand what was happening and had to work harder. And did you try any of the optimization strategies again at 24x? I would be curious if there are differences there.
  - strager 4y ago
    > I don't know enough C++ or Rust to judge but is it possible that they are optimized around Rust that isn't C++ style? I know I have seen articles about other languages where some very minor changes make a huge difference because the optimizations in place didn't understand what was happening and had to work harder. Are you talking about run-time optimizations? My article is focused on build times. I don't think what you mentioned applies to build times. I could be wrong, though. (There are certainly compile time traps you can fall into, but I wouldn't know what those are in Rust.) > And did you try any of the optimization strategies again at 24x? I would be curious if there are differences there. No, I did not. That's a good suggestion. But I was pretty tired of this project by the time I made the scaling benchmarks. xD

ycui1986 4y ago

the author should talk to an FPGA engineer about build/compile time. That will make any computer programming language look like lightning speed.

Gigachad 4y ago
Or talk to a web developer.
strager 4y ago
No, I'll stay far away from them, thanks. ;P

charcircuit 4y ago

This isn't fair. The Rust version was broken up into multiple crates, but the C++ version was not broken up into multiple libraries.

strager 4y ago
I also benchmarked the Rust code with a monolithic crate. It built slower. https://quick-lint-js.com/blog/cpp-vs-rust-build-times/#optimizing-rust-with-different-layouts https://quick-lint-js.com/blog/cpp-vs-rust-build-times/#opti...
- charcircuit 4y ago
  I was more curious about if there was a speed up on the C++ side. Especially if you use Bazel it's common in C++ to compile a lot of small libraries.
  - strager 4y ago
    Ah, good point. I hadn't considered this.

eska 4y ago

I don’t think the Rust code could be shorter than the C++ code when converting pretty much line by line, since that leads to C++-style Rust code. For example idiomatic Rust code can do with less defensive programming due to language features.

strager 4y ago
> I don’t think the Rust code could be shorter than the C++ code when converting pretty much line by line, since that leads to C++-style Rust code. I thought the opposite: header files (and CMakeLists.txt, which was included in the subtotal SLOC) artificially increase C++ SLOC. > For example idiomatic Rust code can do with less defensive programming due to language features. Can you show an example in the Rust code where this might be possible? https://github.com/quick-lint/cpp-vs-rust/tree/master/rust https://github.com/quick-lint/cpp-vs-rust/tree/master/rust
- batmansmk 4y ago
  Just wanted to say thank you for sharing your code! Im not familiar with C++, so Im discovering the union, UnsafeCell, ManuallyDrop and all those concepts that are rare in the Rust projects I see everyday. Im learning a lot.
  - strager 4y ago
    You're welcome! But maybe my code (written by a Rust newbie) isn't the best to learn from... ;P
- xiphias2 4y ago
  I just randomly looked at one part: linked vector. I know that the comparision here is a line by line comparison, but Rust gives a more stable interface for reusing libraries than C++. It would be a great experiment to use lots of external libraries. You might find that the code doesn't get slower (build times would probably go up though )
  - 4y ago

junon 4y ago

I don't really like these comparisons looking at these languages through the lens of commonality. The only thing the two have in common is that they're systems languages. C++ is a bloated mess that does exactly zero checking beyond type checking. Rust is a language that favors safety over build times and its serious users understand that. Rust is also developed completely in the open. C++'s Standards committee is rife with resignation letters, accusations of misconduct, and walled gardens - not to mention years between standards updates, each costing a ton of money to even look at. Yes, I understand. Rust has no standard. It has no stable ABI yet. Depending on who you are or what you're doing these may or may not be issues for you. Alas, comparing these two as though one should outperform the other is nonsense to me. Of course Rust is slower than C++. C++ has had decades of maturity and has enjoyed many contributors to its tooling. Rust has not, and its feature set (as in, the things the compiler needs to do) is much broader than any C++ compiler is specified to do. The end result here is this will only work to fuel more language wars and be used as a cheap weapon in further language debates. In my opinion, anyone who argued about which language is better than which other language has completely missed the point of computer science and programming.

charcircuit 4y ago
>only work to fuel more language wars Iteration time is an important part of the developer experience. The trade off of slower iteration time for improved safety is a real point that needs to be considered when choosing between C++ or Rust. This isn't something silly like curly braces vs whitespace.
- vegerot 4y ago
  Agreed! …but while we’re at it curly gang all day
strager 4y ago
> I don't really like these comparisons looking at these languages through the lens of commonality. The only thing the two have in common is that they're systems languages. > > [...] In my opinion, anyone who argued about which language is better than which other language has completely missed the point of computer science and programming. When I start a new programming project, I need to choose a language. For this project, three years ago, I had three serious choices: * C * C++ * Rust I rejected C because I wanted the productivity boost from C++ or Rust. I rejected Rust because Rust was new to me (thus risky) but I was very familiar with C++. I don't see why it's a bad idea to compare languages—especially along specific dimensions, like run-time perf or build time or safety—when that's what us software engineers do whenever we start a new project.

t43562 4y ago

I think it's hard to draw absolute conclusions because small things can make a huge difference to build times. I worked on optimising lots of builds from Symbian to Android and other proprietary ones. C++ compiler choice is very important - gcc might seem slow but it is a speed demon compared to the old ARM compilers. There are a lot of tricks based on trying not to do the same work twice like pre-compiled headers which might help massively (since headers can be much bigger than your code) but quite often these are also very fragile because changing one #define in one central header file might invalidate everything. It's not that easy to be sure that you are truly handling all dependencies. Hence one tends to have to regularly build from scratch to be certain that everything is done properly. If we really could handle it all perfectly then nobody would ever bother to build from clean. It's also very important to be able to see how your build tool is scheduling tasks and making use of your CPUs - often some odd dependencies force most of your cores to sit idle while something gets done. There aren't good visualisation tools for this that aren't proprietary imo. I have cobbled together something for gmake which can output something you can visualise in chrome's profiler but it's crap compared to the best commercial tool I've used. You can remember the previous build times of various objects and use them to try to schedule large tasks as early as possible so they don't extend the build time by randomly getting started towards the end and then leaving the build going on and on just to get them finished. The structure of your code matters too - a simple optimisation for C/C++ is just to have much larger source files rather than splitting everything up into many small ones. This acts to reduce the number of repeated includes. If I was thinking about how to get fast builds it would force the language to change - header files would be banned and that means macros and the whole idea of preprocessing. That means a different approach to multi-platform support. Android benefits from having a java layer where you can pretty much build once and run anywhere and that is very good. Eventually, on one big project, we got to the point where it was the packaging that was the slowest component and that was being done by the CI system .... for no good reason other than organisational inertia. Another team was responsible and didn't want to take advice. Since we couldn't bring it into the build and use all our tricks to optimise and redesign it we hit a wall where all our efforts to improve build performance had little impact. The best tricks for improvement, of course, are the ones which always work and don't need a lot of maintenance to keep them functioning properly. I think we need builds to be integrated with source control. Getting new versions of source and new binaries is all far too "separate" a process. Checking in 5 files should result in a very accurate rebuild of only the dependencies and a subsequent checkout should be able to get me those binaries. Developers should never really have to build all of Android just to work on some small part and they shouldn't really have to do more than one checkout to get everything they need including binaries.

varajelle 4y ago

Regarding the sloc count, the default automated Rust formating tool is very eager to adds lot of lines by basically keeping only one word per line. Something I'm not a fan of, I must say.

sph 4y ago
It usually does that on iterator chains, which AFAIK do not exist as such in C++, so multiple operations would be expressed as multiple imperative statements. My C++ is rusty (no pun intended) but I struggle to imagine their variant of `vector.iter().map().collect()` to be as concise and fit in fewer than 4 lines. I wonder if OP's C++ port doesn't use iterators that much, and how idiomatic it is. EDIT: the code is not idiomatic at all.
- strager 4y ago
  > I wonder if OP's C++ port doesn't use iterators that much, and how idiomatic it is. I think I only used iterators in places where there's no built-in function on slices like C++'s strchr and strspn. (I think Rust's str has these, but not [u8].) For example: C++: https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac5d0e676ead553a3610284d47a3e/cpp/src/quick-lint-js/i18n/locale.cpp#L38-L43 https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac... std::size_t length = std::strcspn(c, separators); if (c[length] == '\0') { return found_separator{.length = length, .which_separator = static_cast<std::size_t>(-1)}; } const char* separator = std::strchr(separators, c[length]); Rust: https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac5d0e676ead553a3610284d47a3e/rust/libs/i18n/src/locale.rs#L53-L64 https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac... match s .as_bytes() .iter() .position(|c: &u8| separators.contains(c)) { None => FoundSeparator { length: s.len(), which_separator: INVALID_WHICH_SEPARATOR, }, Some(length) => { let found_separator: u8 = unsafe { *s.as_bytes().get_unchecked(length) }; match separators.iter().position(|c: &u8| *c == found_separator) {
- mgaunard 4y ago
  Of course it exists in C++, and has done since before Rust even existed. Syntax is usually `vector | map | collect`.

Dowwie 4y ago

Note how the OP uses an M1 laptop w/64MB of ram. If you want to compile a monolithic web application written in Rust within a reasonable amount of time, you need that much top-of-the-line hardware for your work. You're still waiting several minutes to compile, but compile times get even worse with older hardware running debian and mold/lld. CI test/build runs also take a long time to complete. Edits: 64GB, not 64MB. 32GB is really the minimum recommended

zozbot234 4y ago
> w/64MB of RAM. So, an Apple Newton in a laptop form factor?
- Dowwie 4y ago
  Oh, I did just say MB. It's early here.
- brabel 4y ago
  No one has corrected this yet, so I'll do it: it's 64GB of RAM! The Mac M1 Max the author used is an absolute beast of a machine. I have a M1 Pro (which is not as fast) and it's ridiculously faster (and quieter - it barely needs a fan) than my Linux Dell XPS13 and older Macbook Pro.
  - strager 4y ago
    It is a beast. I love my MacBook Pro!
josephg 4y ago
I don’t think 64mb of ram is necessary at all. And minutes? What are you compiling? How big is your codebase? Are you doing everything in docker or something? Rustc is slow but I’ve never seen it be that bad. I’ve been working on a rust project for a couple of years, currently sitting at about 40k loc. Incremental debug builds still only take a second or two. Full compiles take about 10 seconds.

sph 4y ago

Re: the SLOC count: the code is here https://github.com/quick-lint/cpp-vs-rust/tree/master/rust https://github.com/quick-lint/cpp-vs-rust/tree/master/rust That... doesn't seem to be idiomatic Rust at all. unsafe and passing pointers everywhere? Reimplementing vectors and linked lists. Also, makes use of a ton of macros. That's basically writing C++ in Rust.

throw827474737 4y ago
Read also the article about what he did? Could have said that in advance without looking at any code ;) (And further hint, that shouldn't be the significant hook here..)
strager 4y ago
> unsafe and passing pointers everywhere? Do you think this would impact Rust build times? > Reimplementing vectors and linked lists. Someone has to implement them. Note that they're also implemented in the C++ code, so the comparison is fair. > Also, makes use of a ton of macros. Can you show an example where the Rust code uses "a ton of macros"? The only place I can think of is the tests, but I thought macros were what you're supposed to use in Rust for assertions. If you're referring to proc macros, I thought those were very popular in Rust.
- scaredginger 4y ago
  I gave up only a little beyond where they revealed that they ported line-by-line. I think this is an interesting experiment for those considering a Rust rewrite, but has no utility for a greenfield project. The scales are stacked strongly in favour of C++ here, without a doubt. Software designed for C++ has essentially zero chance of becoming idiomatic Rust without a huge amount of work, and tooling tends to be designed for idiomatic use
  - sebastianz 4y ago
    > I gave up only a little beyond where they revealed that they ported line-by-line. The commenter you are so dismissively replying to is the author. He did not "give up only a little beyond something" like you did, he made a serious effort that probably took hundreds of hours, and documented it. I am not sure why you are posting such a comment full of disrespect for his work, after reading a couple of paragraphs of his article.

pharmakom 4y ago

I think it’s acceptable for Rust compilation to end up slower than C++. After all, the compiler does a lot more work for you with the borrow checker and all. That doesn’t mean we shouldn’t strive to make it better though.

strager 4y ago
I'd wager that the C++ compiler does more work churning through #include-s than the Rust compiler does tracking borrows.
- bluGill 4y ago
  C++20 modules is supposed to fix that. I wish I could get compilers that implemented them.
  - pjmlp 4y ago
    There you go, https://visualstudio.microsoft.com/vs/community/ https://visualstudio.microsoft.com/vs/community/ One example to get you started, https://github.com/pjmlp/RaytracingWeekend-CPP https://github.com/pjmlp/RaytracingWeekend-CPP
    - bluGill 4y ago
      I target linux with my C++.
      - pjmlp 4y ago
        That wasn't part of the comment.
Narishma 4y ago
What makes you think it is slower because of the borrow checker?
- 4y ago

_448 4y ago

Very thorough write-up, and an objective comparison. I would like to mention my subjective concerns for Rust. I hope the Rust community can think me as a canary who is from the C++ world. I am desperately looking for an alternative to C++, and I am a mid-weight C++ programmer. And here is what Rust could face down the line (remember, in early days C++ too was a grockable language and hence its popularity): 0. Rust's C++ problem: Learning curve. From the get go if there is murmur that Rust has a learning curve then think what will happen when Rust reaches the maturity of C++. 1. Rust's NPM problem: supply-chain. If I decide to use npm, and use the npm toolchain to download a project then to my horror it downloads god knows what. Lots of dependency hierarchies, some of them have not even reached 1.0. This raises the concern for supply-chain attacks. Rust with the aim of "boosting developer productivity" just decided to follow the path of all those who are susceptible to supply-chain attacks. 2. Rust's Haskell problem. Haskell is a very beautiful language. Even then it has not reached the level of adoption of other popular languages. It has nothing to do with the language, but some in the community who have a tendency to project a coolness level by saying something like "A monad is just a monoid in the category of endofunctors". What that ultimately does is alienate lot of software developers. Early Rust evangelist tried very hard to project Rust as having a very welcoming community, but if acronyms and terms from category theory and lambda calculus, that are alien to most developers, are thrown around without taking into consideration the larger development community then I fear that Rust will go the Haskell or OCaml way i.e it will become a niche language with small community. Just 2cents from an average everyday software developer.

trenchgun 4y ago
I can understand that category theory is alien to most developers, but lambda calculus? Isn't it similarly basic knowledge in CS as Turing machines, finite state automata etc?
- donatzsky 4y ago
  Why do you assume that all developers have studied CS?
- alltheworlds 4y ago

the8472 4y ago

A recent change[0] on nightly rustc might help with incremental builds. And for repeated clean + full build cycles there's sccache[1]. [0] https://github.com/rust-lang/rust/pull/84762 https://github.com/rust-lang/rust/pull/84762 [1] https://github.com/mozilla/sccache https://github.com/mozilla/sccache

strager 4y ago
> A recent change[0] on nightly rustc might help with incremental builds. I tested with rustc Git commit c7572670a1302f5c7e245d069200e22da9df0316, which (I think) includes that change. > And for repeated clean + full build cycles there's sccache[1]. You're right. I included full builds in the article because almost-full builds happen a lot in C++ (after common certain header files, or if you think the build system broke something). I imagine almost-full builds rarely happen when working in Rust though, so maybe I should have deemphasized my full-build benchmarks.
- nicoburns 4y ago
  Yeah, I pretty much only do a clean build when I update the compiler. Or when I’m tweaking the build configuration itself.
- laund 4y ago
  Rust by default only does a full build really rarely. Ive gone through days, even weeks, of working on a project without a full build. Partial builds are of course way faster, especially if you use many dependencies (i know you don't). I mainly work with the bevy game engine in Rust, which has a lot of dependencies. Even if i don't use its dylib feature, i get 2-3s compiles. And that's on a project with multiple hundreds of thousands LoC when you include dependencies. With dylib, it goes down to 0.5-1 second builds. If your main conclusion is based on full builds, i would urge you to re-evaluate. The normal experience is just "cargo run" which rarely does a full build.
  - berkut 4y ago
    I've got a ~9k loc Bevy "game" I'm writing that doesn't use dylib and takes > 20 secs every update, purely because of the linking (I'm not using mold yet, but am using lld)... i.e., I type 'cargo build', it compiles the single .rs I changed almost instantly, but then I'm staring at: Building [=======================> ] 314/315: landscape(bin) for the next 20 seconds.

diimdeep 4y ago

Kudos. Are there languages on the horizon that could also be included as possible serious choices? Nim, zig, jai, or we all will be eternally waiting for future version of C++ ?

vegerot 4y ago
The author said he liked Rust and will switch to Rust if/when it builds faster than C++. So for the author at least, his answer is Rust!
strager 4y ago
I'd love to try Zig and Jai for this purpose, but they are not stable yet. If either becomes stable, I'll happily re-run this experiment. Nim doesn't look like the language for me.
- henry_viii 4y ago
  Would love to hear your opinion about V too especially since one of its main selling points is fast build times: https://vlang.io/#:~:text=Small%20and%20easy%20to%20build%20compiler https://vlang.io/#:~:text=Small%20and%20easy%20to%20build%20...

scns 4y ago

An OT hint to the author: I noticed that your RAM runs at 3800, that leads to a mismatch with the Infinity Fabrics' clock of 1800, i.e. increased latency. There are two solutions to this: Raise the FCLK (Fabric Clock) to 1900. This should be well tolerated by a Ryzen from the 5000 line. Lower the RAM clock to 3600 and set sharp sub timings. This will take a little more time but this tool [0] will help. Runs on Windows though, i have an old partition around for stuff like this, dunno if a VM will do. RAM OC became a time consuming hobby for some people, the rabbit hole... [0] https://www.techpowerup.com/download/ryzen-dram-calculator/ https://www.techpowerup.com/download/ryzen-dram-calculator/ (edit) add sentence bout windows

strager 4y ago
> Raise the FCLK (Fabric Clock) to 1900. Hmm, I think I did this, or something like this. I'llcheck tomorrow. (If I did, then my comment about not overclocking the CPU is misleading!)
Arch-TK 4y ago
>[0] https://www.techpowerup.com/download/ryzen-dram-calculator/ https://www.techpowerup.com/download/ryzen-dram-calculator/ Why on earth is this a binary program? Is there a document online which explains the maths or just implements it in javascript? I don't feel like spinning up a windows VM just to do this kind of calculation.
- scns 4y ago
  The author links a google sheet somewhere where he draws the values from. It is several miles long ;)
strager 4y ago
Current settings according to BIOS: Target CPU Speed: 3400MHz Target DRAM Frequency: 800MHz Target FCLK Frequency: 1800MHz DRAM timings: 19/20-19-19-19-39 (The DRAM CAS# Latency setting is set to 19, but "CHA" and "CHB" show 20. I don't know what this means exactly.) So I think you're right, scns. I was not getting the best performance. --- I changed FCLK to 1900MHz and re-ran the Linux C++-vs-Rust benchmarks. Some results: build+test w/o deps: 1847ms Rust, 1874ms C++ --> 1801ms Rust, 1837ms C++ incremental test-utf-8: 288ms Rust, 358ms C++ --> 280ms Rust, 350ms C++ So I did get a performance win for both C++ and Rust builds by fixing my FCLK. Thanks!

minigamedev 4y ago

A lot of times I felt like deleting the cargo.lock file should have been done inbetween optimising the rust build. There were changes to the used crates, and sometimes those changes don't propogate before a `cargo clean`, `cargo update` or something like that.

strager 4y ago
I'm not sure what you mean. Are you talking about the benchmarks? I run `cargo clean` before each benchmark. That should clean up changes to third-party crates, right?

Jyaif 4y ago

Very impressive investigation. Unfortunately matches my experience. There's a chance that Rust is the programming language with the longest build times ever.

Narishma 4y ago
Scala has it easily beaten in that regard.

kibwen 4y ago

I'm curious if the results at the very end for the copy-pasted code benchmarks are a result of linking. They seem to have chosen to use mold for C++ and not for Rust after seeing that it gave little benefit for small projects, but I would expect that to change as the project scales. In addition, I'd be interested in seeing how `cargo check` fares. In practice I do `cargo build` quite rarely, in favor of `cargo check` and `cargo test`.

chrismorgan 4y ago
I think https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac5d0e676ead553a3610284d47a3e/tools/bench-build-charts.py#L873 https://github.com/quick-lint/cpp-vs-rust/blob/f8d31341f5cac... indicates Rust is using Mold too. And I get the impression that the thing unit being benchmarked is a single crate—that is, one crate 8×, 16×, 24× the size (which puts the burden on the compiler, in areas that don’t parallelise well), rather than 8×, 16×, 24× as many crates (which puts the weight on the linker and on well-parallelised parts of the compiler). On the hardware in question, which has loads of cores, I’m very confident that Rust would fare considerably better with 24 17.1k-line crates (410k lines, larger due to duplicating the entire thing rather than just the lexer) than with the one 104.4k-line crate apparently tested.
- strager 4y ago
  > I’m very confident that Rust would fare considerably better with 24 17.1k-line crates (410k lines, larger due to duplicating the entire thing rather than just the lexer) than with the one 104.4k-line crate apparently tested. I didn't consider this in my scaling benchmark. You make a good point.
eximius 4y ago
Yea, comparing C++ build times and Rust build times seems to ignore the practical workflow differences between writing the two. Also, I think it's a little silly to say that Rust build times scale poorly (even if it isn't quite as good as C++). 100k LOC in, what, 6s at the end there is not so shabby if you're typically never building all of it.
4y ago

deleted 4y ago

[deleted]

sylware 4y ago

I would be more interesting to know how easier it is to write a naive rust compiler than a c++ naive compiler, aka actual syntax complexity comparison. To put that in perspective, how easier it is to write a naive rust compiler than a naive C compiler.

epage 4y ago

A surprising source of slow compile times can be declarative macros in Rust [0]. I believe the core of the problem is that it has to reparse the code to pattern match for the macro. One egregious patter is tt-munchers [1] where your macro is implemented recursively, requiring it to reparse the source on each call [2]. In one of my projects, someone decided to wrap a lot of core functions in simple macros (ie nt tt-munchers) to simplify the signatures. Unlike most macros which are used occasionally and have small inputs, this was a lot of input. When I refactored the code, I suspect dropping the macros is the reason CI times were cut in half and a clean `cargo check` went from 3s to 0.5s. [0]: https://nnethercote.github.io/2022/04/12/how-to-speed-up-the-rust-compiler-in-april-2022.html https://nnethercote.github.io/2022/04/12/how-to-speed-up-the... [1]: https://veykril.github.io/tlborm/decl-macros/patterns/tt-muncher.html https://veykril.github.io/tlborm/decl-macros/patterns/tt-mun... [2]: https://github.com/dtolnay/quote/blob/31c3be473d0457e29c4f47ab9cff73498ac804a7/src/lib.rs#L664-L746 https://github.com/dtolnay/quote/blob/31c3be473d0457e29c4f47...

deleted 4y ago
[deleted]
vegerot 4y ago
cool! Does quick-lint use this pattern? https://github.com/quick-lint/cpp-vs-rust https://github.com/quick-lint/cpp-vs-rust

bjt2n3904 4y ago

This author has a very meticulous experimental approach. I'm not a rust developer, but I enjoyed the article none the less.

tulio_ribeiro 4y ago

You should definitely keep an eye on Val and Carbon. They're being designed to be interoperable with C++. Both are designed to match C++'s performance while still being able to work with your existing C++ code. Both of these languages offer a more modern developer experience and are built with software and language evolution in mind. They have practical safety and testing mechanisms, etc. Definitely check them out if you're looking for a C++ replacement! https://github.com/val-lang https://github.com/val-lang https://github.com/carbon-language https://github.com/carbon-language

vegerot 4y ago
Do they compile fast? If not, why not just use Rust? The author said he’ll switch to Rust when the compilation speed improves
strager 4y ago
> Definitely check [Val and Carbon] out if you're looking for a C++ replacement! I don't want to base my project on an experimental language like Carbon or Val. I want a C++ replacement which compiles quickly. Do Carbon or Val compile quickly? If not, I don't know why you mention these languages.

throw10920 4y ago

> In theory, if you split your code into multiple crates, Cargo can parallelize rustc invocations. Because I have a 32-thread CPU on my Linux machine, and a 10-thread CPU on my macOS machine, I expect unlocking parallelization to reduce build times. Why can't Cargo parallelize rustc invocations per-file, like C++ can?

lights0123 4y ago
rustc needs to look at the entire crate because there's no header files that provide information about other items like C++. rustc does parallelize internally though: https://doc.rust-lang.org/rustc/codegen-options/index.html#codegen-units https://doc.rust-lang.org/rustc/codegen-options/index.html#c...
- throw10920 4y ago
  > Increasing parallelism may speed up compile times, but may also produce slower code Why do I have to make the choice between faster compiles and faster binaries?
  - lights0123 4y ago
    The exact same thing happens in C++—when you're not using LTO, you can pass multiple source files to the compiler at a time to allow these files to be optimized together, but keeping them separated at a file-level prevents optimizations such as inlining functions between them. Compiling with and without optimizations is another example of that choice.
steveklabnik 4y ago
Because the unit of compilation is a crate, not a file. You don't pass each file to rustc, you pass the crate root only.

dcow 4y ago

This is a nice article. But, I don't really understand the conclusion that Rust compile times are a problem (and that C++ times are too). Rust is not a tool I pick up because 3s compile times are a nonstarter and I need sub 1s to feed my developer ADAH. I spend orders of magnitude more time writing Rust than I do compiling it so a 2s difference is utterly marginal. I don't know.. this obsession with compile times in the last 5 years seems rather irrelevant to me, not that I don't appreciate lower ones. But "problematic" is not an adjective I'd use.

epolanski 4y ago
I think the article points a good reason why: even with an incredibly powerful machine it takes many, many minutes to compile projects like chromium. On a "normal" machine, it takes several hours. This is non-ideal and chromium is probably and already heavily optimized and well written project with lots of documentation.
- dcow 4y ago
  In what language or toolset can you compile a project like Chromium faster? Big projects take a while to compile. Nobody has broken this rule yet so why are we fixated on it being problematic? Obviously, we could not compile the project and eschew all the optimizations we get, but then the end user would be unhappy. I don't know, I've always found the tradeoff to be quite understandable and inoffensive: if I want faster development (among other things), I use an interpreted scripting language. If I want a fast end product (among other things) I compile and optimize to machine code beforehand (obviously it's not that simple, but you get the point).
  - vegerot 4y ago
    Something to stress is that the article is not just about compile times. It’s about compile+test times. If Chrome were written in Python it’d be hella slower iteration times
  - netr0ute 4y ago
    > In what language or toolset can you compile a project like Chromium faster? Go is super duper fast, so there's that.

ho_schi 4y ago

For full builds, C++ will take longer to compile than Rust (i.e. Rust wins). This is because of C++'s #include feature and C++ templates, which need to be compiled once per .cpp file. This compilation is done in parallel, but parallelism is imperfect. The author gives later on within the article a hint upon "and externing template instantiations". This allows to compile a template just once with a given set of template arguments. Shall reduce reduce compile-time. I assume it also reduces size of created object-files? Explicit instantiation of templates is supported since C++11. Therefore templates with one set of template arguments can be are declared in multiple files and defined just once - in one file. The keyword for this is extern upon each declaration. Please note, that all member of a template will be instantiated whether used or not. The user "strager" mentioned that briefly. see: https://en.cppreference.com/w/cpp/language/class_template#Class_template_instantiation https://en.cppreference.com/w/cpp/language/class_template#Cl... https://en.cppreference.com/w/cpp/language/function_template#Function_template_instantiation https://en.cppreference.com/w/cpp/language/function_template...

strager 4y ago
> I assume it also reduces size of created object-files? Normal template instantiations are deduplicated by the linker. Total object file size is smaller with explicit template instantiations, but the size of the final executable is the same with both styles.
- ho_schi 4y ago
  Wow. Thanks :)

Night_Thastus 4y ago

C++ build systems and compilation time in general is basically my most significant complaint as a C++ developer. There are so many disparate build systems, none of them are intuitive, none of them do much in the way of error checking or give good user feedback, and the resulting build is always slow. And as for the code itself, there's no type of linter or checker that will tell you if there's a way to save on compile time. "Hey, this could be a forward decl instead of a include!", or "Hey, this include is completely unused!" or some such. (There's include-what-you-use but I could not for the life of me figure out how to make it work) I know the problems are hard, I just wish more effort was expended to make the whole process less painful and slow.

strager 4y ago
I have found Clang's -ftime-trace flag helpful in finding bloated #include-s and templates. Also, I have analyzed the .ninja_log file (for CMake+Ninja) to find slow-to-compile .cpp files.

hacksoi 4y ago

Rust's only selling point is the way it tries to solve memory errors, but it's based on a style of memory management that is inferior. There are other styles that prevent such errors from occurring in the first place.

roeles 4y ago
Please elaborate. I have zero knowledge about rust, so I am wondering what you're referring to.

teknico 4y ago

> (Unless I become enchanted by Zig first.) You probably will. :-)