14 ms·

The Rust Compilation Model Calamity

• shaggie76 7y ago
  What I'm more interested in their link times; how often do you build everything from scratch in a day?
• Eikon 7y ago
  > TiKV is a relatively large Rust codebase, with 2 million lines of Rust. I don't understand why so many absolutely wants to inflate this number, as it doesn't mean anything about the product. This inflation also leads to making part of the analysis of this article just wrong. Running scc in TIKV's "src" folder: Lines Blanks Comments Code 78442 7259 5476 65707
  • azakai 7y ago
    The article says the 2 million includes vendored code. I assume that means all the dependencies and their dependencies. (So yes, the project itself is just a small fraction of the total lines, but I'm not sure if the measurement here was of compiling the dependencies too, or not.)
• xiaodai 7y ago
  Is it slow compared to C++?
  • pjmlp 7y ago
    Yes. C++ is horribly slow if you overuse templates, and not take advantage of binary dependencies. If every module compiles to its own library (not object file), got proper translation units, and a fast linker, compiling the stuff you're currently working on is relatively fast.
  • antientropic 7y ago
    No. My experience is that Rust compile times are pretty reasonable compared to typical C++ projects (and it's often an apples-to-oranges comparison: an initial Cargo build does a lot more, building all your dependencies in the configuration that you want, rather than using prebuilt binary dependencies as is common in the C/C++ world). To be honest, I don't really understand the self-flagellation in the Rust community about compile times.
    • nestorD 7y ago
      > I don't really understand the self-flagellation in the Rust community about compile times. I believe the two main factor are the fact that: - many Rust users come from languages with a faster writing->running cycle (there is a surprising number of Python users starting to use Rust) - Rust takes prides in its speed, if something is slow it is thus seen as a failure that should be fixed (even if it has perfectly reasonable reasons to be slow)
• dwohnitmok 7y ago
  Looking at the TiKV compile time graph, if that's for 2 million lines of real-world (not synthetically generated) code, that actually seems kind of reasonable. ~12 min for a full release build? ~5 min for a full dev build? What looks like several seconds for an incremental build? I suppose it's good that a (former?) core contributor and large user of Rust holds the language to such high standards, but this doesn't seem especially bad. I'm kind of curious how much faster compiling a large Go codebase is. How fast, e.g. does the entirety of Kubernetes compile? I'd imagine it's probably under a minute, but is it several seconds?
  • nneonneo 7y ago
    The only relevant info I could dig up was this issue from Kubernetes in 2016: https://github.com/kubernetes/kubernetes/issues/27444 https://github.com/kubernetes/kubernetes/issues/27444 It seems to be discussing the speed of building Kubernetes on their build-servers, targeting multiple architectures at once. Unfortunately all the build servers are internal to Google so I don't know exactly what's being included (or what's being run), but this comment (https://github.com/kubernetes/kubernetes/issues/27444#issuecomment-226438757 https://github.com/kubernetes/kubernetes/issues/27444#issuec...) details the breakdown of time spent: 30m of build 7m on cluster startup 12m on tests 7m on cluster teardown So, taken with an enormous hunk of rock salt, it seems like Kubernetes might take around 30m to build on a cluster, to build (possibly multiple) release artifacts. Seems in line with Rust.
    • melling 7y ago
      What’s the state of parallel builds? We’re getting 16/32 core cpus. In a few years, we’re all going have Threadrippers. Of course everything can’t be done in parallel.
      • granzymes 7y ago
        Almost all parts of the compiler (at least for LLVM) are still strictly single-threaded. You can't even use threads yourself if you write your own LLVM module pass.
• gHosts 7y ago
  Walter Bright has said on several occasions that compilation speed was one of his design goals for D.
  • unlinked_dll 7y ago
    Recently on the "On the Metal" podcast (which is fantastic) Johnathon Blow had a mini rant on compilation speeds and mentioned that the Jai compiler is incredibly fast.
    • atombender 7y ago
      Jai apparently — I think it's mentioned in that podcast — has a dedicated backend for debug builds that's optimized for fast code generation and doesn't use LLVM. Sounds like what Cranelift is trying to do for Rust.
      • unlinked_dll 7y ago
        This seems to be the move for languages moving forward.
        • pjmlp 7y ago
          That is already the approach in Lisp, Scheme, Java, .NET, mainframe language environments, OCaml, Haskell, Eiffel, Dart, D and plenty others. By having a mix of interpreters, repls, JIT and AOT compilers, one can mix and match, using the fastest ones for development and the slow ones for the final release build.
  • zaphar 7y ago
    If I'm reaching for Rust it's more than likely because I want it's borrow checker and pattern matching on enumerated types. To my knowledge D has neither of those so it's not really in the running here.
    • earenndil 7y ago
• pixelpoet 7y ago
  Performance numbers without specifying the machine it's running on is pretty much useless. How is this not common knowledge in 2020? Another thing: I'm not trying to excuse slow builds, but I'm frustrated that people basically always do benchmarks on laptops. Like, come on, if you're doing serious work, use a serious computer; laptops are overpriced, underpowered and overheating compared to a similarly priced PC, not to mention potential inconsistencies due to thermal variation (2nd+ run is more likely to throttle than 1st). This "mobility über alles" weighting truly baffles me in the face of how much better and cost efficient PCs are. How has no one done a cost/benefit analysis of your highly paid devs being slowed down O(time) versus O(1) hardware purchase? tl;dr How does it perform on a 5 GHz i9 9900KS with 32 GB (standard since 2012!) m.2 SSD that still costs less than some shiny web browsing toy? Edit: Wow, crazy downvotes (was at -4 in about 3 minutes) and zero substantial responses to the points raised. I really expected better on HN.
  • jupp0r 7y ago
    > This "mobility über alles" weighting truly baffles me in the face of how much better and cost efficient PCs are. I absolutely disagree. The mental overhead and time it takes to synchronize my work over multiple machines stands in no comparison to the few thousand dollars I'd save every few years by having desktop computers everywhere. If compilation is slow, use a distributed compiler infrastructure (if your language supports this, mostly writing C++ here). If you are doing serious work, do it in the cloud. If you have an actually big codebase to compile, you need to scale that horizontally anyway and an expensive workstation under your desk will take hours anyway.
    • pixelpoet 7y ago
      > If you are doing serious work, do it in the cloud. In some sense I think we agree, except the cloud is just someone else's powerful computer. Laptop as thin client via rdesktop is how I developed (also C++) for years, but only when I was at home away from my work computer. With the cloud you're always away from your work computer. > If you have an actually big codebase to compile, you need to scale that horizontally anyway and an expensive workstation under your desk will take hours anyway. Agreed, and that's further along the same vector: if you have performance issues with compilation, consider also (not only, before someone jumps on me) using bigger machine(s) for the job. Anyway, I'm going to stop here. Feels like I found the most hated tech opinion ever... at least eventually there were some thought-out replies (and thank you for yours).
• okareaman 7y ago
  If the code is modularized and put in libraries, how often do you need to build all 2 million lines of code after making a change. What am I missing?
  • skybrian 7y ago
    This depends on where you are in the stack. The perspective of someone working on a low-level, widely used library is different from someone working on a top-level application.
  • reggieband 7y ago
    He does say: >Per-compilation-unit code-generation — rustc generates machine code each time it compiles a crate, but it doesn't need to — with most Rust projects being statically linked, the machine code isn't needed until the final link step. There may be efficiencies to be achieved by completely separating analysis and code generation. It was my first thought as well but I have no knowledge how Rust linking works so I don't know how reasonable caching module compilation units would be.
    • okareaman 7y ago
      Maybe a slow linker is the problem
      • Ar-Curunir 7y ago
        The system linked is pretty slow; many folks report massive speed ups by using LLD
        • MaulingMonkey 7y ago
          Replacing BFD with Gold (ELF-only) or LLD can be a huge win. On the last major production codebase I did this for, linking a single artifacts went from ~60 seconds (BFD) to ~10 (Gold). Multiply by nearly 100 artifacts (several dozen libraries, several dozen tools, per-library unit tests, benchmarks, applications, etc. all statically linking some common code), and some basic incremental "I touched a single .cpp file" builds went from 30+ minutes to maybe 5 minutes for a single platform x config combination.
• adamnemecek 7y ago
  You have a monorepo with 2 million lines of code? How long is the compilation supposed to take?
  • disconnected 7y ago
    I don't know how long it is "supposed" to take, but you can compile a Linux kernel (27.8M lines of code, though a good chunk of it probably isn't going to be compiled anyway because you don't need it, and another good chunk is architecture specific) in under 10 minutes on relatively modest (but modern) hardware. On the other hand, something like Chromium (25M lines of code) will take about 8 hours, and bring your machine to its knees as it consumes ALL available resources (granted, last I did this I only had 8GB of RAM, and I was running my desktop at the time... including Chromium). I don't remember exactly how long Firefox takes to build, but I remember it was significantly less time (maybe 3 hours?). So... it depends? On a lot of things? (btw, LoC numbers were pulled from the first legitimate looking result I could find on a quick search... take with a grain of salt... also, compilation times are a rough approximation based on my observations... that it with a truckload of salt)
    • gpderetta 7y ago
      Linking can consume huge amount of memory, especially for C++ code. For a large project 8GB might be very low. On our codebase we saw huge differences between 16GB machines and 24GB machines as the 16GB could not run some linking steps in parallel without swapping. Does Chromium build use LTO (I'm pretty sure FF does)? That's also a huge resource sink a doesn't parallelize as well (a lot of the optimization will be delayed to linking)
    • Yoric 7y ago
      Last time I checked (admittedly a few years ago), a full build of Firefox on my beefy laptop was ~2h while working on something else and Chromium was 10h+ bringing the system to its knees.
  • azakai 7y ago
    The 2 million figure includes vendored code. A quick count suggests the repo itself has less than 200K, https://codetabs.com/count-loc/count-loc-online.html https://codetabs.com/count-loc/count-loc-online.html
• epage 7y ago
  I feel like the breakdown in problems should include real world data to back it up. Hopefully future articles will do so. In particular, people like to put blame on the borrow checker when I thought it wasn't all that bad. From my understanding, the worst offenders are LLVM (because rustc is using it in a way that they haven't optimized for) and linking (which there are experiments with using lld which people report huge gains with. Besides the tone, my main gripe with the article and some discussions I've seen elsewhere is mixing implementation trade offs with design trade offs. For example, LLVM and not doing your own optimization passes can be important for time-to-market. The only reasonable alternative that I can think of without sacrificing time-to-market is between LLVM or a C backend. Delaying Rust would have made it irrelevant. Now for some context for those not as familiar with Rust: > Stack unwinding — stack unwinding after unrecoverable exceptions traverses the callstack backwards and runs cleanup code. It requires lots of compile-time book-keeping and code generation. This is for asserts (panics) and can be toggled with a flag. It isn't inherent to the language though some older code uses it extensively (like rustc) because it predates the current language design (from what I've read). > Tests next to code — Rust encourages tests to reside in the same codebase as the code they are testing. With Rust's compilation model, this requires compiling and linking that code twice, which is expensive, particularly for large crates. This is in a list of negative impacts of features but people outside of Rust reading this might not catch the why this is done. Tests inline to your code have access to your private interfaces. External tests are for integration testing and build against your library like anyone else would.
  • gameswithgo 7y ago
    The worst offenders vary a lot by particular codebase. I had a crate where the compile times were bad due to trait bounds. I was able to refactor to drop times from ~2 minutes to ~6seconds! That should'nt have been necessary for me to refactor though. Sometimes its llvm, sometimes its rust macros encouraging HUGE functions which then leads to slow llvm optimizations. Sometimes it's large use of generics. Sometimes it is the linker. There are various ways to hit slow compile times.
    • rhn_mk1 7y ago
• bitwize 7y ago
  Uh, yeah, no shit. Code is hopefully run many, many more times than it's compiled. If there's computation that can be front-loaded into compilation time, it makes sense to do it there rather than waste your or your clients' CPU cycles doing the compiler's job at runtime.
  • gpderetta 7y ago
    It is reasonable to spend very long times for particularly optimized build (via LTO for example). But it is also important to make dev builds as fast as possible.
• deleted 7y ago
  [deleted]
• hermitdev 7y ago
  Complaining about 15-30 minute build time? Bitch, please. I used to work on a C++ system that took +16 hours to do a clean build.
• WalterBright 7y ago
  I don't know how the Rust compiler is built. However, I am implementing an Ownership/Borrowing system for the D programming language, and to make it work requires Data Flow Analysis. DFA is slow. It's normally only used for optimized builds, which is why optimized builds are slow. But when DFA is a required semantic feature of the language, it has to be done for debug builds, too, and so they're going to be slow.
  • jez 7y ago
    Can you elaborate on why data flow analysis is slow, necessarily? For example, does it need whole-program information, or does it do some sort of fixed point, or is the algorithmic complexity super-linear, or something else?
    • WalterBright 7y ago
      It tends to be quadratic, based on the number of variables and the cyclomatic complexity. The DFA equations cannot be solved directly, but only iteratively until a solution is reached. Debug code generation, on the other hand, involves none of that and the compiler just blasts out code expression by expression. The time goes up linearly with the number of expressions. DFA (and the O/B DFA) is done at the function level. It can be done globally, but I try to keep the compile times reasonable.
      • mntmoss 7y ago
        Is an expensive DFA something that could be mitigated with source code hints? An 80% solution might do the trick.
        • WalterBright 7y ago
          People using DFA tend to want a 100% solution. Rust, for example, sets a store on a 100% solution.
      • kragen 7y ago
        Can you cache the solution from the last compile and check that it still works in linear time without introducing nondeterminism?
• toolslive 7y ago
  One of the main reason I got into Ocaml (coming from C++) was that it had bytecode compilation (and obviously a bytecode interpreter) next to the native one. This is really fast (5-7 times faster last time I measured) and perfect during development, when you don't really care about performance.
  • pdimitar 7y ago
    I wanted to use OCaml several types but the lack of parallelism and the GIL have put me off. Still waiting for Multicore OCaml.
    • toolslive 7y ago
      yes. if you need parallelism, you need to go via the process model. (we did this, but it's quite painful) Anyway, the official road map states something along the lines of "Multicore: probably next release" (but that was also said for previous releases)
      • pdimitar 7y ago
        Sadly all you said has been confirmed by other people as well. I really like OCaml. It's mind-bogglingly fast and well-made in basically almost every regard I can think of. The lack of proper parallelism nowadays however is a huge NOPE. I know JaneStreet and Inria have a lot of valid usages for it and don't care what the rest of us think but it's very sad to have one of of the highest quality languages and compilers be left to fringe usage of several organisations only. :(
    • pjmlp 7y ago
      It supports parallelism the same way UNIX clones did for their first two decades, which is quite good for plenty of workloads, and then there is Lwt for concurrency. And it is safer. :) In any, case multicore runtime is getting closer, https://discuss.ocaml.org/t/multicore-ocaml-january-2020-update/5090 https://discuss.ocaml.org/t/multicore-ocaml-january-2020-upd...
• jillesvangurp 7y ago
  Compilation time is mostly an issue during the edit, compile, debug cycle and inside IDEs where things are compiled on almost every key stroke. I was reading about the rust analyzer recently, which is a new language server for Rust that is explicitly designed to address compilation latency in IDEs. This also happened with Java back in the day when IBM developed their own incremental java compiler for use inside of Eclipse (technically this predates their IDE; I was using an early version in 1998). It gives that IDE an edge over things like Intellij in terms of compile latency, which is orders of magnitudes slower in intellij (measured in seconds instead of ms.). Intellij does a lot of work to hide the issues through elaborate caching, lots of things happening asynchronously, etc. They even attempted to integrate the eclipse compiler. But it's very noticable if you are used to fast feedback on your code correctness. Another important aspect that they are trying to address in the Rust Analyzer that the Eclipse Java compiler also addresses is handling partially correct code. If you are editing, the end state is of course correctly compiling code. But in between edits when it doesn't compile is exactly when you need your IDE to be helpful. Eclipse used to be really good at this and update in real time on basically every key-press. The red squiggly lines disappearing basically means "now your code is compiling fine". Intellij works a lot slower and worse, ends up actively lying quite often with both false positives and negatives being very common (the dreaded reset caches feature is a top level menu item for this reason). So, it's an important problem. Rust is optimized for run time safety and performance. The same infrastructure that enables that should in principle also be able to enable a great developer experience when it comes to IDE friendly features. [edit] This article what I'm referring to above: https://www.infoq.com/news/2020/01/rust-analyser-ide-support/ https://www.infoq.com/news/2020/01/rust-analyser-ide-support... (interview with one of the Rust Analyzer devs)
• scoutt 7y ago
  > Servo is a web browser built in Rust, and Rust was created with the explicit purpose of building Servo. I didn't know this. Is Servo the best example of development/coding standard/features/implementation for Rust?
• polskibus 7y ago
  Could the slow compile times be also due to people using laptops with slow CPUs and drives compared to high-end desktop CPUs and NMVes? We're using desktop PCs at my work and it made a huge difference in our dev flow (C#, VS) compared to the Thinkpads we used before.
  • deleted 7y ago
    [deleted]
• jedisct1 7y ago
  The great thing about Rust is that when you switch to Go, the development experience feels incredibly fast and great. Autocompletion in VSCode works, instantaneously, all the time. Errors are shown in real time. Compilation is fast. Error messages are human readable. Updating Go or dependencies doesn't break existing code.
• ansible 7y ago
  Of all the things mentioned in the article that keep Rust compiles from being fast, the one that sticks out to me is the multi-threaded compilation. That could result in some dramatic reductions in compile time. With a lot of the other things that are a part of the design of the Rust language itself, I'm happy with how they are, even if that slows down complication somewhat.
• carapace 7y ago
  > When I worked daily on the Rust compiler, it was common for me to have at least three copies of the repository on the computer, hacking on one while all the others were building and testing. I would start building workspace 1, switch terminals, remember what's going on over here in workspace 2, hack on that for a while, start building in workspace 2, switch terminals, etc. Little flow, constant context switching. And you didn't see the problem!? When Prof. Wirth was making the Oberon compiler he had a heuristic that any language feature which made the compiler slower at compiling itself was reworked or discarded.
• Ericson2314 7y ago
  I'm glad the comments mostly get it: only incremental build granularity and parallelism matter. Fully rebuilds I wholely don't care about, and neither should anyone else: the goal is to never need to do them.