16 ms·

Why is the Rust compiler so slow?

• ahartmetz 1y ago
  That person seems to be confused. Installing a single, statically linked binary is clearly simpler than managing a container?!
  • hu3 1y ago
    From the article, the goal was not to simplify, but rather to modernize: > So instead, I'd like to switch to deploying my website with containers (be it Docker, Kubernetes, or otherwise), matching the vast majority of software deployed any time in the last decade. Containers offer many benefits. To name some: process isolation, increased security, standardized logging and mature horizontal scalability.
    • adastra22 1y ago
      So put the binary in the container. Why does it have to be compiled within the container?
      • hu3 1y ago
        That is what they are doing. It's a 2 stage Dockerfile. First stage compiles the code. This is good for isolation and reproducibility. Second stage is a lightweight container to run the compiled binary. Why is the author being attacked (by multiple comments) for not making things simpler when that was not claimed that as the goal. They are modernizing it. Containers are good practice for CI/CD anyway.
        • MobiusHorizons 1y ago
          That’s a reasonable deployment strategy, but a pretty terrible local development strategy
          • taberiand 1y ago
            Devcontainers are a good compromise though - you can develop within a context that can be very nearly identical to production; with a bit of finagling you could even use the same dockerfile for the devcontainer, and the build image and the deployed image
        • AndrewDucker 1y ago
          I'm not sure why "complicate things unnecessarily" is considered more modern. Don't do what you don't need to do.
          • hu3 1y ago
            You realize the author is compiling a Rust webserver for a static website right? They are already long past the point of "complicate things unnecessarily". A simple Dockerfile pales in comparison.
        • adastra22 1y ago
          Because he spends a good deal of the intro complaining that this makes his dev practice slow. So don’t do it! It has nothing to do with docker but rather the fact he is wiping the cache on every triggered build.
    • dwattttt 1y ago
      Mightily resisting the urge to be flippant, but all of those benefits were achieved before Docker. Docker is a (the, in some areas) modern way to do it, but far from the only way.
    • a3w 1y ago
      Increased security compared to bare hardware, lower than VMs. Also, lower than Jails and RKT (Rocket) which seems to be dead.
    • eeZah7Ux 1y ago
      > process isolation, increased security no, that's sandboxing.
  • jerf 1y ago
    Also strikes me as not fully understanding what exactly docker is doing. In reference to building everything in a docker image: "Unfortunately, this will rebuild everything from scratch whenever there's any change." In this situation, with only one person as the builder, with no need for CI or CD or whatever, there's nothing wrong with building locally with all the local conveniences and just slurping the result into a docker container. Double-check any settings that may accidentally add paths if the paths have anything that would bother you. (In my case it would merely reveal that, yes, someone with my username built it and they have a "src" directory... you can tell how worried I am about both those tidbits by the fact I just posted them publicly.) It's good for CI/CD in a professional setting to ensure that you can build a project from a hard drive, a magnetic needle, and a monkey trained to scratch a minimal kernel on to it, and boot strap from there, but personal projects don't need that.
    • scuff3d 1y ago
      Thank you! I got a couple minutes in and was confused as hell. There is no reason to do the builds in the container. Even at work, I have a few projects where we had to build a Java uber jar (all the dependencies bundled into one big far) and when we need it containerized we just copy the jar in. I honestly don't see much reason to do builds in the container unless there is some limitation in my CICD pipeline where I don't have access to necessary build tools.
      • mike_hearn 1y ago
        It's pretty clear that this whole project was god-tier level procrastination so I wouldn't worry too much about the details. The original stated problem could have been solved with a 5-line shell script.
        • scuff3d 1y ago
          Not strictly related, but I got to the parts about using a dependency to speed up builds in the container, and that his website has "hundreds" of Rust dependencies, and I was reminded why I get so annoyed with Rust. It's a great language, but the practice of just duct taping a bunch of dependencies together drives me nuts.
  • vorgol 1y ago
    Exactly. I immediately thought of the grug brain dev when I read that.
• kelnos 1y ago
  > This is... not ideal. What? That's absolutely ideal! It's incredibly simple. I wish deployment processes were always that simple! Docker is not going to make your deployment process simpler than that. I did enjoy the deep dive into figuring out what was taking a long time when compiling.
  • quectophoton 1y ago
    One thing I like about Alpine Linux is how easy and dumbproof it is to make packages. It's not some wild beast like trying to create `.deb` files. If anyone out there is already fully committed to using only Alpine Linux, I'd recommend trying creating native packages at least once.
    • eddd-ddde 1y ago
      I'm not familiar with .deb packages, but one thing I love about Arch Linux is PKGBUILD and makepkg. It is ridiculously easy to make a package.
• tmtvl 1y ago
  Just set up a build server and have your docker containers fetch prebuilt binaries from that?
• adastra22 1y ago
  As a former C++ developer, claims that rust compilation is slow leave me scratching my head.
  • shadowgovt 1y ago
    I thorougly enjoy all the work on encapsulation and reducing the steps of compilation to compile, then link that C does... Only to have C++ come along and undo almost all of it through the simple expedient of requiring templates for everything. Oops, changed one template in one header. And that impacts.... 98% of my code.
  • eikenberry 1y ago
    Which is one of the reasons why Rust is considered to be targeting C++'s developers. C++ devs already have the Stockholm syndrome needed to tolerate the tooling.
    • MyOutfitIsVague 1y ago
      Rust's compilation is slow, but the tooling is just about the best that any programming language has.
      • GuB-42 1y ago
        How good is the debugger? "edit and continue"? Hot reload? Full IDE? I don't know enough Rust, but I find these aspects are seriously lacking in C++ on Linux, and it is one of the few things I think Windows has it better for developers. Is Rust better?
        • adastra22 1y ago
          No idea because I never do that. Nor does any rust programmer I know. Which may answer your question ;)
          • frollogaston 1y ago
• ecshafer 1y ago
  The Rust compiler is slow. But if you want more features from your compiler you need to have a slower compiler, there isn't a way around that. However this blog post doesn't really seem to be around that and more an annoyance in how they deploy binaries.
• MangoToupe 1y ago
  I don't really consider it to be slow at all. It seems about as performant as any other language this complexity, and it's far faster than the 15 minute C++ and Scala build times I'd place in the same category.
  • randomNumber7 1y ago
    When C++ templates are turing complete is it pointless to complain about the compile times without considering the actual code :)
  • mountainriver 1y ago
    I also don’t understand this, the rust compiler hardly bothers me at all when I’m working. I feel like this is due to how bad it was early on and people just sticking to that narrative
  • BanterTrouble 1y ago
    The memory usage is quite large compared to C/C++ when compiling. I use Virtual Machines for Demos on my YouTube Channel and compiling something large in Rust requires 8GB+. In C/C++ I don't even have to worry about it.
    • windward 1y ago
      I can't say the same. Telling people to use `-j$(nproc)` in lieu of `-j` to avoid the wrath of the OOM-killer is a rite of passage
    • gpm 1y ago
      I can't agree, I've had C/C++ builds of well known open source projects try to use >100GB of memory...
      • BanterTrouble 1y ago
        Maybe something else is going on then. I've done builds of some large open source projects and most of the time they was maxing the cores (I was building j32) but memory usage was fine. Out of interest what were they?
• AndyKelley 1y ago
  My homepage takes 73ms to rebuild: 17ms to recompile the static site generator, then 56ms to run it. andy@bark ~/d/andrewkelley.me (master)> zig build --watch -fincremental Build Summary: 3/3 steps succeeded install success └─ run exe compile success 57ms MaxRSS:3M └─ compile exe compile Debug native success 331ms Build Summary: 3/3 steps succeeded install success └─ run exe compile success 56ms MaxRSS:3M └─ compile exe compile Debug native success 17ms watching 75 directories, 1 processes
  • echelon 1y ago
    Zig is a small and simple language. It doesn't need a complicated compiler. Rust is a large and robust language meant for serious systems programming. The scope of problems Rust addresses is large, and Rust seeks to be deployed to very large scale software problems. These two are not the same and do not merit an apples to apples comparison. edit: I made some changes to my phrasing. I described Zig as a "toy" language, which wasn't the right wording. These languages are at different stages of maturity, have different levels of complexity, and have different customers. They shouldn't be measured against each other so superficially.
    • steveklabnik 1y ago
      Come on now. This isn't acceptable behavior. (EDIT: The parent has since edited this comment to contain more than just "zig bad rust good", but I still think the combative-ness and insulting tone at the time I made this comment isn't cool.)
      • echelon 1y ago
        > but I still think the combative-ness and insulting tone at the time I made this comment isn't cool Respectfully, the parent only offers up a Zig compile time metric. That's it. That's the entire comment. This HN post about Rust is now being dominated by a cheap shot Zig one liner humblebrag from the lead author of Zig. I think this thread needs a little more nuance.
• namibj 1y ago
  Incremental compilation good. If you want, freeze the initial incremental cache after a single fresh build to use for building/deploying updates, to mitigate the risk of intermediate states gradually corrupting the cache. Works great with docker: upon new compiler version or major website update, rebuild the layer with the incremental cache; otherwise just run from the snapshot and build newest website update version/state, and upload/deploy the resulting static binary. Just set so that mere code changes won't force rebuilding the layer that caches/materializes the fresh clean build's incremental compilation cache.
  • maccard 1y ago
    The intermediates for my project are 150GB+ alone. Last time I worked with docker images that large we had massive massive problems.
  • deleted 1y ago
    [deleted]
• luppy47474 1y ago
  [flagged]
• RS-232 1y ago
  Is there an equivalent of ninja for rust yet?
  • steveklabnik 1y ago
    It depends on what you mean by 'equivalent of ninja.' Cargo is the standard build system for Rust projects, though some users use other ones. (And some build those on top of Cargo too.)
• o11c 1y ago
  TL;DR `async` considered harmful. For all the C++ laughing in this thread, there's really only one thing that makes C++ slow - non-`extern` templates - and C++ gives you a lot more space to speed them up than Rust does.
  • int_19h 1y ago
    C++ also has async these days. As for templates, I can't think of anything about them that would speed up things substantially wrt Rust aside from extern template and manually managing your instantiations in separate .cpp files. Since otherwise it's fundamentally the same problem - recompiling the same code over and over again because it's parametrized with different types every time. Indeed, out of the box I would actually expect C++ to do worse because a C++ header template has potentially different environment in every translation unit in which that header is included, so without precompiled headers the compiler pretty much has to assume the worst...
    • sgt 1y ago
      What happened with Zig and async? Last I heard they might never implement it.
• b0a04gl 1y ago
  [dead]
• senderista 1y ago
  WRT compilation efficiency, the C/C++ model of compiling separate translation units in parallel seems like an advance over the Rust model (but obviously forecloses opportunities for whole-program optimization).
  • woodruffw 1y ago
    Rust can and does compile separate translation units in parallel; it's just that the translation unit is (roughly) a crate instead of a single C or C++ source file.
    • EnPissant 1y ago
      And even for crates, Rust has incremental compilation.
  • deleted 1y ago
    [deleted]
• ic_fly2 1y ago
  This is such a weird cannon on sparrows approach. The local builds are fast, why would you rebuild docker for small changes? Also why is a personal page so much rust and so many dependencies. For a larger project with more complex stuff you’d have a test suite that takes time too. Run both in parallel in your CI and call it a day.
• taylorallred 1y ago
  So there's this guy you may have heard of called Ryan Fleury who makes the RAD debugger for Epic. The whole thing is made with 278k lines of C and is built as a unity build (all the code is included into one file that is compiled as a single translation unit). On a decent windows machine it takes 1.5 seconds to do a clean compile. This seems like a clear case-study that compilation can be incredibly fast and makes me wonder why other languages like Rust and Swift can't just do something similar to achieve similar speeds.
  • troupo 1y ago
    There's also Jonathan Blow's jai where he routinely builds an entire game from scratch in a few seconds (hopefully public beta will be released by the end of this year).
  • js2 1y ago
    "Just". Probably because there's a lot of complexity you're waving away. Almost nothing is ever simple as "just".
    • pixelpoet 1y ago
      At a previous company, we had a rule: whoever says "just" gets to implement it :)
      • forrestthewoods 1y ago
        I wanted to ban “just” but your rule is better. Brilliant.
    • taylorallred 1y ago
      That "just" was too flippant. My bad. What I meant to convey is "hey, there's some fast compiling going on here and it wasn't that hard to pull off. Can we at least take a look at why that is and maybe do the same thing?".
      • steveklabnik 1y ago
• charcircuit 1y ago
  Why doesn't the Rust ecosystem optimize around compile time? It seems a lot of these frameworks and libraries encourage doing things which are slow to compile.
  • steveklabnik 1y ago
    The ecosystem is vast, and different people have different priorities. Simple as that.
  • nicoburns 1y ago
    It's starting to, but a lot of people are using Rust because they need (or want) the best possible runtime performance, so that tends to be prioritised a lot of the time.
  • int_19h 1y ago
    It would be more accurate to say that idiomatic Rust encourages doing things which are slow to compile: lots of small generic functions everywhere. And the most effective way to speed this up is to avoid monomorphization by using RTTI to provide a single generic compiled implementation that can be reused for different types, like what Swift does when generics across the module boundary. But this is less efficient at runtime because of all the runtime checks and computations that now need to be done to deal with objects of different sizes etc, many direct or even inlined calls now become virtual etc. Here's a somewhat dated but still good overview of various approaches to generics in different languages including C++, Rust, Swift, and Zig and their tradeoffs: https://thume.ca/2019/07/14/a-tour-of-metaprogramming-models-for-generics/ https://thume.ca/2019/07/14/a-tour-of-metaprogramming-models...
  • kzrdude 1y ago
    Lots of developers in the ecosystem avoid proc macros for example. But going as far as avoiding monomorphisation and generics is not that widespread
• aappleby 1y ago
  you had a functional and minimal deployment process (compile copy restart) and now you have...
  • canyp 1y ago
    ...Kubernetes. Damn, this makes such a great ad.
• OtomotO 1y ago
  It's not. It's just doing way more work than many other compilers, due to a sane type system. Personally I don't care anymore, since I do hotpatching: https://lib.rs/crates/subsecond https://lib.rs/crates/subsecond Zig is faster, but then again, Zig isn't memory save, so personally I don't care. It's an impressive language, I love the syntax, the simplicity. But I don't trust myself to keep all the memory relevant invariants in my head anymore as I used to do many years ago. So Zig isn't for me. Simply not the target audience.
• gz09 1y ago
  Unfortunately, removing debug symbols in most cases isn't a good/useful option
  • magackame 1y ago
    What "most" cases are you thinking of? Also don't forget that a binary that in release weights 10 MB, when compiled with debug symbols can weight 300 MB, which is way less practical to distribute.
• kenoath69 1y ago
  Where is Cranelift mentioned My 2c on this is nearly ditching rust for game development due to the compile times, in digging it turned out that LLVM is very slow regardless of opt level. Indeed it's what the Jai devs have been saying. So Cranelift might be relevant for OP, I will shill it endlessly, took my game from 16 seconds to 4 seconds. Incredible work Cranelift team.
  • jiehong 1y ago
    I think that’s what zig team is also doing to allow very fast build times: remove LLVM.
    • norman784 1y ago
      Yes, Zig author commented[0] that a while ago [0] https://news.ycombinator.com/item?id=44390972 https://news.ycombinator.com/item?id=44390972
  • norman784 1y ago
    Nice, I checked a while ago and was no support for macOS aarch64, but seems that now it is supported.
  • lll-o-lll 1y ago
    Wait. You were going to ditch rust because of 16 second build times?
    • sarchertech 1y ago
      16 seconds is infuriating for something that needs to be manually tested like does this jump feel too floaty. But it’s also probable that 16 seconds was fairly early in development and it would get much worse from there.
    • metaltyphoon 1y ago
      Over time that adds up when your coding consists of REPL like workflow.
    • 1y ago
• smcleod 1y ago
  I've got to say when I come across an open source project and realise it's in rust I flinch a bit know how incredibly slow the build process is. It's certainly been one of the deterrents to learning it.
• juped 1y ago
  I don't think rustc is that slow. It's usually cargo/the dozens of crates that make it take a long time, even if you've set up a cache and rustc is doing nothing but hitting the cache.
• Devasta 1y ago
  Slow compile times are a feature, get to make a cuppa.
  • zozbot234 1y ago
    > Slow compile times are a feature xkcd is always relevant: https://xkcd.com/303/ https://xkcd.com/303/
    • randomNumber7 1y ago
      On the other hand you get mentally insane if you try to work in a way that you do s.th. usefull during the 5-10 min compile times you often have with C++ projects. When I had to deal with this I would just open the newspaper and read an article in front of my boss.
      • PhilipRoman 1y ago
        Slow compile times really mess with your brain. When I wanted to test two different solutions, I would keep multiple separate clones (each one takes about 80GB, mind you) and do the manual equivalent of branch prediction by compiling both, just in case I needed the other one as well.
• duped 1y ago
  A lot of people are replying to the title instead of the article. > To get your Rust program in a container, the typical approach you might find would be something like: If you have `cargo build --target x86_64-unknown-linux-musl` in your build process you do not need to do this anywhere in your Dockerfile. You should compile and copy into /sbin or something. If you really want to build in a docker image I would suggest using `cargo --target-dir=/target ...` and then run with `docker run --mount type-bind,...` and then copy out of the bind mount into /bin or wherever.
  • edude03 1y ago
    Many docker users develop on arm64-darwin and deploy to x86_64 (g)libc, so I don't think that'll work generally.
    • duped 1y ago
      Those users are wrong :shrug:
  • remram 1y ago
    The author dismissed that option saying "I value that docker build can have a clean environment every time", so this is self-inflicted.
• cratermoon 1y ago
  Some code that can make Rust compilation pathologically slow is complex const expressions. Because the compiler can evaluate a subset of expressions at compile time[1], a complex expression can take an unbounded amount of time to evaluate. The long-running-const-eval will by default abort the compilation if the evaluation takes too long. 1 https://doc.rust-lang.org/reference/const_eval.html https://doc.rust-lang.org/reference/const_eval.html
• ozgrakkurt 1y ago
  Rust compiler is very very fast but language has too many features. The slowness is because everyone has to write code with generics and macros in Java Enterprise style in order to show they are smart with rust. This is really sad to see but most libraries abuse codegen features really hard. You have to write a lot of things manually if you want fast compilation in rust. Compilation speed of code just doesn’t seem to be a priority in general with the community.
  • aquariusDue 1y ago
    Yeah, for application code in my experience the more I stick to the dumb way to do it the less I fight the borrow checker along with fewer trait issues. Refactoring seems to take about the same time too so no loss on that front. After all is said and done I'm just left with various logic bugs to fix which is par for the course (at least for me) and a sense of wondering if I actually did everything properly. I suppose maybe two years from now we'll have people that suggest avoiding generics and tempering macro usage. These days most people have heard the advice about not stressing over cloning and unwraping (though expect is much better imo) on the first pass more or less. Something something shiny tool syndrome?
  • skeezyboy 1y ago
    >Compilation speed of code just doesn’t seem to be a priority in general with the community. they have only one priority, memory safety (from a certain class of memory bugs)
• rednafi 1y ago
  I’m glad that Go went the other way around: compilation speed over optimization. For the kind of work I do — writing servers, networking, and glue code — fast compilation is absolutely paramount. At the same time, I want some type safety, but not the overly obnoxious kind that won’t let me sloppily prototype. Also, the GC helps. So I’ll gladly pay the price. Not having to deal with sigil soup is another plus point. I guess Google’s years of experience led to the conclusion that, for software development to scale, a simple type system, GC, and wicked fast compilation speed are more important than raw runtime throughput and semantic correctness. Given the amount of networking and large - scale infrastructure software written in Go, I think they absolutely nailed it. But of course there are places where GC can’t be tolerated or correctness matters more than development speed. But I don’t work in that arena and am quite happy with the tradeoffs that Go made.
  • galangalalgol 1y ago
    That is exactly what go was meant for and there is nothing better than picking the right tool for the job. The only foot gun I have seen people run into is parallelism with mutable shared state through channels can be subtly and exploitably wrong. I don't feel like most people use channels like that though? I use rust because that isn't the job I have. I usually have to cramb slow algorithms into slower hardware, and the problems are usually almost but not quite embarrassingly parallel.
    • bjackman 1y ago
      I think a lot of the materials that the Go folks put out in the early days encourage a very channel-heavy style of programming that leads to extremely bad places. Nowadays the culture seems to have evolved a bit. I now go into high alert mode if I see a channel cross a function boundary or a goroutine that wasn't created via errgroup or similar. People also seem to have chilled out about the "share by communicating" thing. It's usually better to just use a mutex and I think people recognise that now.
      • rednafi 1y ago
        This is true. I have been writing Go for years and still think channel is a bit too low level. It probably would've benefited from a different layer of abstraction.
• jmyeet 1y ago
  Early design decisions favored run-time over compile-time [1]: > * Borrowing — Rust’s defining feature. Its sophisticated pointer analysis spends compile-time to make run-time safe. > * Monomorphization — Rust translates each generic instantiation into its own machine code, creating code bloat and increasing compile time. > * Stack unwinding — stack unwinding after unrecoverable exceptions traverses the callstack backwards and runs cleanup code. It requires lots of compile-time book-keeping and code generation. > * Build scripts — build scripts allow arbitrary code to be run at compile-time, and pull in their own dependencies that need to be compiled. Their unknown side-effects and unknown inputs and outputs limit assumptions tools can make about them, which e.g. limits caching opportunities. > * Macros — macros require multiple passes to expand, expand to often surprising amounts of hidden code, and impose limitations on partial parsing. Procedural macros have negative impacts similar to build scripts. > * LLVM backend — LLVM produces good machine code, but runs relatively slowly. Relying too much on the LLVM optimizer — Rust is well-known for generating a large quantity of LLVM IR and letting LLVM optimize it away. This is exacerbated by duplication from monomorphization. > * Split compiler/package manager — although it is normal for languages to have a package manager separate from the compiler, in Rust at least this results in both cargo and rustc having imperfect and redundant information about the overall compilation pipeline. As more parts of the pipeline are short-circuited for efficiency, more metadata needs to be transferred between instances of the compiler, mostly through the filesystem, which has overhead. > * Per-compilation-unit code-generation — rustc generates machine code each time it compiles a crate, but it doesn’t need to — with most Rust projects being statically linked, the machine code isn’t needed until the final link step. There may be efficiencies to be achieved by completely separating analysis and code generation. > * Single-threaded compiler — ideally, all CPUs are occupied for the entire compilation. This is not close to true with Rust today. And with the original compiler being single-threaded, the language is not as friendly to parallel compilation as it might be. There are efforts going into parallelizing the compiler, but it may never use all your cores. > * Trait coherence — Rust’s traits have a property called “coherence”, which makes it impossible to define implementations that conflict with each other. Trait coherence imposes restrictions on where code is allowed to live. As such, it is difficult to decompose Rust abstractions into, small, easily-parallelizable compilation units. > * Tests next to code — Rust encourages tests to reside in the same codebase as the code they are testing. With Rust’s compilation model, this requires compiling and linking that code twice, which is expensive, particularly for large crates. [1]: https://www.pingcap.com/blog/rust-compilation-model-calamity/ https://www.pingcap.com/blog/rust-compilation-model-calamity...
• edude03 1y ago
  First time someone I know in real life has made it to the HN front page (hey sharnoff, congrats) anyway - I think this post (accidentally?) conflates two different sources of slowness: 1) Building in docker 2) The compiler being "slow" They mention they could use bind mounts, yet wanting a clean build environment - personally, I think that may be misguided. Rust with incremental builds is actually pretty fast and the time you lose fighting dockers caching would likely be made up in build times - since you'd generally build and deploy way more often than you'd fight the cache (which, you'd delete the cache and build from scratch in that case anyway) So - for developers who build rust containers, I highly recommend either using cache mounts or building outside the container and adding just the binary to the image. 2) The compiler being slow - having experienced ocaml, go and scala for comparisons the rust compiler is slower than go and ocaml, sure, but for non interactive (ie, REPL like) workflows, this tends not to matter in my experience - realistically, using incremental builds in dev mode takes seconds, then once the code is working, you push to CI at which point you can often accept the (worst case?) scenario that it takes 20 minutes to build your container since you're free to go do other things. So while I appreciate the deep research and great explanations, I don't think the rust compiler is actually slow, just slower than what people might be use to coming from typescript or go for example.
• leoh 1y ago
  tl;dr: it’s slow because it finds far more bugs before runtime than literally any other mainstream compiled language
• ac130kz 1y ago
  tldr as always, don't use Musl, if you want performance, compatibility.
  • ac130kz 1y ago
    Some "smart" folks even downvote this advice. Yeah, I've seen articles on musl's horrible performance back in 2017-2018, and apparently it still holds, yet I get a downvote.
    • 9rx 1y ago
      You make it sound like a "downvote" has meaning or something. It doesn't.
• amelius 1y ago
  Meanwhile, other languages have a JIT compiler which compiles code as it runs. This would be great for development even if it turns out to be slower overall.
  • akkad33 1y ago
    Actually JITs can be faster than AOT compilation because they can be optimized for the current architecture they are running in. There were claims Julia, a JIT language can beat C in some benchmarks
    • amelius 1y ago
      In fact, JITs can be faster because they can specialize code, i.e. make optimizations based on live data.
• feelamee 1y ago
  > Vim hangs when you open it you can enable word wrapping as a workaround ( `:set wrap`). Lifehack: it can be hard to navigate in such file with just `h, j, k, l`, but you can use `gh, gj, etc`. With `g` vim will work with visual lines, while without it with just lines splitted with LF/CRLF
  • mmh0000 1y ago
    With a little bit of vimrc magic you can make it transparent: "Make k/j up/down work more naturally by going to the next displayed line vs "going to the next logical line (for when word-wrapping is on): noremap k gk noremap j gj noremap <up> gk noremap <down> gj "Same as above, but for arrow keys in insert mode: inoremap <up> <Esc>gka inoremap <down> <Esc>gja
• TZubiri 1y ago
  >Every time I wanted to make a change, I would: >Build a new statically linked binary (with --target=x86_64-unknown-linux-musl) >Copy it to my server >Restart the website Isn't it a basic C compiler feature that you can compile a file as an Object, and then link the objects into a single executable? Then you only recompile the file you changed. Not sure what I'm missing.
  • pornel 1y ago
    That's how Rust works already. The problem has been created by Docker which destroys all of the state. If this was C, you'd also end up losing all of the object files and rebuilding them every time.
    • TZubiri 1y ago
      Nope, reread the article, docker wasn't part of the problem it's part of the 'solution' according to OP.
• s_ting765 1y ago
  OP could have skipped all this by doing the compilation with cache on the host system and copying the compiled statically linked binary back to the docker image build.
• mellosouls 1y ago
  Related from a couple of weeks ago: https://news.ycombinator.com/item?id=44234080 https://news.ycombinator.com/item?id=44234080 (Rust compiler performance; 287 points, 261 comments)
• jeden 1y ago
  why rust compiler create so BIG executable!
• fschuett 1y ago
  For deploying Rust servers, I use Spin WASM functions[1], so no Docker / Kubernetes is necessary. Not affiliated with them, just saying. I just build the final WASM binary and then the rest is managed by the runtime. Sadly, the compile time is just as bad, but I think in this case the allocator is the biggest culprit, since disabling optimization will degrade run-time performance. The Rust team should maybe look into shipping their own bundled allocator, "native" allocators are highly unpredictable. [^1]: https://www.fermyon.com https://www.fermyon.com
• dminik 1y ago
  One aspect that I find interesting is that Rust projects often seem deceivingly small. First, dependencies don't translate easily to the perceived size. In C++ dependencies on large projects are often vendored (or even not used at all). And so it is easy to look at your ~400000 line codebase and go "it's slow, but there's a lot of code here after all". Second (and a much worse problem) are macros. They actually hit the same issue. A macro that expands to 10s or 100s of lines can very quickly take your 10000 line project and turn it into a million line behemoth. Third are generics. They also suffer the same problem. Every generic instantiation is eating your CPU. But I do want to offer a bit of an excuse for rust. Because these are great features. They turn what would have taken 100000 lines of C or 25000 lines of C++ to a few 1000s lines of Rust. However, there is definitely an overuse here making the ecosystem seem slow. For instance, at work we use async-graphql. The library itself is great, but it's an absolute proc-macro hog. There's issues in the repository open for years about the performance. You can literally feel the compiler getting slower for each type you add.
  • jvanderbot 1y ago
    You can literally feel the compiler getting slower for each type you add. YSK: The compiler performance is IIRC exponential in the "depth" of types. And oh boy does GraphQL love their nested types.
    • deleted 1y ago
      [deleted]
    • tukantje 1y ago
      Unironically why typescript is a perfect fit for graphql
    • subarctic 1y ago
      Actually exponential or quadratic?
      • jvanderbot 1y ago
        The rust devs said exponential for one step. Whether or not that step is required each compilation I don't recall. It was a git issue on rustc from a while back IIRC. Unfortunately I'm not at my computer to do a search for you.
• WalterBright 1y ago
  I suspect that it is because the borrow checker needs to do data flow analysis. DFA is normally done in the optimizer, and we all know that optimized builds are quite a bit slower, and that is due to DFA. DFA in the front end means slow.
  • steveklabnik 1y ago
    As said many times in this thread already, the borrow checker is virtually never a significant amount of time spent while compiling. Also, the article goes into great depth as to what is happening there, and the borrow checker never comes up.
    • WalterBright 1y ago
      That makes me curious as to how Rust does the DFA. What I do is construct the data flow equations as bit vectors, which are then solved iteratively until a solution is converged on. Doing this on the whole program eats a lot of memory.
      • steveklabnik 1y ago
        I'm not an expert on this part of the compiler, but what I can tell you is that Rust uses multiple forms of IR, and the IR that the borrow checker operates on (https://rustc-dev-guide.rust-lang.org/mir/index.html https://rustc-dev-guide.rust-lang.org/mir/index.html) already encodes control flow. So doing that construction, at least, is just a normal part of the compiler, and isn't part of the borrow checker itself. However, in my understanding, it takes that IR, does build a borrow graph, computes liveliness, and then does inference. There's been a number of details in how this has changed over the years, but it's vaguely datalog shaped.
        • WalterBright 1y ago
          Because there are cycles in the flow graph, I do not see how liveness can be computed in general without iteration and convergence on a solution.
• Scuds 1y ago
  I have a mac m4 pro and it's 2 minutes to compile all of Deno, which is my go-to for bigass rust projects. ``` > cargo clean && time cargo build cargo build 713.80s user 91.57s system 706% cpu 1:53.97 total > cargo clean && time cargo build --release cargo build --release 1619.53s user 142.65s system 354% cpu 8:17.05 total ``` this is without incremental compilation. And it's not like you have to babysit a release build if you have a CI/CD system
  • hu3 1y ago
    Interesting, M1 Max took 6 minutes and M1 Air took 11 minutes according to this article: https://corrode.dev/blog/tips-for-faster-rust-compile-times/#invest-in-better-hardware https://corrode.dev/blog/tips-for-faster-rust-compile-times/...
    • Scuds 1y ago
      Oh yes, apple hardware continues to improve and M4 pro still the single threaded champion of anything under 300 w. FWIW - last stage where the binary is produced takes the longest and is single threaded and that's the largest difference between release and debug.
      • hu3 1y ago
        Sounds suspicious to me. At a quick glance, most benchmarks show only 50% to 60% single threaded improvement between M1 and M4. That wouldn't explain compilation times going from 6 minutes to 2 minutes.
• saghm 1y ago
  My take on Rust build times is that the larger issue isn't the speed of compilation itself, but that it's actually pretty hard to avoid compiling a lot of unneeded code from dependencies. To be clear, this is a distinct concern from having too many dependencies (which is it's own issue, but I'd argue that it's a social one that doesn't need to be solved in order to significantly reduce build times, which can be done with a technical solution). I wrote up a lot of my ideas in a blog post mostly to avoid having to repeat my thoughts on this every time it comes up: https://saghm.com/cargo-features-rust-compile-times/ https://saghm.com/cargo-features-rust-compile-times/
• deleted 1y ago
  [deleted]
• hdjdbdirbrbtv 1y ago
  As a professional rust dev, I will say this, I don't notice. Generally because I am doing partial builds mostly. And when I am not the m1 max I have is fast enough to compile the project. Really there are much larger issues that I have to deal with that build time does not affect.