6 ms·

One of the issue why compile times are so awful is that all dependencies must be compiled for each project. 20 different projects use the same dependency? They

• kibwen 1y ago
  > This is an effect of the language not having a proper ABI for compiling libraries as dynamically loadable modules No, this is a design decision of Cargo to default to using project-local cached artifacts rather than caching them at the user or system level. You can configure Cargo to do so if you'd like. The reason it doesn't do this by default is because Cargo gives crates great latitude to configure themselves via compile-time flags, and any difference in flags means you get a different compiled artifact anyway. On top of that, there's the question of what `cargo clean` should do when you have a global cache rather than a local one.
  • CrendKing 1y ago
    Why can't Cargo have a system like PyPI where library author uploads compiled binary (even with their specific flags) for each rust version/platform combination, and if said binary is missing for certain combination, fallback to local compile? Imagine `cargo publish` handle the compile+upload task, and crates.io be changed to also host binaries.
    • cesarb 1y ago
      > Why can't Cargo have a system like PyPI where library author uploads compiled binary Unless you have perfect reproducible builds, this is a security nightmare. Source code can be reviewed (and there are even projects to share databases of already reviewed Rust crates; IIRC, both Mozilla and Google have public repositories with their lists), but it's much harder to review a binary, unless you can reproducibly recreate it from the corresponding source code.
      • woodruffw 1y ago
        I don’t think it’s that much of a security nightmare: the basic trust assumption that people make about the packaging ecosystem (that they trust their upstreams) remains the same whether they pull source or binaries. I think the bigger issues are probably stability and size: no stable ABI combined with Rust’s current release cadence means that every package would essentially need to be rebuilt every six weeks. That’s a lot of churn and a lot of extra index space.
        • aloha2436 1y ago
          > remains the same whether they pull source or binaries. I don't think that's exactly true, it's definitely _easier_ to sneak something into a binary without people noticing than it is to sneak it into rust source, but there hasn't been an underhanded rust competition for a while so I guess it's hard to be objective about that.
          • littlestymaar 1y ago
            Pretty much nobody does those two things at the same time: - pulling dependencies with cargo - auditing the source code of the dependencies they're building You are either censoring and vetting everything or you're using dependencies from crates.io (ideally after you've done your due diligence on the crate), but should crates.io be compromised and inject malware in the crates' payload, I'm ready to bet nobody would notice for a long time. I fully agree with GP that binary vs source code wouldn't change anything in practice.
            • ninkendo 1y ago
              > Pretty much nobody does those two things at the same time: - pulling dependencies with cargo - auditing the source code of the dependencies they're building Your “pretty much” is probably weaseling you out of any criticism here, but I fully disagree: My IDE (rustrover) has “follow symbol” support, like every other IDE out there, and I regularly drill into code I’m calling in external crates. Like, just as often as my own code. I can’t imagine any other way of working: it’s important to read code you’re calling to understand it, regardless of whether it’s code made by someone else in the company, or someone else in the world. My IDE’s search function shows all code from all crates in my dependencies. With everything equal regardless of whether it’s in my repo or not. It just subtly shades the external dependencies a slightly different color. I regularly look at a trait I need from another crate, and find implementations across my workspace and dependencies, including other crates and impls within the defining crate. Yes, this info is available on docs.rs but it’s 1000x easier to stay within my IDE, and the code itself is available right there inline, which is way more valuable than docs alone. I think it’s insane to not read code you depend on. Does this mean I’m “vetting” all the code I depend on? Of course not. But I’m regularly reading large chunks of it. And I suspect a large chunk of people work the way I do; There are a lot of eyeballs on public crates due to them being distributed as source, and this absolutely has a tangible impact on supply chain attacks.
    • epage 1y ago
      It runs counter to Cargos curreat model where the top-level workspace has complete control over compilation, including dependencies and compiler flags. I've been floating an idea of "opaque dependencies" that are like python depending on C libraries or a C++ library dependening on a dynamic library.
    • littlestymaar 1y ago
      That would work for debug builds (and that's something that I would appreciate) but not for release, as most of the time you want to compile for the exact CPU you're targeting not just for say “x86 Linux” to make sure your code is optimized properly using SIMD instructions.
    • surajrmal 1y ago
      A trustworthy distributed cache would also work very well for this in practice. Cargo works with sccache. Using bazel + rbe can work even better.
  • mcherm 1y ago
    > You can configure Cargo to do so if you'd like How does one do this?
• goodpoint 1y ago
  That's solved with sccache but even with that compilation time is still garbage
• eddd-ddde 1y ago
  Dependencies must compile with the right features enabled. You can't possibly share the 2^n versions of every binary. ABI stability doesn't fix this.
• surajrmal 1y ago
  If you use bazel to compile rust, it doesn't suffer from this problem. In fact you can get distributed caching as well.