6 ms·

Using Bazel to build a new Rust project

17 points by fahhem 2y ago

ramon156 2y ago
Out of curiosity, why do people switch to bazel? I don't have much experience with it, so maybe I'm missing out?
- rwmj 2y ago
  It has caused nothing but problems wherever I've seen it being used.
  - ithkuil 2y ago
    Yep. The only way to reap the benefits with bazel is to have somebody who continuously deals with the problems. This is not that different from many other aspects of software engineering, but as on those other areas you need to run a cost-benefit analysis
    - carlmr 2y ago
      We've had great success in using Bazel for a large mixed C++/Python and some other languages monorepo. The initial setup cost is somewhat high, but the caching aspects and configurability make it a real game changer in this space. If you're building a Rust-only project it's probably better to stick with Cargo, since Cargo caches really well and doesn't have such a large setup cost.
      - ithkuil 2y ago
        Cargo caches well locally, but I had problems getting good performance on CI with cache save/restore. OTOH bazel remote caching works quite well (didn't try with rust yet though)
        carlmr 2y ago
        >OTOH bazel remote caching works quite well (didn't try with rust yet though) I don't think there's a reason it wouldn't work for that.
- chipdart 2y ago
  > Out of curiosity, why do people switch to bazel? From what I've read, even for C++ projects Bazel is pushed by FANG-driven cargo cult mixed with FOMO. Even for its internal use, Bazel sounds an awful lot like a promotion-oriented project.
- mdhb 2y ago
  Large cross language mono repos is what it was built for and for that particular use case I don’t think there is anything better. Everything outside of that use case however it probably isn’t the right choice currently but their stated goal is to eventually become the obvious choice in a lot of other scenarios too.
- lesuorac 2y ago
  I mean if you're in the situation of the author where you already have a bazel project why add Cargo? There's a lot of nice things about bazel like being able to embed optimized JS into an iOS app built from transpiled Java code. But like mdhb points out, if you're not doing a lot of stuff with multiple languages you are probably better off with just that language's recommended build tool.
- benreesman 2y ago
  There are a few reasons why people use Blaze-inspired build systems (in addition to Bazel there are Buck and Buck2, Pants, Please, and others). Such build systems are sometimes called “action-cached”, which refers to a design in which the build is expressed in a way to maximize opportunities for caching based on the inputs to an action rather than the mtime like Make. One big reason is speed: aggressively cached builds are generally much faster. This is very similar to how Turbo gets used in the JS/TS ecosystem, but for many more programming languages. Bazel in particular is very well-supported for transparent remote caching, so your build will often get faster rather than slower as the number of contributors goes up. Another common reason is that in the presence of more than one or two programming languages, and particularly if one language is furnishing libraries to another, having a unified build rapidly makes it much more manageable to learn one build tool well rather than learn multiple build tools and some orchestration glue on top. A common example here would be the C/C++ extensions that are all but ubiquitous in high-performance numeric and scientific Python applications. A third example is that C/C++ don’t really have a package manager in the sense that Rust has Cargo. CMake is probably the most popular these days, but it has its own problems and doesn’t interoperate well with other languages. People seem to like making fun of Bazel et al, and I suppose that’s understandable: it’s difficult to learn and not worth learning until you have the problems it addresses. Add in the default animus for all things FAANG and it’s kind of an easy point to score. But this tendency is what Andre Alexandrescu calls “aggressively intermediate”, when senior people criticize Bazel they do so with valid concrete arguments (and such exist), it’s never “FAANG cargo cult” FUD. Turbo is popular in the front end for a reason, an imprecise but still useful analogy is that Bazel is “Turbo for backend”.
- RandomThoughts3 2y ago
  To have the same build system for many languages with support for distributed artifacts caching and strong guarantees about build reproducibility if used correctly. But yes, it’s like a lot of things whose need is born of scale. If you really needed it, it would be obvious to you and if you use it without needing it, it’s going to be painful and difficult to understand.
vilunov 2y ago
Is it possible to specify locked glibc and cc/linking toolchain in your Bazel project?
- fahhem 2y ago
  In my previous position, we accomplished this using a "dev docker" (which I covered generally in a previous post: https://blog.fahhem.com/2023/12/dev-onboarding-then-now/ https://blog.fahhem.com/2023/12/dev-onboarding-then-now/) that's based around a specific version of Ubuntu (20.04) so the glibc is tied to that. In another project, I used a hermetic C++ toolchain (you can use gcc with https://github.com/f0rmiga/gcc-toolchain/ https://github.com/f0rmiga/gcc-toolchain/ or clang/llvm with https://github.com/bazel-contrib/toolchains_llvm https://github.com/bazel-contrib/toolchains_llvm) that pulled in its own (locked) version of glibc and lld, etc.

17 points by fahhem 2y ago

ramon156 2y ago

Out of curiosity, why do people switch to bazel? I don't have much experience with it, so maybe I'm missing out?

rwmj 2y ago
It has caused nothing but problems wherever I've seen it being used.
- ithkuil 2y ago
  Yep. The only way to reap the benefits with bazel is to have somebody who continuously deals with the problems. This is not that different from many other aspects of software engineering, but as on those other areas you need to run a cost-benefit analysis
  - carlmr 2y ago
    We've had great success in using Bazel for a large mixed C++/Python and some other languages monorepo. The initial setup cost is somewhat high, but the caching aspects and configurability make it a real game changer in this space. If you're building a Rust-only project it's probably better to stick with Cargo, since Cargo caches really well and doesn't have such a large setup cost.
    - ithkuil 2y ago
      Cargo caches well locally, but I had problems getting good performance on CI with cache save/restore. OTOH bazel remote caching works quite well (didn't try with rust yet though)
      - carlmr 2y ago
        >OTOH bazel remote caching works quite well (didn't try with rust yet though) I don't think there's a reason it wouldn't work for that.
chipdart 2y ago
> Out of curiosity, why do people switch to bazel? From what I've read, even for C++ projects Bazel is pushed by FANG-driven cargo cult mixed with FOMO. Even for its internal use, Bazel sounds an awful lot like a promotion-oriented project.
mdhb 2y ago
Large cross language mono repos is what it was built for and for that particular use case I don’t think there is anything better. Everything outside of that use case however it probably isn’t the right choice currently but their stated goal is to eventually become the obvious choice in a lot of other scenarios too.
lesuorac 2y ago
I mean if you're in the situation of the author where you already have a bazel project why add Cargo? There's a lot of nice things about bazel like being able to embed optimized JS into an iOS app built from transpiled Java code. But like mdhb points out, if you're not doing a lot of stuff with multiple languages you are probably better off with just that language's recommended build tool.
benreesman 2y ago
There are a few reasons why people use Blaze-inspired build systems (in addition to Bazel there are Buck and Buck2, Pants, Please, and others). Such build systems are sometimes called “action-cached”, which refers to a design in which the build is expressed in a way to maximize opportunities for caching based on the inputs to an action rather than the mtime like Make. One big reason is speed: aggressively cached builds are generally much faster. This is very similar to how Turbo gets used in the JS/TS ecosystem, but for many more programming languages. Bazel in particular is very well-supported for transparent remote caching, so your build will often get faster rather than slower as the number of contributors goes up. Another common reason is that in the presence of more than one or two programming languages, and particularly if one language is furnishing libraries to another, having a unified build rapidly makes it much more manageable to learn one build tool well rather than learn multiple build tools and some orchestration glue on top. A common example here would be the C/C++ extensions that are all but ubiquitous in high-performance numeric and scientific Python applications. A third example is that C/C++ don’t really have a package manager in the sense that Rust has Cargo. CMake is probably the most popular these days, but it has its own problems and doesn’t interoperate well with other languages. People seem to like making fun of Bazel et al, and I suppose that’s understandable: it’s difficult to learn and not worth learning until you have the problems it addresses. Add in the default animus for all things FAANG and it’s kind of an easy point to score. But this tendency is what Andre Alexandrescu calls “aggressively intermediate”, when senior people criticize Bazel they do so with valid concrete arguments (and such exist), it’s never “FAANG cargo cult” FUD. Turbo is popular in the front end for a reason, an imprecise but still useful analogy is that Bazel is “Turbo for backend”.
RandomThoughts3 2y ago
To have the same build system for many languages with support for distributed artifacts caching and strong guarantees about build reproducibility if used correctly. But yes, it’s like a lot of things whose need is born of scale. If you really needed it, it would be obvious to you and if you use it without needing it, it’s going to be painful and difficult to understand.

vilunov 2y ago

Is it possible to specify locked glibc and cc/linking toolchain in your Bazel project?

fahhem 2y ago
In my previous position, we accomplished this using a "dev docker" (which I covered generally in a previous post: https://blog.fahhem.com/2023/12/dev-onboarding-then-now/ https://blog.fahhem.com/2023/12/dev-onboarding-then-now/) that's based around a specific version of Ubuntu (20.04) so the glibc is tied to that. In another project, I used a hermetic C++ toolchain (you can use gcc with https://github.com/f0rmiga/gcc-toolchain/ https://github.com/f0rmiga/gcc-toolchain/ or clang/llvm with https://github.com/bazel-contrib/toolchains_llvm https://github.com/bazel-contrib/toolchains_llvm) that pulled in its own (locked) version of glibc and lld, etc.