6 ms·
Bazel does a lot of non-obvious things that other build tools don't, that become important in larger codebases (100-10,000 active developers): - Automatic cach
by lihaoyi 2y ago
Bazel does a lot of non-obvious things that other build tools don't, that become important in larger codebases (100-10,000 active developers):
- Automatic caching of everything.
In most build tools, caching is opt in, so the core build/compile steps usually end up getting cached but everything else is not and ends up being wastefully recomputed all the time. In Bazel, caching is the default so everything is cached. Even tests are cached so if you run a test twice on the same code and inputs (transitively), the second time it is skipped
- Dependency based test selection: You can use bazel query to determine the possible targets and tests affected by a code change, allowing you to trivially set up CI to only run tests downstream of a PR diff and skip unrelated ones. Any large codebase that doesn't use Bazel ends up re-inventing this manually, e.g. consider this code in apache/spark that re-implements this in a Python script that wraps `mvn` or `sbt` (which are build tools that do not provide this functionality) https://github.com/apache/spark/blob/290b4b31bae2e02b648d2c5ef61183f337b18f8f/dev/sparktestsupport/modules.py#L108-L126 https://github.com/apache/spark/blob/290b4b31bae2e02b648d2c5...
- Automatic sandboxing of your build steps in CGroup/NS containers, to ensure your build steps do not make use of un-declared files. In most build tools, this kind of mistake results in confusing nondeterministic parallelism and cache invalidation problems down the road, where e.g. your build step may rely on a file on disk but not realize it needs to re-compute when the file changes. In Bazel, these mis-configurations result in a deterministic error up front
- At my last job we extended these cgroups to limit CPU/Memory usage as well, which eliminates the noisy neighbour problem and ensures a build step or test gets the same compute footprint whether run alone during development or 96x parallel on a CI worker (https://github.com/bazelbuild/bazel/pull/21322 https://github.com/bazelbuild/bazel/pull/21322). Otherwise it's common for tests to pass when run during manual development then timeout or OOM when run in CI under resource pressure due to other tests hogging the CPU or RAM
- Built in support for seamless shared caches: e.g. I compile something on my laptop, you download it to your laptop for usage. This also applies to tests: e.g. if TestFoo was run in CI on master, if i pull master and run all tests without changing the Foo code, TestFoo is skipped and uses the CI result
- Built in support for shared compute clusters: e.g. I compile stuff and it automatically happens in the cloud on 96 core machines, or i run a lot of tests (e.g. after a big refactor) on my laptop and it automatically gets farmed out to run 1024x parallel on the cluster (which despite running faster shouldn't cost any more than running 1x parallel, since it costs 1024x as much per-second but should finish in 1024x fewer seconds!)
- Support for deep integration between multiple languages: e.g. building a Go binary and Rust library which are both used in a Python executable, which is then tested using a Bash script and deployed as part of a Java backend server. Most build tools support one language well and others not at all, Bazel supports them all (not perfect but adequately)
If you never hit these needs, you don't need Bazel, and you probably shouldn't use it because it is ferociously complicated. But if you do hit these needs, most other build tools simply do not cut it at all.
We're trying to support some of these use cases and provide a simpler alternative in https://mill-build.org https://mill-build.org, but Bazel really is a high bar to reach in terms of features that support large monorepos
- j1elo 2y agoWoah!! You say "If you never hit these needs, you don't need Bazel", but even for a small-ish project, I would like to have Automatic caching of everything, Dependency based test selection, Automatic sandboxing of build steps, or seamless shared caches! I guess for a small team (say less than 10 people) the complexity of Bazel is too much to make it worth it, but those features you mention seem like universally useful for any project. I bet there are lots and lots of reinvented wheels out there trying to do just a subset of all these.
- lihaoyi 2y agoEmpirically speaking, unless you are already a Bazel expert or are lucky enough to hire one, most orgs I've seen need at least 3-5 full-time engineers to become experts in Bazel and about ~1 person decade of engineering to roll it out (i.e. it takes those 3-5 guys 2-3 years). Adopting Bazel is not something you do over a weekend hackathon; the cost starts becoming reasonable once you have 100-200 engineers who can benefit from the 3-5 guys maintaining their build tool For your small-ish project I'm guessing you have wants, but if the project grows at some point those will transition into needs and then you may have to make hard choices. That's my experience rolling it out Bazel myself and maintaining it for a growing organization and codebase over 7 years I'm hoping the Mill build tool I'm working on can become an easier/friendlier alternative that provides many of the same benefits, even if it can't support the extreme Google scale that Bazel does. So if you think these things sound cool, but you don't want to spend 1 person decade on your build tool, you should check out Mill!
- j1elo 2y agoThat sounds awesome, great work trying to simplify the access to those features! Also thanks for the insights
- dilyevsky 2y agoYou make this sound considerably more involved than it actually is from my experience observing/supporting Bazel infra at three companies ranging from 10 to 1K engineers. Adopting Bazel for C, C++, Java and Go has been pretty straight-forward for years now unless you want power features like remote execution, custom rules/macros, etc