5 ms·

Can you elaborate on some of the lessons of Bazel? I've only just heard of it recently, and while I'm intrigued, my impression is this is similar to Facebook wr

by jiayo 4y ago

Can you elaborate on some of the lessons of Bazel? I've only just heard of it recently, and while I'm intrigued, my impression is this is similar to Facebook writing their own source control: different problems at massive scale. Can a SMB (~50 engineers) benefit from adopting Bazel?

hobofan 4y ago
> Can a SMB (~50 engineers) benefit from adopting Bazel? We are ~8 engineers, and yes, definitely. However there should be good buy-in across the team (as it can be quite invasive), and depending on your choice of languages/tooling the difficulty of adoption may greatly vary. I was the one introducing Bazel to the company and across the ~80 weeks at the company I spent maybe ~4 weeks on setting up and maintaining Bazel. I don't know about your current setup and challenges you have with your CI system. However, compared to the generic type of build system I've seen at companies of that size, I would estimate that with 50 engineers having a single build systems/developer tooling engineer focused on setting up and maintaining Bazel should easily have a positive ROI (through increased development velocity and less time wasted on hunting CI bugs alone).
lxe 4y ago
If you're doing golang in a large monorepo, in a company of 1000+ engineers, then maybe. If you're a a mobile dev in a similar sized company, then also maybe. If you have devops resources and SRE's and dedicated personnel that understand bazel, then maybe. Personally I wouldn't touch it with a 10 foot pole. It's an opinionated task runner, with terrible docs, that if you don't configure correctly, will just hurt your dev process.
IshKebab 4y ago
Yes absolutely (depending on what you do exactly). The core idea behind Bazel is to make build steps truly hermetic, so you know exactly what inputs they are using. This means you can rely on caching, incremental builds, distributed builds and so on. I'm sure if you've had any experience of Make or similar systems you've encountered "a clean build fixed it". That root cause of that is because somewhere there's a mistake in your build system where you forgot to declare a dependency on something, and it just happens to work most of the time, but maybe one time the dependency changed and Make didn't know it had to rebuild some stuff so the build breaks. That's basically why almost everyone's CI system builds everything from scratch every time. Nobody trusts incremental builds. Bazel goes to great lengths to make it so that you have to declare dependencies, otherwise you simply can't access them. That includes: * Cleaning environment variables for build steps * Running build steps in sandboxes * Storing intermediate artefacts in random directories * Including tools themselves (e.g. compilers) as part of the dependency tree Honestly it's not a perfectly hermetic environment, e.g. your build steps can still read the current time, RNGs, etc. so you can still have indeterminacy in your build, but it goes a lot further than anything else. So ultimately the upside is that you can do things like have CI only build and test things that possibly could have been affected by a change. Fix a typo in a README? It won't have to build or test anything. My current company spend 300 compute hours and 2-6 wall hours on every CI run, even for fixing doc typos. Bazel can prevent that. There are downsides though - Bazel was the first system to do this so it has rough edges. And all that sandboxing means there is extra effort to make debugging and IDE integration work. Also because it is super conservative about rebuilding things it can do it sometimes even when it doesn't need to. So I probably wouldn't use it on really small projects, like ones where CI time is under 10-20 minutes anyway. There are a load of newer build systems that use the same idea: Pants, Buck, Pants 2, Please.build, etc. But obviously they don't have the momentum of Bazel.

by jiayo 4y ago

hobofan 4y ago

> Can a SMB (~50 engineers) benefit from adopting Bazel? We are ~8 engineers, and yes, definitely. However there should be good buy-in across the team (as it can be quite invasive), and depending on your choice of languages/tooling the difficulty of adoption may greatly vary. I was the one introducing Bazel to the company and across the ~80 weeks at the company I spent maybe ~4 weeks on setting up and maintaining Bazel. I don't know about your current setup and challenges you have with your CI system. However, compared to the generic type of build system I've seen at companies of that size, I would estimate that with 50 engineers having a single build systems/developer tooling engineer focused on setting up and maintaining Bazel should easily have a positive ROI (through increased development velocity and less time wasted on hunting CI bugs alone).

lxe 4y ago

If you're doing golang in a large monorepo, in a company of 1000+ engineers, then maybe. If you're a a mobile dev in a similar sized company, then also maybe. If you have devops resources and SRE's and dedicated personnel that understand bazel, then maybe. Personally I wouldn't touch it with a 10 foot pole. It's an opinionated task runner, with terrible docs, that if you don't configure correctly, will just hurt your dev process.

IshKebab 4y ago

Yes absolutely (depending on what you do exactly). The core idea behind Bazel is to make build steps truly hermetic, so you know exactly what inputs they are using. This means you can rely on caching, incremental builds, distributed builds and so on. I'm sure if you've had any experience of Make or similar systems you've encountered "a clean build fixed it". That root cause of that is because somewhere there's a mistake in your build system where you forgot to declare a dependency on something, and it just happens to work most of the time, but maybe one time the dependency changed and Make didn't know it had to rebuild some stuff so the build breaks. That's basically why almost everyone's CI system builds everything from scratch every time. Nobody trusts incremental builds. Bazel goes to great lengths to make it so that you have to declare dependencies, otherwise you simply can't access them. That includes: * Cleaning environment variables for build steps * Running build steps in sandboxes * Storing intermediate artefacts in random directories * Including tools themselves (e.g. compilers) as part of the dependency tree Honestly it's not a perfectly hermetic environment, e.g. your build steps can still read the current time, RNGs, etc. so you can still have indeterminacy in your build, but it goes a lot further than anything else. So ultimately the upside is that you can do things like have CI only build and test things that possibly could have been affected by a change. Fix a typo in a README? It won't have to build or test anything. My current company spend 300 compute hours and 2-6 wall hours on every CI run, even for fixing doc typos. Bazel can prevent that. There are downsides though - Bazel was the first system to do this so it has rough edges. And all that sandboxing means there is extra effort to make debugging and IDE integration work. Also because it is super conservative about rebuilding things it can do it sometimes even when it doesn't need to. So I probably wouldn't use it on really small projects, like ones where CI time is under 10-20 minutes anyway. There are a load of newer build systems that use the same idea: Pants, Buck, Pants 2, Please.build, etc. But obviously they don't have the momentum of Bazel.