4 ms·
Hermeticity is precisely what allows you to avoid endlessly downloading and building the same dependencies. Without hermeticity you can't rely on caching. I fe
by IshKebab 4y ago
Hermeticity is precisely what allows you to avoid endlessly downloading and building the same dependencies. Without hermeticity you can't rely on caching.
I feel like 90% of the computer industry is ignoring the lessons of Bazel and is probably going to wake up in 10 years and go "ooooooh, that's how we should have been doing it".
- no_wizard 4y agofor the frontend space, NX gives you bazel caching / features. It just doesn't cache dependencies, but in my experience with GitHub Actions running `pnpm install` or `yarn install` is not the slowest operation, its running the tools after.
- throwaway894345 4y agoI think everyone agrees that the Bazel/Nix approach is correct, the problem is that Bazel/Nix/etc are insanely hard to use. For example, I spent a good chunk of last weekend trying to get Bazel to build a multiarch Go image, and I couldn't figure it out. Someone needs to figure out how to polish Bazel/Nix so they're viable for organizations that can't invest in a team to operate and provide guidance on Bazel/Nix/etc.
- pnut 4y agoCheck out https://www.pantsbuild.org/ https://www.pantsbuild.org/
- throwaway894345 4y agoI’ve used Pants professionally and that was possibly the worst of the three in my experience. Support across build tools varies by language, but I didn’t get the impression that Pants was head-and-shoulders above other tools for any language ecosystem.
- IshKebab 4y agoIs that Pants 1 or Pants 2? I'm curious about your opinion of the others too.
- throwaway894345 4y agoPants 2 IIRC. It has been several years now, so I forget the particulars.
- jiayo 4y agoCan you elaborate on some of the lessons of Bazel? I've only just heard of it recently, and while I'm intrigued, my impression is this is similar to Facebook writing their own source control: different problems at massive scale. Can a SMB (~50 engineers) benefit from adopting Bazel?
- hobofan 4y ago> Can a SMB (~50 engineers) benefit from adopting Bazel? We are ~8 engineers, and yes, definitely. However there should be good buy-in across the team (as it can be quite invasive), and depending on your choice of languages/tooling the difficulty of adoption may greatly vary. I was the one introducing Bazel to the company and across the ~80 weeks at the company I spent maybe ~4 weeks on setting up and maintaining Bazel. I don't know about your current setup and challenges you have with your CI system. However, compared to the generic type of build system I've seen at companies of that size, I would estimate that with 50 engineers having a single build systems/developer tooling engineer focused on setting up and maintaining Bazel should easily have a positive ROI (through increased development velocity and less time wasted on hunting CI bugs alone).
- lxe 4y agoIf you're doing golang in a large monorepo, in a company of 1000+ engineers, then maybe. If you're a a mobile dev in a similar sized company, then also maybe. If you have devops resources and SRE's and dedicated personnel that understand bazel, then maybe. Personally I wouldn't touch it with a 10 foot pole. It's an opinionated task runner, with terrible docs, that if you don't configure correctly, will just hurt your dev process.
- IshKebab 4y agoYes absolutely (depending on what you do exactly). The core idea behind Bazel is to make build steps truly hermetic, so you know exactly what inputs they are using. This means you can rely on caching, incremental builds, distributed builds and so on. I'm sure if you've had any experience of Make or similar systems you've encountered "a clean build fixed it". That root cause of that is because somewhere there's a mistake in your build system where you forgot to declare a dependency on something, and it just happens to work most of the time, but maybe one time the dependency changed and Make didn't know it had to rebuild some stuff so the build breaks. That's basically why almost everyone's CI system builds everything from scratch every time. Nobody trusts incremental builds. Bazel goes to great lengths to make it so that you have to declare dependencies, otherwise you simply can't access them. That includes: * Cleaning environment variables for build steps * Running build steps in sandboxes * Storing intermediate artefacts in random directories * Including tools themselves (e.g. compilers) as part of the dependency tree Honestly it's not a perfectly hermetic environment, e.g. your build steps can still read the current time, RNGs, etc. so you can still have indeterminacy in your build, but it goes a lot further than anything else. So ultimately the upside is that you can do things like have CI only build and test things that possibly could have been affected by a change. Fix a typo in a README? It won't have to build or test anything. My current company spend 300 compute hours and 2-6 wall hours on every CI run, even for fixing doc typos. Bazel can prevent that. There are downsides though - Bazel was the first system to do this so it has rough edges. And all that sandboxing means there is extra effort to make debugging and IDE integration work. Also because it is super conservative about rebuilding things it can do it sometimes even when it doesn't need to. So I probably wouldn't use it on really small projects, like ones where CI time is under 10-20 minutes anyway. There are a load of newer build systems that use the same idea: Pants, Buck, Pants 2, Please.build, etc. But obviously they don't have the momentum of Bazel.
- lxe 4y agoI guess my issue is not hermeticity but ephemeral constrainers and stateless approach to things. Maybe a "one-way" hermeticity... a semi-permeability if you will.