5 ms·

BazelCon 2024 Recap

53 points by kaycebasques 2y ago

dijit 2y ago
What's the option if I want to learn Bazel to an intermediate level and I don't work with former Googlers? There's a decent number of Hello World examples around, and I can get moving with some rulesets, but using it for more than toy projects seems to be really hard for me and there's not a lot of intermediate content from what I can tell. There's "Advanced" reference materials, and countless "getting started" tutorials, but rarely is there anything relating to best practices or even what constitutes good rules. Seems like most of the hobby community prefers NIX for deterministic builds too, so the situation is not improving for novices like myself who really like the idea of hermitic builds, monorepo's and distributed cache.
- shaftway 2y ago
  Most Google engineers don't do anything more complicated than the hello world examples. Probably far less complicated because they don't need to do any repo setup, it all just works out of the box. Most engineers just make simple java build targets. Larger teams would have someone who specializes in tweaking the build and writes Skylark rules, but to other engineers they just look like simple build targets. I'd suggest finding an open source project and playing with it locally. Try to get everything building and running, including unit tests. Make the build as granular as you can. Try to pick a project that has a variety of kinds of things that need to be built, like Java, C++, Protobufs, etc. Then look for patterns in the build targets you wrote and see if you can refactor them to simplify it with custom Skylark rules.
  - shaftway 2y ago
    > but to other engineers they just look like simple build targets. I forgot to mention, this is by design. It is uncommon at Google for a BUILD file to be more than 3 or 4 build targets. Maybe someone will add a for loop to the build so that they can define various flavors of the build easily, but that's it. Anything more complicated and it should be extracted to a separate Skylark file, included, and in the BUILD file it should look like a basic task.
- mbarr 2y ago
  Selenium has a pretty good Bazel setup which uses a lot of different functionality: https://github.com/SeleniumHQ/selenium/ https://github.com/SeleniumHQ/selenium/
- zeroDivisible 2y ago
  You can check some open source projects (envoy is the one I know) which are using it. The bigger projects are fun way to learn some tricks.
- sluongng 2y ago
  > What's the option if I want to learn Bazel to an intermediate level and I don't work with former Googlers? The best way to learn it would be to use it yourself and try solving problems with it. You don't need an xoogler to learn it, my former company did not have one when we first evaluated Bazel. There is a very active community Slack and StackOverflow for questions as well. > rarely is there anything relating to best practices or even what constitutes good rules. I think rules_go has been a stellar example of what a well-made rule should look like: - Managing toolchains and stdlib within Starlark - Provide wrapper around core toolchains to enforce best practices - Break down a build into fine-grained, reusable actions - Provide concrete APIs for downstream rules (i.e. graphql, swagger, proto code gen) to consume. > Seems like most of the hobby community prefers NIX for deterministic builds too, so the situation is not improving for novices like myself who really like the idea of hermitic builds, monorepo's and distributed cache. Nix figured out the external dependencies story much better than Bazel, much thanks to nixpkgs repo. So naturally, in a fragmented opensource ecosystem, Nix thrives much better. However, in a closed ecosystem within an enterprise, Bazel provides a more granular approach to managing your build: smaller actions mean finer, more granular caching. It also has a much better set of APIs for distributed builds and build telemetry that folks could depend on from day 1. It's also copying some for Nix playbook with the new Bazel Central Registry resembling nixpkgs. I think you can learn either build system. Most of the skills and concepts are transferrable: sandboxing, build hermeticity, remote caching, remote build, etc... So you often see some experts in this field being involved in multiple build tools development and not just one.
- aaomidi 2y ago
  Join the Bazel slack imo.
deleted 2y ago
[deleted]
hnrodey 2y ago
Any conference recommendations? I'm tracking 2024 and 2025 software engineering conferences. There are still several marquee conferences before year end if that's your thing. https://www.fullyearcal.com/calendar/iMRr-2l16tI/software-engineering-and-tech-conferences---us-and-canada-2024 https://www.fullyearcal.com/calendar/iMRr-2l16tI/software-en...
steven-xu 2y ago
> Bazel is designed to be optimal and has perfect knowledge of the state of the world, yet it’s too slow for quick operations. This is one of the biggest challenges where Bazel falls short of non-Bazel tooling for us in the Web development/Node.js world. The Node ecosystem has characteristics that push Bazel scaling (node_modules is 100k+ files and 1GB+), and Bazel's insistence on getting builds correct/reproducible is in a way its own enemy. Bazel needs to set up 100s of thousands of file watchers to be correct, but the Node.js ecosystem's "let's just assume that node_modules hasn't changed" is good enough most of the time. For us, many non-Bazel inner devloop steps inflate from <5s to >60s after Bazel overhead, even after significant infra tuning.
- malkia 2y ago
  I always wondered, why `node_modules` growing so big (in terms of file count) was ever ok. Apart from bazel, having that many files sucks anyway.
- aliasxneo 2y ago
  It's not unique to Bazel. Nix also struggles with Node.js. I'm not too fond of either solution, but to me, the problem lies in the Node ecosystem, and it seems unlikely a "language-agnostic" tool will ever be able to crack that nut.
  - GauntletWizard 2y ago
    Node also struggles with Node.js, and that's including its shortcuts that kill reproducibility. A node.js sync from an empty cache is by far the longest part of any build process I maintain. It's still a long step with a full cache and without doing anything.
anothername12 2y ago
We have giant Go monorepo at work. Bazel build. The ex-Google guy who set it up, left. No one around who wants to put in the time to learn it. It seems like a general increase in complexity. It seems to replace the go mod stuff so we have something called gazelle that figured it out and third party ide plug-ins. The plug in for IntelliJ is janky. To top it all off, somehow we managed to get a build that is slow locally, slow on CI.
- Arbortheus 2y ago
  We had the same setup, ex-Googler SRE demanded builds be done with Bazel for our Go monorepo. He convinced someone to switch it all over, all builds, deploys, CI, testing, etc were all dependent on Bazel. That person left, and the Bazel stuff was left unmaintained as no one had the interest nor time to learn it. Later Bazel decide to kill off rules_docker and replace it with rules_oci, which means we can no longer update our Golang version without a super painful migration where we end up breaking production a bunch of times because of quirks in the super difficult migration. Eventually we invested the time to rip the whole thing out and replace it with standard Go tooling with a multistage docker build for the container image. Everything from running tests, using Golang tooling, legibility of builds, CI, and deploys is easier. The best thing we did was remove it and move to standard Golang tooling.
  - fire_lake 2y ago
    I’m guessing the build became much slower due to the lack of caching though?
    - Arbortheus 2y ago
      The builds were much faster afterwards, but Go builds always are fast. Maybe in other languages that are slower to build the Bazel cache is more beneficial.
  - sluongng 2y ago
    It's a humbling experience that the opening talk of this year BazelCon is about research into projects that FAILED to adopt Bazel. You can check out the slides of the talk here https://static.sched.com/hosted_files/bazelcon2024/41/TheClassicsNeverGoOutOfStyle.pdf https://static.sched.com/hosted_files/bazelcon2024/41/TheCla... Personally, most cases I have seen are caused by a systematic under-investment into developer toolings and infrastructure at the company. Management layers often don't understand the software development assembly lines are not composed of just workers(software engineers), but also tools and machines that enable faster workflows. This often results in some critical processes in the pipeline from code to prod to be maintained by 1 guy: monitoring, alerting, deployments, builds, git etc... and when that 1 guy left, the system failed and the company suffered. I think successful Bazel adoption in an org is often a signal that the company has grown in size and values its developer's time and happiness. Failure to adopt often means a lack of investment in dev experience in general.
ninjazee124 2y ago
We tried Bazel for a java monorepo. Not having good IDE support was a deal-breaker because it would complicate things for onboarding and junior devs
- fire_lake 2y ago
  I found maintaining IDE files and a Bazel build side by side to be surprisingly easy. It’s for sure a weak point with Bazel though.
- pianoben 2y ago
  Same experience here; I moved us to Gradle six months ago, and like magic all the whining in Slack about our build system just... went away. Turns out, when your build system doesn't occupy 5-15% of your teams' working days, they get a lot more done!
- mhh__ 2y ago
  Generate a project file as part of the build?
synthos 2y ago
Does anyone have a good summary of the Buck(2) track?
dcsommer 2y ago
I'd love to hear more about the relationship between Buck2 and Bazel such that they share a conference. Sure, they are quite similar systems and both use Starlark DSL, but is there a long term vision of these projects belonging "together" or even merging? Or was Buck2 a one-off guest?
- jmmv 2y ago
  Buck 2 was part of the "Build Meetup", not BazelCon, which is a more-frequent event that has room for more than just Bazel (although it's still pretty focused on Bazel-related topics and similar build systems). More details at https://meetup.build/ https://meetup.build/
sebastos 2y ago
So it's 2024, and the plan is that we're going to wait another few years for BSP to be finished before vscode support is anything more than a toy? If their goal is adoption, that is just _awful_ pathfinding. Probably harkens the beginning of a death spiral for the project, if I'm to make a prediction.
pjmlp 2y ago
Before reading this well written recap, I was convinced that Bazel is a solution to Google problems, now I am fully sure that is indeed the case, and I should only be aware it is something that exists.
- aaomidi 2y ago
  Bazel solves a lot of issues for non Google folk. It has a build and test and coverage cache that actually consistently works. It can run tests in a sandbox to prevent your local environment from getting consumed by the test (this specific thing has actually caused production outages before). A DAG for your codebase is also super useful as you can automate a lot of “clean up once you’re done” type work.
malkia 2y ago
BazelCon2024 playlist here -> https://www.youtube.com/watch?v=-0DRS99dSU4&list=PLbzoR-pLrL6ptKfAQNZ5RS4HMdmeilBcw https://www.youtube.com/watch?v=-0DRS99dSU4&list=PLbzoR-pLrL...

BazelCon 2024 Recap

53 points by kaycebasques 2y ago

What's the option if I want to learn Bazel to an intermediate level and I don't work with former Googlers? There's a decent number of Hello World examples around, and I can get moving with some rulesets, but using it for more than toy projects seems to be really hard for me and there's not a lot of intermediate content from what I can tell. There's "Advanced" reference materials, and countless "getting started" tutorials, but rarely is there anything relating to best practices or even what constitutes good rules. Seems like most of the hobby community prefers NIX for deterministic builds too, so the situation is not improving for novices like myself who really like the idea of hermitic builds, monorepo's and distributed cache.

shaftway 2y ago
Most Google engineers don't do anything more complicated than the hello world examples. Probably far less complicated because they don't need to do any repo setup, it all just works out of the box. Most engineers just make simple java build targets. Larger teams would have someone who specializes in tweaking the build and writes Skylark rules, but to other engineers they just look like simple build targets. I'd suggest finding an open source project and playing with it locally. Try to get everything building and running, including unit tests. Make the build as granular as you can. Try to pick a project that has a variety of kinds of things that need to be built, like Java, C++, Protobufs, etc. Then look for patterns in the build targets you wrote and see if you can refactor them to simplify it with custom Skylark rules.
- shaftway 2y ago
  > but to other engineers they just look like simple build targets. I forgot to mention, this is by design. It is uncommon at Google for a BUILD file to be more than 3 or 4 build targets. Maybe someone will add a for loop to the build so that they can define various flavors of the build easily, but that's it. Anything more complicated and it should be extracted to a separate Skylark file, included, and in the BUILD file it should look like a basic task.
mbarr 2y ago
Selenium has a pretty good Bazel setup which uses a lot of different functionality: https://github.com/SeleniumHQ/selenium/ https://github.com/SeleniumHQ/selenium/
zeroDivisible 2y ago
You can check some open source projects (envoy is the one I know) which are using it. The bigger projects are fun way to learn some tricks.
sluongng 2y ago
> What's the option if I want to learn Bazel to an intermediate level and I don't work with former Googlers? The best way to learn it would be to use it yourself and try solving problems with it. You don't need an xoogler to learn it, my former company did not have one when we first evaluated Bazel. There is a very active community Slack and StackOverflow for questions as well. > rarely is there anything relating to best practices or even what constitutes good rules. I think rules_go has been a stellar example of what a well-made rule should look like: - Managing toolchains and stdlib within Starlark - Provide wrapper around core toolchains to enforce best practices - Break down a build into fine-grained, reusable actions - Provide concrete APIs for downstream rules (i.e. graphql, swagger, proto code gen) to consume. > Seems like most of the hobby community prefers NIX for deterministic builds too, so the situation is not improving for novices like myself who really like the idea of hermitic builds, monorepo's and distributed cache. Nix figured out the external dependencies story much better than Bazel, much thanks to nixpkgs repo. So naturally, in a fragmented opensource ecosystem, Nix thrives much better. However, in a closed ecosystem within an enterprise, Bazel provides a more granular approach to managing your build: smaller actions mean finer, more granular caching. It also has a much better set of APIs for distributed builds and build telemetry that folks could depend on from day 1. It's also copying some for Nix playbook with the new Bazel Central Registry resembling nixpkgs. I think you can learn either build system. Most of the skills and concepts are transferrable: sandboxing, build hermeticity, remote caching, remote build, etc... So you often see some experts in this field being involved in multiple build tools development and not just one.
aaomidi 2y ago
Join the Bazel slack imo.

deleted 2y ago

[deleted]

hnrodey 2y ago

Any conference recommendations? I'm tracking 2024 and 2025 software engineering conferences. There are still several marquee conferences before year end if that's your thing. https://www.fullyearcal.com/calendar/iMRr-2l16tI/software-engineering-and-tech-conferences---us-and-canada-2024 https://www.fullyearcal.com/calendar/iMRr-2l16tI/software-en...

steven-xu 2y ago

> Bazel is designed to be optimal and has perfect knowledge of the state of the world, yet it’s too slow for quick operations. This is one of the biggest challenges where Bazel falls short of non-Bazel tooling for us in the Web development/Node.js world. The Node ecosystem has characteristics that push Bazel scaling (node_modules is 100k+ files and 1GB+), and Bazel's insistence on getting builds correct/reproducible is in a way its own enemy. Bazel needs to set up 100s of thousands of file watchers to be correct, but the Node.js ecosystem's "let's just assume that node_modules hasn't changed" is good enough most of the time. For us, many non-Bazel inner devloop steps inflate from <5s to >60s after Bazel overhead, even after significant infra tuning.

malkia 2y ago
I always wondered, why `node_modules` growing so big (in terms of file count) was ever ok. Apart from bazel, having that many files sucks anyway.
aliasxneo 2y ago
It's not unique to Bazel. Nix also struggles with Node.js. I'm not too fond of either solution, but to me, the problem lies in the Node ecosystem, and it seems unlikely a "language-agnostic" tool will ever be able to crack that nut.
- GauntletWizard 2y ago
  Node also struggles with Node.js, and that's including its shortcuts that kill reproducibility. A node.js sync from an empty cache is by far the longest part of any build process I maintain. It's still a long step with a full cache and without doing anything.

anothername12 2y ago

We have giant Go monorepo at work. Bazel build. The ex-Google guy who set it up, left. No one around who wants to put in the time to learn it. It seems like a general increase in complexity. It seems to replace the go mod stuff so we have something called gazelle that figured it out and third party ide plug-ins. The plug in for IntelliJ is janky. To top it all off, somehow we managed to get a build that is slow locally, slow on CI.

Arbortheus 2y ago
We had the same setup, ex-Googler SRE demanded builds be done with Bazel for our Go monorepo. He convinced someone to switch it all over, all builds, deploys, CI, testing, etc were all dependent on Bazel. That person left, and the Bazel stuff was left unmaintained as no one had the interest nor time to learn it. Later Bazel decide to kill off rules_docker and replace it with rules_oci, which means we can no longer update our Golang version without a super painful migration where we end up breaking production a bunch of times because of quirks in the super difficult migration. Eventually we invested the time to rip the whole thing out and replace it with standard Go tooling with a multistage docker build for the container image. Everything from running tests, using Golang tooling, legibility of builds, CI, and deploys is easier. The best thing we did was remove it and move to standard Golang tooling.
- fire_lake 2y ago
  I’m guessing the build became much slower due to the lack of caching though?
  - Arbortheus 2y ago
    The builds were much faster afterwards, but Go builds always are fast. Maybe in other languages that are slower to build the Bazel cache is more beneficial.
- sluongng 2y ago
  It's a humbling experience that the opening talk of this year BazelCon is about research into projects that FAILED to adopt Bazel. You can check out the slides of the talk here https://static.sched.com/hosted_files/bazelcon2024/41/TheClassicsNeverGoOutOfStyle.pdf https://static.sched.com/hosted_files/bazelcon2024/41/TheCla... Personally, most cases I have seen are caused by a systematic under-investment into developer toolings and infrastructure at the company. Management layers often don't understand the software development assembly lines are not composed of just workers(software engineers), but also tools and machines that enable faster workflows. This often results in some critical processes in the pipeline from code to prod to be maintained by 1 guy: monitoring, alerting, deployments, builds, git etc... and when that 1 guy left, the system failed and the company suffered. I think successful Bazel adoption in an org is often a signal that the company has grown in size and values its developer's time and happiness. Failure to adopt often means a lack of investment in dev experience in general.

ninjazee124 2y ago

We tried Bazel for a java monorepo. Not having good IDE support was a deal-breaker because it would complicate things for onboarding and junior devs

fire_lake 2y ago
I found maintaining IDE files and a Bazel build side by side to be surprisingly easy. It’s for sure a weak point with Bazel though.
pianoben 2y ago
Same experience here; I moved us to Gradle six months ago, and like magic all the whining in Slack about our build system just... went away. Turns out, when your build system doesn't occupy 5-15% of your teams' working days, they get a lot more done!
mhh__ 2y ago
Generate a project file as part of the build?

synthos 2y ago

Does anyone have a good summary of the Buck(2) track?

dcsommer 2y ago

I'd love to hear more about the relationship between Buck2 and Bazel such that they share a conference. Sure, they are quite similar systems and both use Starlark DSL, but is there a long term vision of these projects belonging "together" or even merging? Or was Buck2 a one-off guest?

jmmv 2y ago
Buck 2 was part of the "Build Meetup", not BazelCon, which is a more-frequent event that has room for more than just Bazel (although it's still pretty focused on Bazel-related topics and similar build systems). More details at https://meetup.build/ https://meetup.build/

sebastos 2y ago

So it's 2024, and the plan is that we're going to wait another few years for BSP to be finished before vscode support is anything more than a toy? If their goal is adoption, that is just _awful_ pathfinding. Probably harkens the beginning of a death spiral for the project, if I'm to make a prediction.

pjmlp 2y ago

Before reading this well written recap, I was convinced that Bazel is a solution to Google problems, now I am fully sure that is indeed the case, and I should only be aware it is something that exists.

aaomidi 2y ago
Bazel solves a lot of issues for non Google folk. It has a build and test and coverage cache that actually consistently works. It can run tests in a sandbox to prevent your local environment from getting consumed by the test (this specific thing has actually caused production outages before). A DAG for your codebase is also super useful as you can automate a lot of “clean up once you’re done” type work.

malkia 2y ago

BazelCon2024 playlist here -> https://www.youtube.com/watch?v=-0DRS99dSU4&list=PLbzoR-pLrL6ptKfAQNZ5RS4HMdmeilBcw https://www.youtube.com/watch?v=-0DRS99dSU4&list=PLbzoR-pLrL...