7 ms·
What's New in Bazel 6.0
- nisa 4y agoI'm still not sure if I should go the bazel route... recently found a long good explanation why anki switched away from bazel: https://github.com/ankitects/anki/commit/5e0a761b875fff4c9e4b202c08bd740c7bb37763 https://github.com/ankitects/anki/commit/5e0a761b875fff4c9e4... I'd love to have cachable builds but I wonder how much effort it takes to maintain such a setup - especially in a small team very far away from silicon valley or google were nobody saw bazel before. Would be a perfect fit for my former company but even there I couldn't convince my colleagues - they build a monster using maven which would be more elegant in bazel but maven integration worked different than the maven algorithm at least 2 years ago...so my PoC turned into reading source code of dependency resolution algorithm in the used scala tool coursier... in the end I had to give up after patching maven issues in several projects we build upon - something nobody is willing to do I guess - to be fair I tried to incorporate Alfresco ECM which is quite an enterprise Java monster on it's own. Also I couldn't find a good approach to use multiple git repositories almost all tutorials use a monorepo.
- actionfromafar 4y agoI've been meaning to try out XMake [0]. It has caching and distributed build and seems much easier to understand than Bazel. 0: https://xmake.io/#/getting_started https://xmake.io/#/getting_started
- ranting-moth 4y agoIn my experience you need a Bazel expert on the team. When everything works it's amazing. When it doesn't or additional work is required the hope your local expert is online and willing to sort it out.
- maccard 4y agoThis is true for any language ecosystem or build tool, whether it's bazel, maven, unrealbuildtool, Jenkins, cmake, gradle etc.
- oblio 4y agoThe thing is, if you hire 20 Java devs, at least one of them will already be an advanced Maven user. Same for Python and poetry/pipenv, Javascript/npm, etc. Bazel is kind of to the side everywhere.
- Groxx 4y agoJust speaking to multiple git repositories: could you use submodules for a conglomerated-monorepo? Otherwise: you can pull in repos as bazel-file-containing additions. I forget the details, but I believe it's something like https://bazel.build/extending/repo https://bazel.build/extending/repo . The troubleshooting-while-developing experience is not great though imo, I'd try submodules first.
- nisa 4y agoThanks - repository_rule looks good. So it's either git submodules to maintain updates or checksums / commits in repository_rule - I probably need to just play around with both approaches and choose the one that annoys me the least :)
- eklitzke 4y agoI'm not going to try to convince you to use Bazel. I think it's great, but ultimately choosing one build system or another is a big decision that involves weighing a lot of different factors, including what your coworkers are amenable to and have experience with. However I don't think Bazel is that difficult to get started with. The Bazel docs are admittedly a bit challenging, but the easiest way to learn things when you're first getting started is to just look at other projects using Bazel and see how they do things. Using multiple git repositories is easy, you just use the git_repository() rule which has been in Bazel since pretty much forever. Nowadays you can also use bzlmod (introduced in bazel 5.x) which is a slightly different approach to this problem. There are a bunch of rules in https://github.com/bazelbuild/bazel-central-registry https://github.com/bazelbuild/bazel-central-registry which you can look at to see how to write your own bzlmod registries.
- nisa 4y agobzlmod looks amazing! Thank you! I don't need to be convinced but I need to sell it to my team and I must be able to solve our problems in it - imho getting it working solves 90% of annoying problems in this whole ci/cd/container mess and that's why I'd like to push for it.
- hobofan 4y agoThe BCR also has a web UI at https://registry.bazel.build https://registry.bazel.build
- aseipp 4y agoWas your project built in multiple languages and tied together using Maven? Or was it just JDK languages/Java? Tools like Bazel or Nix or <whatever you enjoy> are incredible for many reasons but they are, to a first approximation, very, very high-effort and complex tools that require care. It's like a racing car. You need an engineer on hand to keep the car running, and someone to drive it too. Maybe you're both of those people, but only a driver isn't enough. They also suffer from another problem which is that most people don't care as much as you or I do. :) So to make it a slam dunk, you have to make a pretty clear case that it's not "just" 2x better, but actually 10x better. And it's actually even harder than that: it can't just be 10x better in practice, it has to appear 10x better at a glance. Like, at the window shopping phase, it needs to catch their eye. That's difficult to nail and basically impossible to do perfectly. Personally? I'm all-in on "whole sale" build tools like this that hit many nails with one hammer. They solve a tremendous amount of issues. But they aren't always good and I can really see why they can fail to gain traction for many teams: they simply aren't necessary for them, even if they're really a lot better in many ways. I get it.
- nisa 4y ago> Was your project built in multiple languages and tied together using Maven? Or was it just JDK languages/Java? Exactly that - for one it was a plugin for Alfresco ECM that disobeyed every Alfresco SDK rule :), then a custom Angular frontend that needs to be build according to the plugin - furthermore the business was of that kind that every customer had a specialized build with customization - Bazel would have solved that in a very neat way - including overriding a few source files for a specific customer. We tried to build then docker images with maven which was just a shitshow considering our problems - however I'm not there anymore but I'd like to have the exact setup that you describe - the big hammer to don't have to deal with all the stupid things - especially as my background is sysadmin and every problem I don't have to solve anymore is good for me :)
- rgoulter 4y ago> Tools like Bazel or Nix or ... I find it interesting that Bazel and Nix don't seem to complement each other as much as I'd think given that they're both 'high effort, high reward tools which deal with hermetic building'. They certainly both suffer from issues related to upstream packages doing their own thing. > They also suffer from another problem which is that most people don't care as much as you or I do. :) ... I use Nix. I haven't use Bazel. I think Nix is wonderful for enthusiasts. It takes some time to learn. However, Nix is excellent at dealing with packages. And developers often do things which involve packages. -- e.g. Nix is good for declaring a shell with programs that can be made available.
- dbaupp 4y agoWe’ve had a good experience using Pants 2 (https://www.pantsbuild.org/ https://www.pantsbuild.org/) and its dependency inference (https://blog.pantsbuild.org/why-dependency-inference/ https://blog.pantsbuild.org/why-dependency-inference/) for Python. Although we are in a monorepo, rather than a “multi-repo”.
- jitl 4y agoIt's cool that Pants has enough community to evolve beyond being a Twitter-led project. It seems Twitter itself migrated from Pants to Bazel, and no longer works on Pants: https://groups.google.com/g/pants-devel/c/PHVIbVDLhx8/m/LpSKIP5cAwAJ https://groups.google.com/g/pants-devel/c/PHVIbVDLhx8/m/LpSK... I find it hard to buy into one of these "mega build system" ecosystems because there are so many competing tools that descend from Blaze: Bazel, Pants, Buck, Buck2, etc. The migration path to these systems takes substantial effort, so I wouldn't want to bet on the wrong one and have to do it all again a few years later. I'm content to wait patiently for Bazel or Buck2 to clearly win before I invest.
- dbaupp 4y agoI have the sense that migrating to any hermetic, cacheable build system will be more effort than migrating from one to another: once a project is “safe” to have everything run in sandboxes, the mechanisms for setting up those sandboxes is less important. For example, once one system has accurate dependency information, that can be used as a base for setting up the second system (and presumably this info can be exported for machine-assisted migration). Of course, the details of each system differ enough that I imagine it’s not trivial.
- jen20 4y agoDo you know what the current maintenance story for Pants is following Twitter's move to Bazel?
- dbaupp 4y agoIt seems to be quite active, with regular new releases. I’m lead to believe that https://www.toolchain.com/ https://www.toolchain.com/ is the main sponsor, but there are maintainers from other companies too. I do not know the details, though.
- deleted 4y ago[deleted]
- jeffreygoesto 4y agoMaybe check out justbuild [0], the main author used to be a bazel dev and wanted to combine it's strengths with the possibility of multi repos and more possibilities to interact with the dependency tree. Quite interesting and very responsive team. [0] https://github.com/just-buildsystem/justbuild https://github.com/just-buildsystem/justbuild
- hobofan 4y agoAaarg, I hate to always be the person that points out "naming issues", but there is already a pretty popular command runner with the name just: https://github.com/casey/just https://github.com/casey/just
- jvolkman 4y agoI suppose that's the risk of picking a common word as your project name.
- tinglymintyfrsh 4y ago0. Does it do creating static libraries yet? 1. Does it do linking to a mix of static and dynamic libraries yet? 2. Anyone try buck2, sort of bzl-like?
- lalaland1125 4y ago0 and 1 have been supported for a while. See the "linkstatic" feature.
- en4bz 4y agoNo, they are extremely broken. You can only choose: - Link everything static. - Link everything dynamic. - Link user libs as static and system libs as dynamic. There is no easy way to link a single user lib static/dynamic without resorting to hacks/workarounds like re-importing the shared library or defining weird intermediate targets. It's completely broken.
- lalaland1125 4y agoYou can use the "linkstatic" feature on the cc_library level. Then that library will be linked statically while other cc_library's will be dynamically linked.
- en4bz 4y agoThe logic is backwards though. I may have multiple consumers of a library some of which may want static some of which may want dynamic. You need to create/import new targets to do this even though the original target creates both static and dynamic libs by default.
- emidln 4y agoThis is just false. Bazel creates both static and dynamic libs by default for every cc_library. The default is to link static, but you can control this on a per-binary basis. You don't need parallel trees of targets or even to write custom rules to access the non-default output groups that contain the shared library. This all just works out of the box.
- lalaland1125 4y agoThe main change here is that Bazel now has a package repository that people can pull/push external dependencies to: https://registry.bazel.build/ https://registry.bazel.build/ This is a huge change as external dependencies used to be one of the big pain points with the Bazel pipeline.
- bradhe 4y agoAwesome but kind of a bummer to see yet another dependency management tool, especially in the C++ ecosystem. Didn’t read that much of the article, but wonder if there’s interop plans with VCPkg or Conan?
- no_wizard 4y agoNX is a very bazel like for frontend focused repositories. I think what I struggle with the most is the opinions of the rules / plugins. Either I use it as a core task runner (easy, but lots of config the team has to maintain setting up script entries in packages etc) or I have to write custom plugins for anyting not on the "happy path". I think thats my general frustration with these tools, they aren't terribly well built for SDK type consumption, they don't expose a core set of APIs directly, instead relying on a plugin system that can sometimes be very opaque
- Leherenn 4y agoWe use Bazel at work for C++, and while defining targets and dependency is easy, I've really struggled with some stuff I would consider basic, like conditionally changing build flags, especially to external dependencies. Basically, I wanted to conditionally use sanitizers for a specific target and propagate the sanitizers compiler/linker flags to the whole dependency tree. For code you own, from what I could find, you need to duplicate each target and each dependency (and their own dependencies) for each possible sanitizer and then select the right one. And for external dependencies, you need to use "Aspects", if I understood correctly, to modify the build dependency graph and inject your flags. There's a question on stackoverflow about it, but the answer is very high level (presumably by a Bazel dev), and unfortunately I didn't have time to dig it further and just abandoned. I also had a lot of issues on an ARM Mac, including the linker issue that is apparently fixed (hooray) and recurrent crashes. Given we do not use any of best features of Bazel (remote/distributed compilation, multiple languages, and caching (because we basically spawn a new docker container for each build)), I think CMake would have made more sense.
- johntb86 4y agoI think you should be using user-defined transitions (https://bazel.build/extending/config#user-defined-transitions https://bazel.build/extending/config#user-defined-transition... ) to use sanitizers. They essentially duplicate the build graph, and you can set build flags on the transition so that all dependencies will be built with those flags.
- simplotek 4y agoFrom the announcement: > One of Bazel's most powerful features is its ability to use remote caching and remote execution. What's the value proposition of Bazel when build systems like cmake support this out of the box with third-party tools like distcc and ccache?
- en4bz 4y agoNone, bazel's caching implementation is broken because they don't even know or specify what constituents a build hash/key. See this issue from 2018 that's still open [1]. [1] https://github.com/bazelbuild/bazel/issues/4558 https://github.com/bazelbuild/bazel/issues/4558
- lopkeny12ko 4y agoProbably because building targets with tools outside of the workspace is an antipattern, as it violates hermiticity principles. In fact, Bazel generally makes it quite hard to do this, so anyone who ends up in this scenario must have jumped through many hoops to get there. I agree that the linked issue is legitimate, but I'd argue that this isn't a problem Bazel itself needs to solve--you should fix your build to be fully hermetic.
- en4bz 4y agoNon-Hermetic is the default for C/C++. And if you plan on using system provided libraries to support multi OSes then you can't use it.
- lopkeny12ko 4y agoThat is precisely the point--using system-provided libraries in your Bazel project is an antipattern that should be avoided.
- en4bz 4y agoIt's literally the default. How can the default be an anti-pattern. I doubt you're using C/C++ because you don't seem to understand the issue.