6 ms·

To ask the obvious unanswered question: Why would someone want to move to Bazel from Maven? What advantages does it bring?

erik_seaberg 6y ago
You get hermetic and cacheable builds, but only if you’re willing to abandon the whole Maven plugin ecosystem and handle a lot of codegen yourself.
- rockwotj 6y ago
  Bazel does embrace code gen, but to be honest I've found rules for most of the code gen I've needed to do and they plug in nicely.
- deepsun 6y ago
  I still don't understand, Maven artifacts are hermetic and cacheable, no?
  - dikei 6y ago
    * Bazel offer much more granular caching, because a Bazel BUILD target is often much smaller than a Maven artifact. For example: a Java BUILD target is often just a single package. * SNAPSHOT maven artifact are not hermetic and not reliable for caching: Maven only re-fetch SNAPSHOT once a day by defaults, so you will miss changes if you do not explicitly update it. In Bazel, any changes in a BUILD target's input are detected immediately and will trigger a rebuild.
  - erik_seaberg 6y ago
    Bazel rules are supposed to only read specified inputs and write specified deterministic outputs, and they’re skipped if nothing changed. Maven plugins can do whatever they want, and you decide whether to publish what's on disk.
georgewfraser 6y ago
Bazel guarantees that each build step depends only on its declared inputs, and that each step is deterministic. So if the inputs to a step have not changed, Bazel can skip that step and use a cached copy of the result of that step. If you break up a large multi module build into a tree of small steps, Bazel can perform an incremental build after a small change in a few minutes, even though running the entire build from scratch would take hours.
- chrisseaton 6y ago
  > Bazel guarantees that each build step depends only on its declared inputs How does it enforce that? Does it sandbox the compiler or something? Java compiler annotation processors can perform arbitrary IO.
  - colatkinson 6y ago
    Exactly that: https://docs.bazel.build/versions/master/sandboxing.html https://docs.bazel.build/versions/master/sandboxing.html There's also (I believe experimental) support for using Docker as a sandbox backend. That ends up being useful if you're using the remote build execution support: you can run a build locally in exactly the same environment it will run on a build farm.
    - chrisseaton 6y ago
      I wonder if this stops you reading the clock and random number sources during build? (I once accidentally baked a time and random number into a binary myself - and the random number thing would have been a serious security bug if we had not found it with test coverage. Would have been better if the build system disallowed it.)
      - rockwotj 6y ago
        You can actually input system stuff like this via: https://docs.bazel.build/versions/master/user-manual.html#flag--workspace_status_command https://docs.bazel.build/versions/master/user-manual.html#fl... They already have support for build times and labels, but you can add arbitrary stuff like got hashes too
Jakobeha 6y ago
Also, how does Bazel compare to Gradle? JetBrains and it seems basically every library has support for Gradle and sometimes Maven, but not Bazel. Gradle is verbose and has a really steep learning curve, but it's battle-tested and does its job really well. I don't really see why anyone would use Bazel instead.
- btbuilder 6y ago
  Gradle’s remote build cache which caches at the task level is nice and granular. Also we’ve never had a problem with a change requiring a rebuild not being detected. One thing that Bazel has that I wish gradle did was remote compilation and remote test execution rather than just remote caching of artifacts.
- Rebelgecko 6y ago
  For the most part, I think you've got it. A gradle build can be very complicated and very long. Bazel is relatively easy to learn and relatively concise (again that's relative to Gradle which for a Java dev basically requires you to learn 1-2 new languages if you want to understand the build file syntax), and IMO is MUCH better if you are working with multiple languages. However I haven't tried to use Bazel with things like native libs in the JVM... so it's possible that my Gradle opinion is biased because I used it with more complicated use cases.
- izacus 6y ago
  I think they're fundamentally opposed in their philosophy. Gradle: Just slap any kind of code into our build files and we'll run it. Everything is code, everything is customizable and you can muck around with everything. Bazel: Here are a few of build rules. All compilation needs to be deterministic, all tasks need clear deterministic outputs and all task generate clear deterministic outputs. It's very rigid and it uses that to get significant performance wins, especially when using a build farm.
  - kaba0 6y ago
    That’s absolutely not what Gradle is. Gradle only provides a DSL that is supposed to be used to describe the build process deterministically, and add some configurable scripting on top. For anything more complex, you should write plugins. Nonetheless, I would be interested in seeing a detailed comparison of the two. As far as I know, gradle is quite capable once someone understands what he/she does. https://melix.github.io/blog/2021/01/the-problem-with-gradle.html https://melix.github.io/blog/2021/01/the-problem-with-gradle...
exabrial 6y ago
I read what you meant, not what you said, had the same question.
dikei 6y ago
For me, the main advantage of Bazel is speed, when properly structured and configured, a Bazel build can: * Perform incremental build a magnitude faster than Maven, zero changes build finishes immediately. * Can have global caches for all builds across organization, so that a workstation build can automatically build incrementally upon a standard build, massively reduces build speed for big monorepo. * Very fast, incremental tests execution: test results are cached, so only tests affected by the changed code are run. Test executions are parallel by default. Some other things: * Pull in dependencies directly from a git repository, without having to publishing it first like in maven, this can be a plus or a minus. * Cross-languages build: you can have each of your module programmed in a different language, declare dependencies between them and use Bazel to perform incrementally build quickly and reliably. That said, maintaining a Bazel build system is also a magnitude more complex than a Maven one, especially if you're not using one of Google main language: Java/C++. Plugins for other languages are of varying quality.
- jhgb 6y ago
  Given that Google has a Common Lisp codebase, I wonder...do they also have Bazel plugins for Common Lisp?
  - bruckie 6y ago
    Yes
- sverhagen 6y ago
  ...and a lot of those arguments seem geared toward big codebases. Small micro-services, such as Spring Boot, with the bulk of the test execution going to @SpringBootTest tests, where parallelization has its drawbacks, may not profit from most of those "benefits". Fun fact, it seems like Maven was also meant to service a variety of languages. There's even some support for polyglot POM files. Aside from the niche C++ or JavaScript ("frontend") plugin, that never did much.
  - jzoch 6y ago
    Actually we have seen a ton of success using it for spring boot microservices - each microservice has its own BUILD file within the monorepo. Test execution in spring can be parallelized as needed (and not used if not needed!) but i find our tests run a lot faster under bazel due to the caching anyway
kylecordes 6y ago
To me, the strongest motivation for Bazel arrives with a complex (slow) build spanning multiple technology ecosystems. Bazel does a great job building all of them, without privileging one over all the others. It has all the key features you'd expect, dependency graph control, caching, distributed computation, etc. You can add support for whatever local complexities invariably arise with custom rules. The Java ecosystem has much of that available also though - even a large Java-only project is somewhat tough to justify moving away from the tools almost every developer will arrive already knowing.
- erik_seaberg 6y ago
  It came from Googlers, who are kind of notorious for not using the rest of the industry’s tools. Sometimes it’s because they pioneered the problem space, and their own tools really are that much better. Sometimes their tools used to be better but are now just tech debt. Sometimes it’s promo-driven development.
  - vips7L 6y ago
    > Sometimes their tools used to be better but are now just tech debt This is how I feel about most of Googles Java libs (guice, guava, gson). They used to be better, but now they're just tech debt.
    - alexhutcheson 6y ago
      What are the best-in-class replacements for those?
      - vips7L 6y ago
        Guava -> The JDK Std library, unless you need more performant collections provided by it. Guice -> If you need to support the jakarta.* namespace the only real answer is Weld or another CDI compliant implementation. Otherwise guice is probably still fine. GSON -> Jackson as always.
qbasic_forever 6y ago
Distributed builds, and even distributed cache of build outputs. At google scale a distributed cache means you don't even need to build anything except the smallest code unit that changed--your CI systems, etc. have already built and warmed the cache for almost any other thing you want to build.
- ma2rten 6y ago
  I thought that the distributed part wasn't opensourced.
  - qbasic_forever 6y ago
    For the cache: https://docs.bazel.build/versions/master/remote-caching.html https://docs.bazel.build/versions/master/remote-caching.html and https://github.com/buchgr/bazel-remote https://github.com/buchgr/bazel-remote You can actually just set it up to use a shared S3 bucket to hold the cache. For builds there are a few options to check out: https://github.com/jin/awesome-bazel#remote-caching-and-execution https://github.com/jin/awesome-bazel#remote-caching-and-exec...
    - londons_explore 6y ago
      That all looks like a massive security vulnerability. Anyone with write access to the cache can upload a malicious build artifact and end up getting it compiled into everyone else's binaries, effectively giving them access to act as any developer or the ability to get their code sneaked into a release undetected. The page doesn't describe that risk at all!

erik_seaberg 6y ago

You get hermetic and cacheable builds, but only if you’re willing to abandon the whole Maven plugin ecosystem and handle a lot of codegen yourself.

rockwotj 6y ago
Bazel does embrace code gen, but to be honest I've found rules for most of the code gen I've needed to do and they plug in nicely.
deepsun 6y ago
I still don't understand, Maven artifacts are hermetic and cacheable, no?
- dikei 6y ago
  * Bazel offer much more granular caching, because a Bazel BUILD target is often much smaller than a Maven artifact. For example: a Java BUILD target is often just a single package. * SNAPSHOT maven artifact are not hermetic and not reliable for caching: Maven only re-fetch SNAPSHOT once a day by defaults, so you will miss changes if you do not explicitly update it. In Bazel, any changes in a BUILD target's input are detected immediately and will trigger a rebuild.
- erik_seaberg 6y ago
  Bazel rules are supposed to only read specified inputs and write specified deterministic outputs, and they’re skipped if nothing changed. Maven plugins can do whatever they want, and you decide whether to publish what's on disk.

georgewfraser 6y ago

Bazel guarantees that each build step depends only on its declared inputs, and that each step is deterministic. So if the inputs to a step have not changed, Bazel can skip that step and use a cached copy of the result of that step. If you break up a large multi module build into a tree of small steps, Bazel can perform an incremental build after a small change in a few minutes, even though running the entire build from scratch would take hours.

chrisseaton 6y ago
> Bazel guarantees that each build step depends only on its declared inputs How does it enforce that? Does it sandbox the compiler or something? Java compiler annotation processors can perform arbitrary IO.
- colatkinson 6y ago
  Exactly that: https://docs.bazel.build/versions/master/sandboxing.html https://docs.bazel.build/versions/master/sandboxing.html There's also (I believe experimental) support for using Docker as a sandbox backend. That ends up being useful if you're using the remote build execution support: you can run a build locally in exactly the same environment it will run on a build farm.
  - chrisseaton 6y ago
    I wonder if this stops you reading the clock and random number sources during build? (I once accidentally baked a time and random number into a binary myself - and the random number thing would have been a serious security bug if we had not found it with test coverage. Would have been better if the build system disallowed it.)
    - rockwotj 6y ago
      You can actually input system stuff like this via: https://docs.bazel.build/versions/master/user-manual.html#flag--workspace_status_command https://docs.bazel.build/versions/master/user-manual.html#fl... They already have support for build times and labels, but you can add arbitrary stuff like got hashes too

Jakobeha 6y ago

Also, how does Bazel compare to Gradle? JetBrains and it seems basically every library has support for Gradle and sometimes Maven, but not Bazel. Gradle is verbose and has a really steep learning curve, but it's battle-tested and does its job really well. I don't really see why anyone would use Bazel instead.

btbuilder 6y ago
Gradle’s remote build cache which caches at the task level is nice and granular. Also we’ve never had a problem with a change requiring a rebuild not being detected. One thing that Bazel has that I wish gradle did was remote compilation and remote test execution rather than just remote caching of artifacts.
Rebelgecko 6y ago
For the most part, I think you've got it. A gradle build can be very complicated and very long. Bazel is relatively easy to learn and relatively concise (again that's relative to Gradle which for a Java dev basically requires you to learn 1-2 new languages if you want to understand the build file syntax), and IMO is MUCH better if you are working with multiple languages. However I haven't tried to use Bazel with things like native libs in the JVM... so it's possible that my Gradle opinion is biased because I used it with more complicated use cases.
izacus 6y ago
I think they're fundamentally opposed in their philosophy. Gradle: Just slap any kind of code into our build files and we'll run it. Everything is code, everything is customizable and you can muck around with everything. Bazel: Here are a few of build rules. All compilation needs to be deterministic, all tasks need clear deterministic outputs and all task generate clear deterministic outputs. It's very rigid and it uses that to get significant performance wins, especially when using a build farm.
- kaba0 6y ago
  That’s absolutely not what Gradle is. Gradle only provides a DSL that is supposed to be used to describe the build process deterministically, and add some configurable scripting on top. For anything more complex, you should write plugins. Nonetheless, I would be interested in seeing a detailed comparison of the two. As far as I know, gradle is quite capable once someone understands what he/she does. https://melix.github.io/blog/2021/01/the-problem-with-gradle.html https://melix.github.io/blog/2021/01/the-problem-with-gradle...

exabrial 6y ago

I read what you meant, not what you said, had the same question.

dikei 6y ago

For me, the main advantage of Bazel is speed, when properly structured and configured, a Bazel build can: * Perform incremental build a magnitude faster than Maven, zero changes build finishes immediately. * Can have global caches for all builds across organization, so that a workstation build can automatically build incrementally upon a standard build, massively reduces build speed for big monorepo. * Very fast, incremental tests execution: test results are cached, so only tests affected by the changed code are run. Test executions are parallel by default. Some other things: * Pull in dependencies directly from a git repository, without having to publishing it first like in maven, this can be a plus or a minus. * Cross-languages build: you can have each of your module programmed in a different language, declare dependencies between them and use Bazel to perform incrementally build quickly and reliably. That said, maintaining a Bazel build system is also a magnitude more complex than a Maven one, especially if you're not using one of Google main language: Java/C++. Plugins for other languages are of varying quality.

jhgb 6y ago
Given that Google has a Common Lisp codebase, I wonder...do they also have Bazel plugins for Common Lisp?
- bruckie 6y ago
  Yes
sverhagen 6y ago
...and a lot of those arguments seem geared toward big codebases. Small micro-services, such as Spring Boot, with the bulk of the test execution going to @SpringBootTest tests, where parallelization has its drawbacks, may not profit from most of those "benefits". Fun fact, it seems like Maven was also meant to service a variety of languages. There's even some support for polyglot POM files. Aside from the niche C++ or JavaScript ("frontend") plugin, that never did much.
- jzoch 6y ago
  Actually we have seen a ton of success using it for spring boot microservices - each microservice has its own BUILD file within the monorepo. Test execution in spring can be parallelized as needed (and not used if not needed!) but i find our tests run a lot faster under bazel due to the caching anyway

kylecordes 6y ago

To me, the strongest motivation for Bazel arrives with a complex (slow) build spanning multiple technology ecosystems. Bazel does a great job building all of them, without privileging one over all the others. It has all the key features you'd expect, dependency graph control, caching, distributed computation, etc. You can add support for whatever local complexities invariably arise with custom rules. The Java ecosystem has much of that available also though - even a large Java-only project is somewhat tough to justify moving away from the tools almost every developer will arrive already knowing.

erik_seaberg 6y ago
It came from Googlers, who are kind of notorious for not using the rest of the industry’s tools. Sometimes it’s because they pioneered the problem space, and their own tools really are that much better. Sometimes their tools used to be better but are now just tech debt. Sometimes it’s promo-driven development.
- vips7L 6y ago
  > Sometimes their tools used to be better but are now just tech debt This is how I feel about most of Googles Java libs (guice, guava, gson). They used to be better, but now they're just tech debt.
  - alexhutcheson 6y ago
    What are the best-in-class replacements for those?
    - vips7L 6y ago
      Guava -> The JDK Std library, unless you need more performant collections provided by it. Guice -> If you need to support the jakarta.* namespace the only real answer is Weld or another CDI compliant implementation. Otherwise guice is probably still fine. GSON -> Jackson as always.

qbasic_forever 6y ago

Distributed builds, and even distributed cache of build outputs. At google scale a distributed cache means you don't even need to build anything except the smallest code unit that changed--your CI systems, etc. have already built and warmed the cache for almost any other thing you want to build.

ma2rten 6y ago
I thought that the distributed part wasn't opensourced.
- qbasic_forever 6y ago
  For the cache: https://docs.bazel.build/versions/master/remote-caching.html https://docs.bazel.build/versions/master/remote-caching.html and https://github.com/buchgr/bazel-remote https://github.com/buchgr/bazel-remote You can actually just set it up to use a shared S3 bucket to hold the cache. For builds there are a few options to check out: https://github.com/jin/awesome-bazel#remote-caching-and-execution https://github.com/jin/awesome-bazel#remote-caching-and-exec...
  - londons_explore 6y ago
    That all looks like a massive security vulnerability. Anyone with write access to the cache can upload a malicious build artifact and end up getting it compiled into everyone else's binaries, effectively giving them access to act as any developer or the ability to get their code sneaked into a release undetected. The page doesn't describe that risk at all!