9 ms·
Exodus: Easily migrate your JVM code from Maven to Bazel
- Twirrim 6y agoTo ask the obvious unanswered question: Why would someone want to move to Bazel from Maven? What advantages does it bring?
- erik_seaberg 6y agoYou get hermetic and cacheable builds, but only if you’re willing to abandon the whole Maven plugin ecosystem and handle a lot of codegen yourself.
- rockwotj 6y agoBazel does embrace code gen, but to be honest I've found rules for most of the code gen I've needed to do and they plug in nicely.
- deepsun 6y agoI still don't understand, Maven artifacts are hermetic and cacheable, no?
- dikei 6y ago* Bazel offer much more granular caching, because a Bazel BUILD target is often much smaller than a Maven artifact. For example: a Java BUILD target is often just a single package. * SNAPSHOT maven artifact are not hermetic and not reliable for caching: Maven only re-fetch SNAPSHOT once a day by defaults, so you will miss changes if you do not explicitly update it. In Bazel, any changes in a BUILD target's input are detected immediately and will trigger a rebuild.
- erik_seaberg 6y agoBazel rules are supposed to only read specified inputs and write specified deterministic outputs, and they’re skipped if nothing changed. Maven plugins can do whatever they want, and you decide whether to publish what's on disk.
- georgewfraser 6y agoBazel guarantees that each build step depends only on its declared inputs, and that each step is deterministic. So if the inputs to a step have not changed, Bazel can skip that step and use a cached copy of the result of that step. If you break up a large multi module build into a tree of small steps, Bazel can perform an incremental build after a small change in a few minutes, even though running the entire build from scratch would take hours.
- chrisseaton 6y ago> Bazel guarantees that each build step depends only on its declared inputs How does it enforce that? Does it sandbox the compiler or something? Java compiler annotation processors can perform arbitrary IO.
- colatkinson 6y agoExactly that: https://docs.bazel.build/versions/master/sandboxing.html https://docs.bazel.build/versions/master/sandboxing.html There's also (I believe experimental) support for using Docker as a sandbox backend. That ends up being useful if you're using the remote build execution support: you can run a build locally in exactly the same environment it will run on a build farm.
- chrisseaton 6y agoI wonder if this stops you reading the clock and random number sources during build? (I once accidentally baked a time and random number into a binary myself - and the random number thing would have been a serious security bug if we had not found it with test coverage. Would have been better if the build system disallowed it.)
- rockwotj 6y agoYou can actually input system stuff like this via: https://docs.bazel.build/versions/master/user-manual.html#flag--workspace_status_command https://docs.bazel.build/versions/master/user-manual.html#fl... They already have support for build times and labels, but you can add arbitrary stuff like got hashes too
- Jakobeha 6y agoAlso, how does Bazel compare to Gradle? JetBrains and it seems basically every library has support for Gradle and sometimes Maven, but not Bazel. Gradle is verbose and has a really steep learning curve, but it's battle-tested and does its job really well. I don't really see why anyone would use Bazel instead.
- btbuilder 6y agoGradle’s remote build cache which caches at the task level is nice and granular. Also we’ve never had a problem with a change requiring a rebuild not being detected. One thing that Bazel has that I wish gradle did was remote compilation and remote test execution rather than just remote caching of artifacts.
- Rebelgecko 6y agoFor the most part, I think you've got it. A gradle build can be very complicated and very long. Bazel is relatively easy to learn and relatively concise (again that's relative to Gradle which for a Java dev basically requires you to learn 1-2 new languages if you want to understand the build file syntax), and IMO is MUCH better if you are working with multiple languages. However I haven't tried to use Bazel with things like native libs in the JVM... so it's possible that my Gradle opinion is biased because I used it with more complicated use cases.
- izacus 6y agoI think they're fundamentally opposed in their philosophy. Gradle: Just slap any kind of code into our build files and we'll run it. Everything is code, everything is customizable and you can muck around with everything. Bazel: Here are a few of build rules. All compilation needs to be deterministic, all tasks need clear deterministic outputs and all task generate clear deterministic outputs. It's very rigid and it uses that to get significant performance wins, especially when using a build farm.
- kaba0 6y agoThat’s absolutely not what Gradle is. Gradle only provides a DSL that is supposed to be used to describe the build process deterministically, and add some configurable scripting on top. For anything more complex, you should write plugins. Nonetheless, I would be interested in seeing a detailed comparison of the two. As far as I know, gradle is quite capable once someone understands what he/she does. https://melix.github.io/blog/2021/01/the-problem-with-gradle.html https://melix.github.io/blog/2021/01/the-problem-with-gradle...
- exabrial 6y agoI read what you meant, not what you said, had the same question.
- dikei 6y agoFor me, the main advantage of Bazel is speed, when properly structured and configured, a Bazel build can: * Perform incremental build a magnitude faster than Maven, zero changes build finishes immediately. * Can have global caches for all builds across organization, so that a workstation build can automatically build incrementally upon a standard build, massively reduces build speed for big monorepo. * Very fast, incremental tests execution: test results are cached, so only tests affected by the changed code are run. Test executions are parallel by default. Some other things: * Pull in dependencies directly from a git repository, without having to publishing it first like in maven, this can be a plus or a minus. * Cross-languages build: you can have each of your module programmed in a different language, declare dependencies between them and use Bazel to perform incrementally build quickly and reliably. That said, maintaining a Bazel build system is also a magnitude more complex than a Maven one, especially if you're not using one of Google main language: Java/C++. Plugins for other languages are of varying quality.
- jhgb 6y agoGiven that Google has a Common Lisp codebase, I wonder...do they also have Bazel plugins for Common Lisp?
- bruckie 6y agoYes
- sverhagen 6y ago...and a lot of those arguments seem geared toward big codebases. Small micro-services, such as Spring Boot, with the bulk of the test execution going to @SpringBootTest tests, where parallelization has its drawbacks, may not profit from most of those "benefits". Fun fact, it seems like Maven was also meant to service a variety of languages. There's even some support for polyglot POM files. Aside from the niche C++ or JavaScript ("frontend") plugin, that never did much.
- jzoch 6y agoActually we have seen a ton of success using it for spring boot microservices - each microservice has its own BUILD file within the monorepo. Test execution in spring can be parallelized as needed (and not used if not needed!) but i find our tests run a lot faster under bazel due to the caching anyway
- kylecordes 6y agoTo me, the strongest motivation for Bazel arrives with a complex (slow) build spanning multiple technology ecosystems. Bazel does a great job building all of them, without privileging one over all the others. It has all the key features you'd expect, dependency graph control, caching, distributed computation, etc. You can add support for whatever local complexities invariably arise with custom rules. The Java ecosystem has much of that available also though - even a large Java-only project is somewhat tough to justify moving away from the tools almost every developer will arrive already knowing.
- erik_seaberg 6y agoIt came from Googlers, who are kind of notorious for not using the rest of the industry’s tools. Sometimes it’s because they pioneered the problem space, and their own tools really are that much better. Sometimes their tools used to be better but are now just tech debt. Sometimes it’s promo-driven development.
- vips7L 6y ago> Sometimes their tools used to be better but are now just tech debt This is how I feel about most of Googles Java libs (guice, guava, gson). They used to be better, but now they're just tech debt.
- alexhutcheson 6y agoWhat are the best-in-class replacements for those?
- vips7L 6y agoGuava -> The JDK Std library, unless you need more performant collections provided by it. Guice -> If you need to support the jakarta.* namespace the only real answer is Weld or another CDI compliant implementation. Otherwise guice is probably still fine. GSON -> Jackson as always.
- qbasic_forever 6y agoDistributed builds, and even distributed cache of build outputs. At google scale a distributed cache means you don't even need to build anything except the smallest code unit that changed--your CI systems, etc. have already built and warmed the cache for almost any other thing you want to build.
- ma2rten 6y agoI thought that the distributed part wasn't opensourced.
- qbasic_forever 6y agoFor the cache: https://docs.bazel.build/versions/master/remote-caching.html https://docs.bazel.build/versions/master/remote-caching.html and https://github.com/buchgr/bazel-remote https://github.com/buchgr/bazel-remote You can actually just set it up to use a shared S3 bucket to hold the cache. For builds there are a few options to check out: https://github.com/jin/awesome-bazel#remote-caching-and-execution https://github.com/jin/awesome-bazel#remote-caching-and-exec...
- londons_explore 6y agoThat all looks like a massive security vulnerability. Anyone with write access to the cache can upload a malicious build artifact and end up getting it compiled into everyone else's binaries, effectively giving them access to act as any developer or the ability to get their code sneaked into a release undetected. The page doesn't describe that risk at all!
- deleted 6y ago[deleted]
- voidfunc 6y agoSeriously tried adopting Bazel on a team once and never again. Its a huge headache and requires a dedicated engineer resource to be available to assist/debug/teach when inevitably someone wants to do (or does) something outside the structure and expectations of Bazel. Every so often I try again on a personal project and realize im spending more time futzing with Bazel than writing code. Its a tool that’s perfect for Google.
- sto_hristo 6y agoIt's a build framework like gradle but with python instead of groovy. It's extremely powerful, just like gradle is, so it requires considerable time to get comfortable with it. I never regretted spending the time with gradle as my builds get more and more flexible and good in time. I don't think i will ever push to bazel though, as to me it looks like gradle with a python dsl.
- rtpg 6y agoAn alternate voice here: it took a long time for us to get Bazel (and there's still a lot of headbanging on stuff) but if you want fast CI times, a huge test suite, and not have to deal with like 20 repositories (especially if you want cross-cutting PRs) it's basically the only reasonable tool that will get you there. And yeah, you gotta have someone spend a lot of time with it (maybe comparable to futzing about with some JS webpack spaghetti-code... thing). I think if you're _not_ using JS (and to a lesser extent, Python), it feels like a no-brainer. Unfortunately it's super oriented around getting build artifacts, which doesn't work so well with Python/JS so you're going to have to fight stuff a bit (though the Bazel Slack channel is filled with pretty helpful people). Overall I've gotten way madder at the other tooling for being "wrong" more than Bazel itself (lots of people really do random stuff when faced with symlinks for example). I don't know what the "right" amount of time is, but if you current have 30min+ CI times, this can cut stuff down way more and get you much closer to a nice CI/CD flow. The thing that kinda kills me is that it feels purposefully obtuse at time (in particular about obsfucating the custom rules stuff, despite it being _the_ way to wrap existing projects nicely). But all the Xooglers who end up writing Bazel clones somehow end up with even more obtuse tools...
- deleted 6y ago[deleted]
- bedobi 6y agoI know nothing about Bazel, but I've never understood the JVM community's obsession with getting off Maven. My experience of Gradle is Gradle projects require a PhD in Gradle AND intimate familiarity with the tastes of whoever implemented that particular project .gradle files likes to use it. Is Bazel like that too? When we inherit codebases from other teams, often one of the first things we do is convert it from Gradle to Maven. I honestly couldn't care less about how flexible and powerful Gradle or Bazel or some other build tool is compared to Maven. Maven is simple, the same in every project (this is huuuuuge), just works and gets out the way.
- Freak_NL 6y ago> […] the same in every project (this is huuuuuge) […] Hard to overstate how useful that is. Configuring something in a Maven build may be verbose or sometimes hard, but most of the time the build just does its thing, and all developers know at least `mvn clean install`.
- brown9-2 6y agoOne of the main goals of the project was “convention over configuration” and it succeeded well at that.
- utf_8x 6y agoExactly. You could have a million line pom.xml and you'd still most likely just need `mvn clean package` to get a usable JAR. XML is a nasty format but it just works...
- Traubenfuchs 6y agoMaven is one of the most stable and "good enough" tools there is -which is why a significant part of software engineers hate it. If you enjoy using gradle for its customizability and flexibility and power -you are doing it wrong. You have created an overengineered monster I would dread to work with. I don't want to learn groovy, I want to develop applications.
- 6y ago
- rendaw 6y agoWith Maven you can have circular dependencies within a module, whereas Bazel IIRC gets its speed from having a graph of small modules where only modules that change get rebuilt, but which places strict requirements on circle-free code. So how does this work with maven? Does it just make one huge bazel module that gets recompiled every time? I'd be interested in some more details in the readme. Also FWIW bazel is the only official way to use j2cl to compile java to javascript.