16 ms·

Bazel 1.0

411 points by vips7L 7y ago

1propionyl 7y ago
Because I have a twisted fascination with controversy and tribalism on HN, here's a little red meat: I'm going to stick with using make to glue together whatever tools I need for my build and test processes. Edit: -5 in under 10m. And yet I didn't even say anything remotely controversial or normative. Fantastic.
- vore 7y ago
  Maybe it's because your stated goal is baiting a flamewar in bad faith.
- taneq 7y ago
  It's probably because you came off as a bit unhinged and opened with a salvo against HN (which many people here understandably like). If you'd phrased it as "meh, I like make" you'd probably have half a dozen upvotes from the minimalists.
- mrath 7y ago
  For me the obstacle for using make is, it is not available for windows and even if I install via msys2 or cygwin, it is hard it make Makefiles independent of OS. I am not sure Bazel is the answer. BUILD files feel verbose and need more effort to maintain than platform specific files(go.mod, Cargo.toml, pom.xml).
  - srgpqt 7y ago
    Visual Studio (used to?) come with nmake, a mostly compatible version of Make.
    - 1propionyl 7y ago
      <Microsoft Product> used to come with mostly compatible version of <Unix tool>. Yup.
      - flukus 7y ago
        I'm not sure if MS is to blame there, in my very brief experience it was compatible enough, it was the lack of unix tools and gnu extensions that made it difficult to work with. You'd probably get the same issue trying to be posix compatible.
  - flukus 7y ago
    As long as you've got a basic unix like environment what's hard about being cross platform? Last time I tried it was very simple, at least with higher level languages and using gnu make specific features. For build tools I think "defining their own platform" tends to be a better description than being cross platform, the have there own languages, tools and idioms. In this case it's on top of the java platform as well.
- dang 7y ago
  Please don't troll here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
krapht 7y ago
Is it worth giving up language-specific build tools (CMake, Maven, Gradle and others) for one build tool to rule them all?
- jrockway 7y ago
  My experience with using the Google-internal version of Bazel is that yes, you want to give up all other tools to use this. Having a proper dependency graph makes it possible to cache maximally and accurately, and that results in fast builds. You don't need it if you only have like one go program that you're building, but you will start to see the disadvantages when you have a handful of things you build. Did you build all the apps that depend on the proto you updated? Do you need to build your PHP app because you change an internal go library? Tools like "docker build" have no idea and tend to rebuild too much or too little, and yield different results on different machines. This causes a lot of headaches. The thing that kills me about Bazel is that it is quite painful to deal with Java. I never seem to get a working version on a Linux distribution. On Windows it's always sitting down in the taskbar showing ads. Oracle calls you to demand money. You check the bounds of an array and they sue you. It's just not worth it. So I don't actually use Bazel.
  - klodolph 7y ago
    On Linux I just install a headless OpenJDK 1.8. Something similar is available through brew on the Mac. Oracle’s version is dead as far as I care.
    - pjmlp 7y ago
      Who do you think writes 90% of OpenJDK and drives language design?
  - roneythomas6 7y ago
    I use Amazon Corretto. There are few other openJDK distributions.
  - vips7L 7y ago
taneq 7y ago
Problem: There are N confusing build systems with arcane rules and you just want to compile your program. Solution: Design a simple, straightforward build tool. Problem: There are N+1 confusing build systems with arcane rules.
- new_realist 7y ago
  I wouldn’t call Bazel confusing or arcane, but it is powerful. I find it rather delightful to use.
  - mook 7y ago
    As a relatively newcomer to bazel, I did find it arcane: - I couldn't find an easy way to list targets - in particular, a way to show me the path to the build artifacts (so that I could inspect it in my editor) - I couldn't find a way to do build configurations (especially intersections of build configurations) - it didn't seem great at handling non-file build steps (set up a cluster / server / whatever so I can run my tests) - the documentation was slow to load and hard to find things in (you basically had to use search, instead of browsing a good table of contents) As I said, though, I'm a newcomer; it's quite possible there are good ways to do those things, I just hadn't managed to find it in the documenting yet. That and the issues are by no means unique to bazel. I also do like some aspects of it (like enforced out-of-tree builds), but that's off-topic ;)
    - joshuamorton 7y ago
      Configs: look into select() and modes. Non-file steps: normally in bazel you would consider the server setup a part of the test. So `blaze build :my_test` creates an artifact, and blaze run or blaze test runs the artifact which does non-hermetic stuff like setting up a server.
    - emidln 7y ago
      Listing targets in a package: bazel query //some/package:* Listing targets in a package including subpackages: bazel query //some/package/... These and more are covered in the documentation on querying: https://docs.bazel.build/versions/1.0.0/query.html#target-patterns https://docs.bazel.build/versions/1.0.0/query.html#target-pa...
RcouF1uZ4gsC 7y ago
I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.
- RhodesianHunter 7y ago
  Why do you care if a JVM is on your dev machine?
  - thejosh 7y ago
    - One more thing to install - Does this work for 8/11/12/13/14/15/16?
    - cromwellian 7y ago
      It's a stripped down JVM is bundled with bazel, you don't need to install it.
  - weberc2 7y ago
    My 2015 MBP is crawling between Safari, Docker, VS Code, and BitDefender (antivirus). Not sure how bulky the JVM is these days. That said, I wish it supported Python.
    - harrygeez 7y ago
      Why do you even have Bitdefender installed?
      - the_mitsuhiko 7y ago
        A lot of people have it for compliance reasons.
        slrz 7y ago
lenkite 7y ago
Whats the advantage of using Bazel over CMake ? What does one do about package management ?
- sayrer 7y ago
  For packages, you download things by specifying an archive (zip file, git repo, etc), and a SHA256 hash. Think of it as extremely-precise version pinning. You can also refer to other Bazel projects and use their internal rules if permitted. The biggest advantages of Bazel over something like CMake are: 1) It is very general, so you can easily express dependencies of this sort: - concat three text files, then - run them through a parser generator, then - use the generated parser in a C library, then - use that C library in a Go server library and a Rust client library, then - package the Go server into a Docker container and build an iOS binary that uses the Rust client This is a cross-language dependency chain that also runs a few Unix tools. In this example, if you edit the Rust client, only the Rust library and the iOS client will be rebuilt. If you edit one of the text files that compose the grammar, everything will get rebuilt. The dependencies are also a DAG, so if your projects have a shared Go library and you change that, the Go server would get rebuilt, as would the Docker container. While all this might seem distressingly general, it removes the need for things like "bootstrap.sh" files, "build.rs" files, and all of the unstated dependencies they entail. 2) If you download all of the toolchains (C++, Java, Rust, Go, etc), it's impossible to have "works on my machine" problems. Exceptions are use of "local_*" rules, and Apple binaries (you need machine-local SDKs). 3) This allows great caching. You can run the equivalent of ccache for all of this stuff, locally or shared on a LAN. RBE is even more advanced.
  - fanf2 7y ago
    (1) sounds like table stakes for any build system - `make` has done all that for decades
    - sayrer 7y ago
      Well, sort of. The way make tracks changes triggers more rebuilds and the build products are much less reusable. There's a good comparison table at the bottom of section 2.4 in https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems-final.pdf https://www.microsoft.com/en-us/research/uploads/prod/2018/0... Bazel's dependency graph tends to be much more detailed than Makefiles, extends all the way to toolchains at the base, and all the way up to build products and test results. It can also be composed together across huge file layouts in ways make doesn't support.
lngnmn1 7y ago
Those who wrote make (or mk of Plan9) were way, way smarter and much humbler. No, we don't have to invent an idiotic DSL where declarative approach and sane defaults will do.
jdkee 7y ago
What does Bazel do exactly?
- mc3 7y ago
  Bazel is an open-source build and test tool similar to Make, Maven, and Gradle. It uses a human-readable, high-level build language. Bazel supports projects in multiple languages and builds outputs for multiple platforms. Bazel supports large codebases across multiple repositories, and large numbers of users.
  - kajecounterhack 7y ago
    It also builds code repeatably & hermetically (interpreted languages excepted; at google python is built super hermetically where a python binary + environment is bundled with python script(s) into an executable; some companies like dropbox have copied this). You run build, and you get binaries for N kinds of systems (e.g x86_64, ARM). It builds a dependency graph of your code that you can query and results in cache-able builds (only rebuild the part you need). Since building dependencies in a tree can be massively parallelized, it also allows you to parallelize your builds well across machines (with some effort). You use the one tool to compile C++, Java, Python, Go, Rust, etc.
pcj-github 7y ago
For me, the difference between other build systems and bazel was like the shift from CVS to git... Profoundly better internal data model, but definitely a learning curve.
- spuz 7y ago
  That makes sense but there aren't many people out there using CVS right now. What were you using before Bazel? I.e. what are you comparing Bazel to?
dwrodri 7y ago
My only experience with Bazel is using it for Tensorflow (TF). It seemed very roundabout for what was required and definitely had some painful version issues as TF got closer to 2.0. Hopefully this means more stability and ease of use going forward for TF users.
person_of_color 7y ago
Learn CMake.
throwaway1777 7y ago
In my opinion Bazel/Blaze and Buck are mainly useful to solve the problem of having too much code to build on one machine. Reproducible builds which can be done in a distributed fashion aren’t really useful when you can build your whole project in a minute or two on one machine. That being said, once you get the hang of it the overhead is not too high compared to simpler systems.
- a_t48 7y ago
  None of the last three companies I worked at could build in under a minute, sadly.
- indolering 7y ago
  There are security reasons to want a reproducible build.
- kartayyar 7y ago
  Xoogler here who used the internal version of Bazel ( blaze ) extensively, liked it and now opted to use it for our 2 person startup ( at least for the backend ). You don't need a giant codebase to start benefitting from it. Even if you have a smaller codebase, and properly use bazel to setup tests, you only test what needs testing, and that makes a huge difference in productivity.
  - MapleWalnut 7y ago
    What languages are you using with Bazel? I have a project that uses Typescript and Python. Are you using Bazel with interpreted languages?
    - evmar 7y ago
      Bazel has experimental but pretty sophisticated support for TypeScript, with arguably a better compilation experience for large projects than the best known alternatives. (Disclaimer: I'm biased because I work on it, but I also work on non-bazel TypeScript projects so I am aware of the tradeoffs.) https://bazelbuild.github.io/rules_nodejs/TypeScript.html https://bazelbuild.github.io/rules_nodejs/TypeScript.html
slavik81 7y ago
The lack of backwards compatibility between Bazel versions has been the deal-killer for me thus far. Old build instructions should work with newer releases of the same tools. It's important for the tools to remain compatible because you can't change the instructions contained in old releases. I hope the release of 1.0 is an indication that Bazel is taking interface stability more seriously.
- ageofwant 7y ago
  Right. Every time I build tensorflow I have to build bazel first, or get the bazel version for the tf version. I have to say, having used a number of Google's tools over the years: basel, coral sdk, maps api and perhaps others, I'm always surprised at how 'how ya doing' the experience is. Don't get me wrong, I appreciate their tools and its openness, but I wish Google spend a bit more time doing software engineering, and less throwing shit against a wall ascertaining stickiness.
  - philwo 7y ago
    > Every time I build tensorflow I have to build bazel first, or get the bazel version for the tf version. This is fixed now: https://github.com/tensorflow/tensorflow/commit/991aec351b574c4ec9157b9e557fb0f7f53bc4d9 https://github.com/tensorflow/tensorflow/commit/991aec351b57... If you use Bazelisk (https://github.com/bazelbuild/bazelisk https://github.com/bazelbuild/bazelisk), you'll automatically get the matching Bazel version for TensorFlow in the future (and all other projects that have adopted the .bazelversion file).
    - slavik81 7y ago
      Awesome. Bazelisk provides exactly the sort of compatibility I was hoping to see. Though, manually specifying a maximum Bazel version doesn't seem like it should be necessary. The authors of Bazel know what features they have changed since the version specified as the minimum, and Bazel itself knows what features the BUILD file uses. That information together could be used to calculate the maximum version.
- 7y ago
bazza451 7y ago
Does anyone know of a real beginners level guide to Bazel that isn’t the docs. Got a typescript monorepo with a few backend services/react frontends I wanted to build and I got a bit lost in the complexity of it due to package.json handling
- G2H 7y ago
  We wrote a bit about how we build a TS monorepo with Bazel on our company's blog [1] if that can be helpful [1] https://dataform.co/blog/typescript-monorepo-with-bazel/ https://dataform.co/blog/typescript-monorepo-with-bazel/
  - bazza451 7y ago
    thanks will check it out
choppaface 7y ago
One of the major time savers of bazel is Remote Build Execution (RBE), which allows you to build modules in parallel in the cloud. So if you have 1000 CPUs, you can really just have a client do `bazel build -j 1000 //...` and you can get a huge speed-up. Remote (and local) builds all happen in a sandbox, so you don't have to worry about e.g. preparing a docker image with the worker / build slave environment. (You do, however, have to register your dependencies with Bazel, which can be a hard at first). To add to this, bazel also has a remote global cache which can benefit large teams. For fairly large C++ codebases, RBE is really a competitive advantage. I've seen RBE cut down iteration time by an order of magnitude. I love CMake, and CMake can get you plenty of parallelism, but CMake doesn't really provide a tool for building several CMake sub-projects in parallel, and bazel handles this really well. Sadly Bazel RBE is still primarily a Google Cloud product. Also, GCE is slow to work on supporting auto-scale, so you have to pay for unused workers. (Like most products in Google Cloud, Google is ripping you off with alpha-quality stuff). There's some very rough open source RBE stuff on Github that you can run yourself, but nothing really production-grade yet. gg ( https://github.com/StanfordSNR/gg https://github.com/StanfordSNR/gg ) is a promising-looking alternative. It's research code, but it might be the community's best hope for a non-Google alternative (that e.g. supports AWS Lambda for parallelism). Bazel is great, but without independence (e.g. what Kubernetes achieved) it's difficult to see bazel as dependable as make or CMake long term.
- 9nGQluzmnq3M 7y ago
  > GCE is slow to work on supporting auto-scale, so you have to pay for unused workers. Can you expand on what you mean here? GCE has had autoscaling for years and it's quite configurable, I don't see why you would need to keep unused workers around unless your build load is extremely spiky.
  - choppaface 7y ago
    Not auto-scaling VM compute instances, auto-scaling the Google Cloud RBE service (I think it's called Cloud Build right now?). If you have a worker pool of size 1,000, you have to pay for that even if your dev team only uses it at full capacity 8-10hrs per day. Or maybe the Google Cloud sales people working with my company are misinformed? IDK, but my experience with TPUs, GPUs, AI Engine, Dataproc, BigQuery, etc... every GCloud feature I've used (besides vanilla Compute VMs) have gotchas and/or time sinks that are costly unless you have a load of free credits. For AI Engine, it took like 6 months to upgrade their internal Kubernetes cluster; in the meantime, crashed jobs would burn an extra 15 minutes of GPU dollars. There's a lot of good stuff in Google Cloud, but be sure to get Google to pay you to wade through the construction with free credits.
shusson 7y ago
Anyone had any experience using Bazel with Angular projects?
panpanna 7y ago
Looking forward to seeing this as a requirement in projects that already require npm, phyton, cmake, gradle and ninja to compile :(
human_error 7y ago
How is it different than Buck or Pants? What makes Bazel better than them? If I already use Buck or Pants what benefit would I get if I move to Bazel?
- adgasf 7y ago
  Bazel is more popular, with a larger community and better language support than the other two. It seems that Bazel has "won".
wpdev_63 7y ago
It would great to see benchmarks against gradle. Gradle can become really slow in large projects.
tln 7y ago
> We will have a window of at least three months between major (breaking) releases That seems... short
- wffurr 7y ago
  Well you can keep using the LTS releases. I can't find offhand how long "long" is. Presumably much longer than three months...
ahupp 7y ago
I used buck at Facebook quite a bit, and it's my favorite build system. I assume most of the stuff I liked about buck translates to bazel: * Build rules were extremely simple to write * Sandboxing meant we could safely share a global cache of build artifacts, massively reducing build times * It runs a persistent server process that caches dependency and file change state, so incremental builds are fast * Buck had a lot of android-specific magic to make APK generation fast as well, not sure if that's in bazel. A lot of the stuff that's nice about these build systems only really becomes an issue on large codebases (thousands of engineers, tens of millions of LOC). The main weakness of buck (not sure about bazel) was handling 3rd party code; either you'd checkin prebuilts or port the build to buck.
gravypod 7y ago
Bazel, the tool, is fantastic. Haven't had a better build tool that I've seen yet. The build rules are awful. Almost every single rule is broken in some catastrophic way and, as is typical with Google open source, it is very difficult to ask for things to either be fixed or designed differently. Major examples: rules_python flat out does not work for python 3, rules_docker does not work without python 2, rules_proto cannot generate stubs for languages, you cannot specify third party deps at a package level and there is no explanation as to why (they will say it would destroy hermetic builds or reproducable builds but that doesn't make sense when you think about ways to support both).
- EdSchouten 7y ago
  This has improved lately. Rules_docker's image pusher has been replaced with a copy written in Go. This pusher is built using rules_go, so that it does not rely on any host tooling.
  - gravypod 7y ago
    I'll have to update my tag and try again. I wish it could just use my docker daemon which likely already had the image and layers.
scele 7y ago
One thing that I like about bazel is the way it integrates test and build actions to the same action graph and behind the same CLI. Tests are then also hermetic and sandboxed, and can be cached (or executed) remotely. For example, you could configure your CI to push test results to a remote cache, and when developers check out a clean master, ”bazel test //...” against that cache will report all tests as pass without running anything.

411 points by vips7L 7y ago

Because I have a twisted fascination with controversy and tribalism on HN, here's a little red meat: I'm going to stick with using make to glue together whatever tools I need for my build and test processes. Edit: -5 in under 10m. And yet I didn't even say anything remotely controversial or normative. Fantastic.

vore 7y ago
Maybe it's because your stated goal is baiting a flamewar in bad faith.
taneq 7y ago
It's probably because you came off as a bit unhinged and opened with a salvo against HN (which many people here understandably like). If you'd phrased it as "meh, I like make" you'd probably have half a dozen upvotes from the minimalists.
mrath 7y ago
For me the obstacle for using make is, it is not available for windows and even if I install via msys2 or cygwin, it is hard it make Makefiles independent of OS. I am not sure Bazel is the answer. BUILD files feel verbose and need more effort to maintain than platform specific files(go.mod, Cargo.toml, pom.xml).
- srgpqt 7y ago
  Visual Studio (used to?) come with nmake, a mostly compatible version of Make.
  - 1propionyl 7y ago
    <Microsoft Product> used to come with mostly compatible version of <Unix tool>. Yup.
    - flukus 7y ago
      I'm not sure if MS is to blame there, in my very brief experience it was compatible enough, it was the lack of unix tools and gnu extensions that made it difficult to work with. You'd probably get the same issue trying to be posix compatible.
- flukus 7y ago
  As long as you've got a basic unix like environment what's hard about being cross platform? Last time I tried it was very simple, at least with higher level languages and using gnu make specific features. For build tools I think "defining their own platform" tends to be a better description than being cross platform, the have there own languages, tools and idioms. In this case it's on top of the java platform as well.
dang 7y ago
Please don't troll here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

krapht 7y ago

Is it worth giving up language-specific build tools (CMake, Maven, Gradle and others) for one build tool to rule them all?

jrockway 7y ago
My experience with using the Google-internal version of Bazel is that yes, you want to give up all other tools to use this. Having a proper dependency graph makes it possible to cache maximally and accurately, and that results in fast builds. You don't need it if you only have like one go program that you're building, but you will start to see the disadvantages when you have a handful of things you build. Did you build all the apps that depend on the proto you updated? Do you need to build your PHP app because you change an internal go library? Tools like "docker build" have no idea and tend to rebuild too much or too little, and yield different results on different machines. This causes a lot of headaches. The thing that kills me about Bazel is that it is quite painful to deal with Java. I never seem to get a working version on a Linux distribution. On Windows it's always sitting down in the taskbar showing ads. Oracle calls you to demand money. You check the bounds of an array and they sue you. It's just not worth it. So I don't actually use Bazel.
- klodolph 7y ago
  On Linux I just install a headless OpenJDK 1.8. Something similar is available through brew on the Mac. Oracle’s version is dead as far as I care.
  - pjmlp 7y ago
    Who do you think writes 90% of OpenJDK and drives language design?
- roneythomas6 7y ago
  I use Amazon Corretto. There are few other openJDK distributions.
- vips7L 7y ago

taneq 7y ago

Problem: There are N confusing build systems with arcane rules and you just want to compile your program. Solution: Design a simple, straightforward build tool. Problem: There are N+1 confusing build systems with arcane rules.

new_realist 7y ago
I wouldn’t call Bazel confusing or arcane, but it is powerful. I find it rather delightful to use.
- mook 7y ago
  As a relatively newcomer to bazel, I did find it arcane: - I couldn't find an easy way to list targets - in particular, a way to show me the path to the build artifacts (so that I could inspect it in my editor) - I couldn't find a way to do build configurations (especially intersections of build configurations) - it didn't seem great at handling non-file build steps (set up a cluster / server / whatever so I can run my tests) - the documentation was slow to load and hard to find things in (you basically had to use search, instead of browsing a good table of contents) As I said, though, I'm a newcomer; it's quite possible there are good ways to do those things, I just hadn't managed to find it in the documenting yet. That and the issues are by no means unique to bazel. I also do like some aspects of it (like enforced out-of-tree builds), but that's off-topic ;)
  - joshuamorton 7y ago
    Configs: look into select() and modes. Non-file steps: normally in bazel you would consider the server setup a part of the test. So `blaze build :my_test` creates an artifact, and blaze run or blaze test runs the artifact which does non-hermetic stuff like setting up a server.
  - emidln 7y ago
    Listing targets in a package: bazel query //some/package:* Listing targets in a package including subpackages: bazel query //some/package/... These and more are covered in the documentation on querying: https://docs.bazel.build/versions/1.0.0/query.html#target-patterns https://docs.bazel.build/versions/1.0.0/query.html#target-pa...

RcouF1uZ4gsC 7y ago

I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.

RhodesianHunter 7y ago
Why do you care if a JVM is on your dev machine?
- thejosh 7y ago
  - One more thing to install - Does this work for 8/11/12/13/14/15/16?
  - cromwellian 7y ago
    It's a stripped down JVM is bundled with bazel, you don't need to install it.
- weberc2 7y ago
  My 2015 MBP is crawling between Safari, Docker, VS Code, and BitDefender (antivirus). Not sure how bulky the JVM is these days. That said, I wish it supported Python.
  - harrygeez 7y ago
    Why do you even have Bitdefender installed?
    - the_mitsuhiko 7y ago
      A lot of people have it for compliance reasons.
      - slrz 7y ago

lenkite 7y ago

Whats the advantage of using Bazel over CMake ? What does one do about package management ?

sayrer 7y ago
For packages, you download things by specifying an archive (zip file, git repo, etc), and a SHA256 hash. Think of it as extremely-precise version pinning. You can also refer to other Bazel projects and use their internal rules if permitted. The biggest advantages of Bazel over something like CMake are: 1) It is very general, so you can easily express dependencies of this sort: - concat three text files, then - run them through a parser generator, then - use the generated parser in a C library, then - use that C library in a Go server library and a Rust client library, then - package the Go server into a Docker container and build an iOS binary that uses the Rust client This is a cross-language dependency chain that also runs a few Unix tools. In this example, if you edit the Rust client, only the Rust library and the iOS client will be rebuilt. If you edit one of the text files that compose the grammar, everything will get rebuilt. The dependencies are also a DAG, so if your projects have a shared Go library and you change that, the Go server would get rebuilt, as would the Docker container. While all this might seem distressingly general, it removes the need for things like "bootstrap.sh" files, "build.rs" files, and all of the unstated dependencies they entail. 2) If you download all of the toolchains (C++, Java, Rust, Go, etc), it's impossible to have "works on my machine" problems. Exceptions are use of "local_*" rules, and Apple binaries (you need machine-local SDKs). 3) This allows great caching. You can run the equivalent of ccache for all of this stuff, locally or shared on a LAN. RBE is even more advanced.
- fanf2 7y ago
  (1) sounds like table stakes for any build system - `make` has done all that for decades
  - sayrer 7y ago
    Well, sort of. The way make tracks changes triggers more rebuilds and the build products are much less reusable. There's a good comparison table at the bottom of section 2.4 in https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems-final.pdf https://www.microsoft.com/en-us/research/uploads/prod/2018/0... Bazel's dependency graph tends to be much more detailed than Makefiles, extends all the way to toolchains at the base, and all the way up to build products and test results. It can also be composed together across huge file layouts in ways make doesn't support.

lngnmn1 7y ago

Those who wrote make (or mk of Plan9) were way, way smarter and much humbler. No, we don't have to invent an idiotic DSL where declarative approach and sane defaults will do.

jdkee 7y ago

What does Bazel do exactly?

mc3 7y ago
Bazel is an open-source build and test tool similar to Make, Maven, and Gradle. It uses a human-readable, high-level build language. Bazel supports projects in multiple languages and builds outputs for multiple platforms. Bazel supports large codebases across multiple repositories, and large numbers of users.
- kajecounterhack 7y ago
  It also builds code repeatably & hermetically (interpreted languages excepted; at google python is built super hermetically where a python binary + environment is bundled with python script(s) into an executable; some companies like dropbox have copied this). You run build, and you get binaries for N kinds of systems (e.g x86_64, ARM). It builds a dependency graph of your code that you can query and results in cache-able builds (only rebuild the part you need). Since building dependencies in a tree can be massively parallelized, it also allows you to parallelize your builds well across machines (with some effort). You use the one tool to compile C++, Java, Python, Go, Rust, etc.

pcj-github 7y ago

For me, the difference between other build systems and bazel was like the shift from CVS to git... Profoundly better internal data model, but definitely a learning curve.

spuz 7y ago
That makes sense but there aren't many people out there using CVS right now. What were you using before Bazel? I.e. what are you comparing Bazel to?

dwrodri 7y ago

My only experience with Bazel is using it for Tensorflow (TF). It seemed very roundabout for what was required and definitely had some painful version issues as TF got closer to 2.0. Hopefully this means more stability and ease of use going forward for TF users.

person_of_color 7y ago

Learn CMake.

throwaway1777 7y ago

In my opinion Bazel/Blaze and Buck are mainly useful to solve the problem of having too much code to build on one machine. Reproducible builds which can be done in a distributed fashion aren’t really useful when you can build your whole project in a minute or two on one machine. That being said, once you get the hang of it the overhead is not too high compared to simpler systems.

a_t48 7y ago
None of the last three companies I worked at could build in under a minute, sadly.
indolering 7y ago
There are security reasons to want a reproducible build.
kartayyar 7y ago
Xoogler here who used the internal version of Bazel ( blaze ) extensively, liked it and now opted to use it for our 2 person startup ( at least for the backend ). You don't need a giant codebase to start benefitting from it. Even if you have a smaller codebase, and properly use bazel to setup tests, you only test what needs testing, and that makes a huge difference in productivity.
- MapleWalnut 7y ago
  What languages are you using with Bazel? I have a project that uses Typescript and Python. Are you using Bazel with interpreted languages?
  - evmar 7y ago
    Bazel has experimental but pretty sophisticated support for TypeScript, with arguably a better compilation experience for large projects than the best known alternatives. (Disclaimer: I'm biased because I work on it, but I also work on non-bazel TypeScript projects so I am aware of the tradeoffs.) https://bazelbuild.github.io/rules_nodejs/TypeScript.html https://bazelbuild.github.io/rules_nodejs/TypeScript.html

slavik81 7y ago

The lack of backwards compatibility between Bazel versions has been the deal-killer for me thus far. Old build instructions should work with newer releases of the same tools. It's important for the tools to remain compatible because you can't change the instructions contained in old releases. I hope the release of 1.0 is an indication that Bazel is taking interface stability more seriously.

ageofwant 7y ago
Right. Every time I build tensorflow I have to build bazel first, or get the bazel version for the tf version. I have to say, having used a number of Google's tools over the years: basel, coral sdk, maps api and perhaps others, I'm always surprised at how 'how ya doing' the experience is. Don't get me wrong, I appreciate their tools and its openness, but I wish Google spend a bit more time doing software engineering, and less throwing shit against a wall ascertaining stickiness.
- philwo 7y ago
  > Every time I build tensorflow I have to build bazel first, or get the bazel version for the tf version. This is fixed now: https://github.com/tensorflow/tensorflow/commit/991aec351b574c4ec9157b9e557fb0f7f53bc4d9 https://github.com/tensorflow/tensorflow/commit/991aec351b57... If you use Bazelisk (https://github.com/bazelbuild/bazelisk https://github.com/bazelbuild/bazelisk), you'll automatically get the matching Bazel version for TensorFlow in the future (and all other projects that have adopted the .bazelversion file).
  - slavik81 7y ago
    Awesome. Bazelisk provides exactly the sort of compatibility I was hoping to see. Though, manually specifying a maximum Bazel version doesn't seem like it should be necessary. The authors of Bazel know what features they have changed since the version specified as the minimum, and Bazel itself knows what features the BUILD file uses. That information together could be used to calculate the maximum version.
7y ago

bazza451 7y ago

Does anyone know of a real beginners level guide to Bazel that isn’t the docs. Got a typescript monorepo with a few backend services/react frontends I wanted to build and I got a bit lost in the complexity of it due to package.json handling

G2H 7y ago
We wrote a bit about how we build a TS monorepo with Bazel on our company's blog [1] if that can be helpful [1] https://dataform.co/blog/typescript-monorepo-with-bazel/ https://dataform.co/blog/typescript-monorepo-with-bazel/
- bazza451 7y ago
  thanks will check it out

choppaface 7y ago

One of the major time savers of bazel is Remote Build Execution (RBE), which allows you to build modules in parallel in the cloud. So if you have 1000 CPUs, you can really just have a client do `bazel build -j 1000 //...` and you can get a huge speed-up. Remote (and local) builds all happen in a sandbox, so you don't have to worry about e.g. preparing a docker image with the worker / build slave environment. (You do, however, have to register your dependencies with Bazel, which can be a hard at first). To add to this, bazel also has a remote global cache which can benefit large teams. For fairly large C++ codebases, RBE is really a competitive advantage. I've seen RBE cut down iteration time by an order of magnitude. I love CMake, and CMake can get you plenty of parallelism, but CMake doesn't really provide a tool for building several CMake sub-projects in parallel, and bazel handles this really well. Sadly Bazel RBE is still primarily a Google Cloud product. Also, GCE is slow to work on supporting auto-scale, so you have to pay for unused workers. (Like most products in Google Cloud, Google is ripping you off with alpha-quality stuff). There's some very rough open source RBE stuff on Github that you can run yourself, but nothing really production-grade yet. gg ( https://github.com/StanfordSNR/gg https://github.com/StanfordSNR/gg ) is a promising-looking alternative. It's research code, but it might be the community's best hope for a non-Google alternative (that e.g. supports AWS Lambda for parallelism). Bazel is great, but without independence (e.g. what Kubernetes achieved) it's difficult to see bazel as dependable as make or CMake long term.

9nGQluzmnq3M 7y ago
> GCE is slow to work on supporting auto-scale, so you have to pay for unused workers. Can you expand on what you mean here? GCE has had autoscaling for years and it's quite configurable, I don't see why you would need to keep unused workers around unless your build load is extremely spiky.
- choppaface 7y ago
  Not auto-scaling VM compute instances, auto-scaling the Google Cloud RBE service (I think it's called Cloud Build right now?). If you have a worker pool of size 1,000, you have to pay for that even if your dev team only uses it at full capacity 8-10hrs per day. Or maybe the Google Cloud sales people working with my company are misinformed? IDK, but my experience with TPUs, GPUs, AI Engine, Dataproc, BigQuery, etc... every GCloud feature I've used (besides vanilla Compute VMs) have gotchas and/or time sinks that are costly unless you have a load of free credits. For AI Engine, it took like 6 months to upgrade their internal Kubernetes cluster; in the meantime, crashed jobs would burn an extra 15 minutes of GPU dollars. There's a lot of good stuff in Google Cloud, but be sure to get Google to pay you to wade through the construction with free credits.

shusson 7y ago

Anyone had any experience using Bazel with Angular projects?

panpanna 7y ago

Looking forward to seeing this as a requirement in projects that already require npm, phyton, cmake, gradle and ninja to compile :(

human_error 7y ago

How is it different than Buck or Pants? What makes Bazel better than them? If I already use Buck or Pants what benefit would I get if I move to Bazel?

adgasf 7y ago
Bazel is more popular, with a larger community and better language support than the other two. It seems that Bazel has "won".

wpdev_63 7y ago

It would great to see benchmarks against gradle. Gradle can become really slow in large projects.

tln 7y ago

> We will have a window of at least three months between major (breaking) releases That seems... short

wffurr 7y ago
Well you can keep using the LTS releases. I can't find offhand how long "long" is. Presumably much longer than three months...

ahupp 7y ago

I used buck at Facebook quite a bit, and it's my favorite build system. I assume most of the stuff I liked about buck translates to bazel: * Build rules were extremely simple to write * Sandboxing meant we could safely share a global cache of build artifacts, massively reducing build times * It runs a persistent server process that caches dependency and file change state, so incremental builds are fast * Buck had a lot of android-specific magic to make APK generation fast as well, not sure if that's in bazel. A lot of the stuff that's nice about these build systems only really becomes an issue on large codebases (thousands of engineers, tens of millions of LOC). The main weakness of buck (not sure about bazel) was handling 3rd party code; either you'd checkin prebuilts or port the build to buck.

gravypod 7y ago

Bazel, the tool, is fantastic. Haven't had a better build tool that I've seen yet. The build rules are awful. Almost every single rule is broken in some catastrophic way and, as is typical with Google open source, it is very difficult to ask for things to either be fixed or designed differently. Major examples: rules_python flat out does not work for python 3, rules_docker does not work without python 2, rules_proto cannot generate stubs for languages, you cannot specify third party deps at a package level and there is no explanation as to why (they will say it would destroy hermetic builds or reproducable builds but that doesn't make sense when you think about ways to support both).

EdSchouten 7y ago
This has improved lately. Rules_docker's image pusher has been replaced with a copy written in Go. This pusher is built using rules_go, so that it does not rely on any host tooling.
- gravypod 7y ago
  I'll have to update my tag and try again. I wish it could just use my docker daemon which likely already had the image and layers.

scele 7y ago

One thing that I like about bazel is the way it integrates test and build actions to the same action graph and behind the same CLI. Tests are then also hermetic and sandboxed, and can be cached (or executed) remotely. For example, you could configure your CI to push test results to a remote cache, and when developers check out a clean master, ”bazel test //...” against that cache will report all tests as pass without running anything.