4 ms·

Is it worth giving up language-specific build tools (CMake, Maven, Gradle and others) for one build tool to rule them all?

by krapht 7y ago

jrockway 7y ago
My experience with using the Google-internal version of Bazel is that yes, you want to give up all other tools to use this. Having a proper dependency graph makes it possible to cache maximally and accurately, and that results in fast builds. You don't need it if you only have like one go program that you're building, but you will start to see the disadvantages when you have a handful of things you build. Did you build all the apps that depend on the proto you updated? Do you need to build your PHP app because you change an internal go library? Tools like "docker build" have no idea and tend to rebuild too much or too little, and yield different results on different machines. This causes a lot of headaches. The thing that kills me about Bazel is that it is quite painful to deal with Java. I never seem to get a working version on a Linux distribution. On Windows it's always sitting down in the taskbar showing ads. Oracle calls you to demand money. You check the bounds of an array and they sue you. It's just not worth it. So I don't actually use Bazel.
- klodolph 7y ago
  On Linux I just install a headless OpenJDK 1.8. Something similar is available through brew on the Mac. Oracle’s version is dead as far as I care.
  - pjmlp 7y ago
    Who do you think writes 90% of OpenJDK and drives language design?
- roneythomas6 7y ago
  I use Amazon Corretto. There are few other openJDK distributions.
- vips7L 7y ago
  > I never seem to get a working version on a Linux distribution. On Windows it's always sitting down in the taskbar showing ads. I've never experienced any of this. Maybe because I use OpenJDK?
- roseandking 7y ago
  Dude, IDK if you have used bazel OUTSIDE of Google, but I've used it both inside and outside of Google and I can tell with 99.999% confidence that the experiences are dramatically different. Bazel != Blaze (internal version of Bazel). 1. Google has a well maintained monorepo. Most companies don't. That diminishes the meaning of a good build system in the very first place; even if Bazel is powerful, with separate repos it's power isn't the shiniest. 2. Google open sources a part of Blaze, which is the external Bazel, but not all to it. That's what Google does with basically everything it open-sources. The outcome then, is that the external tool require you to build a lot of other not open-sourced tools to replicate the excellence of the entire Google internal eco-system that makes the internal tool so amazing. Same goes with Bazel. It works amazingly internally because there are so many people supporting it, and so many other tools work nicely with it. But not all the features and tools are open-sourced together with Bazel, so the diff of experiences internally VS externally is significant. TL, DR: if you're a small company without a good monorepo strategy and someone experienced on this, DON'T EXPECT BAZEL TO BE YOUR MAGIC CURE. Disclaimer: my current company uses Bazel pre-1.0 and it's definitely dramatically worse than what I used internally at Google. I haven't tried 1.0 but I don't expect the ecosystem problem to be solved in 1.0 anyways. Also Bazel sort of sucks for python anyways. Really hoping someone can prove me wrong and say 1.0 is actually the lit shit.
  - plicense 7y ago
    Can you quantify what you mean by worse? What makes you feel that way?
- pjmlp 7y ago
  How you have an ads taskbar Java app. Doing Java since it was introduced in 1996 and never got to see Java ads on the taskbar, must be a special version.
- loeg 7y ago
  > Having a proper dependency graph makes it possible to cache maximally and accurately, and that results in fast builds. Pretty much every other build system (e.g., make) is rooted in having a proper dependency graph. This is not a unique property of Bazel.
  - jingwen 7y ago
    That's right. However, there are distinct implications on how this dependency graph is constructed, analyzed, and evaluated. "Build Systems à la Carte" (Mokhov, Mitchell, Peyton Jones) [1] goes deeper into formalizing these differences. [1] https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems-final.pdf https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
  - rienbdj 7y ago
    Make (to use your example) does not do any sandboxing to ensure that the declared dependency graph is actually correct. This is the big innovation in Bazel (and Co).
    - emelski 7y ago
      Electric Make has had sandboxing since 2002, no conversion from your familiar make-based builds to a new shiny build tool required, and it can make on-the-fly corrections to execution order if that sandboxing reveals that incomplete dependency specifications caused something to run in the wrong order (relative to a strictly serial build). "Sandboxing" in a build tool cannot be claimed as Bazel's innovation.
      - adgasf 7y ago
        Can you link to the project?
        jburgess777 7y ago
        He was probably talking about a product from Electric Cloud: https://electric-cloud.com/plugins/directory/p/emake/ https://electric-cloud.com/plugins/directory/p/emake/
klodolph 7y ago
Long-run, yes. Projects are too often multi-language these days, and if you mix C++ and Java, there might not be a clear or convenient seam between the C++ and Java parts. Short-term, it will take a while after 1.0 for the surrounding tools and libraries to stabilize. It will take a little while for better documentation, blog posts, etc. Personally, I find it a huge step up from e.g. CMake for C or C++ projects. But I’ve been using it for a while.
kccqzy 7y ago
Depends on your project. My experience is that if you use a single language throughout, then your language-specific build tool probably suffices. But once you go polyglot, interactions between these language-specific build tools are bad and you'd be better off replacing all of them with one tool that understands everything. Especially if you have cross-language dependencies (e.g. your Go program depends on C++ using cgo, or your Python has a module written in C). Whether or not you want that tool to be Bazel is your choice.

Is it worth giving up language-specific build tools (CMake, Maven, Gradle and others) for one build tool to rule them all?

by krapht 7y ago

Is it worth giving up language-specific build tools (CMake, Maven, Gradle and others) for one build tool to rule them all?

jrockway 7y ago

My experience with using the Google-internal version of Bazel is that yes, you want to give up all other tools to use this. Having a proper dependency graph makes it possible to cache maximally and accurately, and that results in fast builds. You don't need it if you only have like one go program that you're building, but you will start to see the disadvantages when you have a handful of things you build. Did you build all the apps that depend on the proto you updated? Do you need to build your PHP app because you change an internal go library? Tools like "docker build" have no idea and tend to rebuild too much or too little, and yield different results on different machines. This causes a lot of headaches. The thing that kills me about Bazel is that it is quite painful to deal with Java. I never seem to get a working version on a Linux distribution. On Windows it's always sitting down in the taskbar showing ads. Oracle calls you to demand money. You check the bounds of an array and they sue you. It's just not worth it. So I don't actually use Bazel.

klodolph 7y ago
On Linux I just install a headless OpenJDK 1.8. Something similar is available through brew on the Mac. Oracle’s version is dead as far as I care.
- pjmlp 7y ago
  Who do you think writes 90% of OpenJDK and drives language design?
roneythomas6 7y ago
I use Amazon Corretto. There are few other openJDK distributions.
vips7L 7y ago
> I never seem to get a working version on a Linux distribution. On Windows it's always sitting down in the taskbar showing ads. I've never experienced any of this. Maybe because I use OpenJDK?
roseandking 7y ago
Dude, IDK if you have used bazel OUTSIDE of Google, but I've used it both inside and outside of Google and I can tell with 99.999% confidence that the experiences are dramatically different. Bazel != Blaze (internal version of Bazel). 1. Google has a well maintained monorepo. Most companies don't. That diminishes the meaning of a good build system in the very first place; even if Bazel is powerful, with separate repos it's power isn't the shiniest. 2. Google open sources a part of Blaze, which is the external Bazel, but not all to it. That's what Google does with basically everything it open-sources. The outcome then, is that the external tool require you to build a lot of other not open-sourced tools to replicate the excellence of the entire Google internal eco-system that makes the internal tool so amazing. Same goes with Bazel. It works amazingly internally because there are so many people supporting it, and so many other tools work nicely with it. But not all the features and tools are open-sourced together with Bazel, so the diff of experiences internally VS externally is significant. TL, DR: if you're a small company without a good monorepo strategy and someone experienced on this, DON'T EXPECT BAZEL TO BE YOUR MAGIC CURE. Disclaimer: my current company uses Bazel pre-1.0 and it's definitely dramatically worse than what I used internally at Google. I haven't tried 1.0 but I don't expect the ecosystem problem to be solved in 1.0 anyways. Also Bazel sort of sucks for python anyways. Really hoping someone can prove me wrong and say 1.0 is actually the lit shit.
- plicense 7y ago
  Can you quantify what you mean by worse? What makes you feel that way?
pjmlp 7y ago
How you have an ads taskbar Java app. Doing Java since it was introduced in 1996 and never got to see Java ads on the taskbar, must be a special version.
loeg 7y ago
> Having a proper dependency graph makes it possible to cache maximally and accurately, and that results in fast builds. Pretty much every other build system (e.g., make) is rooted in having a proper dependency graph. This is not a unique property of Bazel.
- jingwen 7y ago
  That's right. However, there are distinct implications on how this dependency graph is constructed, analyzed, and evaluated. "Build Systems à la Carte" (Mokhov, Mitchell, Peyton Jones) [1] goes deeper into formalizing these differences. [1] https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems-final.pdf https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
- rienbdj 7y ago
  Make (to use your example) does not do any sandboxing to ensure that the declared dependency graph is actually correct. This is the big innovation in Bazel (and Co).
  - emelski 7y ago
    Electric Make has had sandboxing since 2002, no conversion from your familiar make-based builds to a new shiny build tool required, and it can make on-the-fly corrections to execution order if that sandboxing reveals that incomplete dependency specifications caused something to run in the wrong order (relative to a strictly serial build). "Sandboxing" in a build tool cannot be claimed as Bazel's innovation.
    - adgasf 7y ago
      Can you link to the project?
      - jburgess777 7y ago
        He was probably talking about a product from Electric Cloud: https://electric-cloud.com/plugins/directory/p/emake/ https://electric-cloud.com/plugins/directory/p/emake/

klodolph 7y ago

Long-run, yes. Projects are too often multi-language these days, and if you mix C++ and Java, there might not be a clear or convenient seam between the C++ and Java parts. Short-term, it will take a while after 1.0 for the surrounding tools and libraries to stabilize. It will take a little while for better documentation, blog posts, etc. Personally, I find it a huge step up from e.g. CMake for C or C++ projects. But I’ve been using it for a while.

kccqzy 7y ago

Depends on your project. My experience is that if you use a single language throughout, then your language-specific build tool probably suffices. But once you go polyglot, interactions between these language-specific build tools are bad and you'd be better off replacing all of them with one tool that understands everything. Especially if you have cross-language dependencies (e.g. your Go program depends on C++ using cgo, or your Python has a module written in C). Whether or not you want that tool to be Bazel is your choice.