4 ms·

What do you dislike about bazel? Not championing it, just curious.

adev_ 8y ago
The list would be too long but out of order : - Using a JVM to compile mainly C++ and python, implying to deploy a JDK/JRE just for that. - Awful RAM consumption, compiling tensorflow takes 16GB of RAM to create a python wheel - Try to download the world and compile everything without allowing to specify external dependencies. - Compile time that are crazy long due to previous reason. - Invasive. Make very hard to integrate with anything else that does not build with Bazel - Make very hard, or sometimes, impossible to tune compiler flags - Just not reproducible. - Unstable, try to build a recent Google package with a 6 month old Bazel and good luck. I strongly advise you to watch this video of the last FOSDEM : https://archive.fosdem.org/2018/schedule/event/how_to_make_package_managers_cry/ https://archive.fosdem.org/2018/schedule/event/how_to_make_p...
- haberman 8y ago
  I strongly agree with you on the first one. Many of your other ones are byproducts of the fact that Bazel is primarily a build-from-source system. This has some benefits, particularly in a C++ ecosystem where binary compatibility across versions basically doesn't exist. But it also has some big drawbacks when it comes to compile times. I do see Bazel seems to support depending on a prebuilt .so, though I have not tried this: https://docs.bazel.build/versions/master/cpp-use-cases.html#adding-dependencies-on-precompiled-libraries https://docs.bazel.build/versions/master/cpp-use-cases.html#... > Invasive. Make very hard to integrate with anything else that does not build with Bazel I think your main options here are: 1. prebuild the other projects, then depend on the .so from Bazel (https://docs.bazel.build/versions/master/cpp-use-cases.html#adding-dependencies-on-precompiled-libraries https://docs.bazel.build/versions/master/cpp-use-cases.html#...) 2. write a BUILD file for the external project. Here is an example of a project that builds a bunch of non-Bazel deps by writing Bazel BUILD files for each of them: https://github.com/googlecartographer/cartographer/tree/master/bazel/third_party https://github.com/googlecartographer/cartographer/tree/mast... > Make very hard, or sometimes, impossible to tune compiler flags "bazel --copt=<your options here> :target"? > Just not reproducible. What isn't reproducible? I tend to think of reproducibility as a strength of Bazel. Because all of your dependencies are explicit and Bazel fetches a known version, the build is less dependent on your system environment and whatever you happen to have installed there. Disclosure: I am a Googler. I have some gripes with Bazel, but overall I think it gets some important ideas right. You have a BUILD file that is declarative, then any imperative code you need goes into separate .bzl files to define the rules you need.
  - adev_ 8y ago
    > Many of your other ones are byproducts of the fact that Bazel is primarily a build-from-source system. This has some benefits, particularly in a C++ ecosystem where binary compatibility across versions basically doesn't exist. But it also has some big drawbacks when it comes to compile times. Nix, Guix and Spack packager managers solved the C++ ABI issue a long time ago already without the crazy needs of Bazel in term of resource consumption, integration and compile time. They even supports binary distributions for some of them. > I think your main options here are: > 1. prebuild the other projects, then depend on the .so from Bazel (https://docs.bazel.build/versions/master/cpp-use-cases.html#... https://docs.bazel.build/versions/master/cpp-use-cases.html#...) > 2. write a BUILD file for the external project. Here is an example of a project that builds a bunch of non-Bazel deps by writing Bazel BUILD files for each of them: https://github.com/googlecartographer/cartographer/tree/mast.. https://github.com/googlecartographer/cartographer/tree/mast.... I know that. But all of them are terrible options. I do not want to depend of SQLite, OpenSSL, libxml or whatever other system library compiled by Bazel, nor I want Bazel to takes 45 minutes to recompile them. Additionally, this will cause diamond dependency problem with other softwares that use Bazel artefact without compiling with Bazel. > "bazel --copt=<your options here> :target"? Can I use that to specify a flag to some target and not some other ? Without having to build sequentially each of them ? Concrete example of the Maddness: SQLITE will not compile if you enable some options that would make tensorflow faster.... Bazel recursively compile both. > What isn't reproducible? I tend to think of reproducibility as a strength of Bazel. Because all of your dependencies are explicit and Bazel fetches a known version, the build is less dependent on your system environment and whatever you happen to have installed there. Bazel try to build in isolated environment but do half of the job. It still depends on system compiler, and do not chroot nor "compiler-wrap" ( c.f Spack ) making the build still very vulnerable of system side effect and update. > Disclosure: I am a Googler. I have some gripes with Bazel, but overall I think it gets some important ideas right. You have a BUILD file that is declarative, then any imperative code you need goes into separate .bzl files to define the rules you need. I can understand that Bazel is very convenient in Google environment with Google resources. But it's a nightmare for everyone I talked to outside of Google.
    - haberman 8y ago
Q6T46nT668w6i3m 8y ago
I’m not the OP (but I share their annoyance), I use a dozen of build systems but TensorFlow is the only project I use that uses Bazel.
- solomatov 8y ago
  Having experience with bazel, I wouldn't say that tensorflow is the best possible use of bazel. You can use of it in other project, for example, in the bazel itself: https://github.com/bazelbuild/bazel https://github.com/bazelbuild/bazel

The list would be too long but out of order : - Using a JVM to compile mainly C++ and python, implying to deploy a JDK/JRE just for that. - Awful RAM consumption, compiling tensorflow takes 16GB of RAM to create a python wheel - Try to download the world and compile everything without allowing to specify external dependencies. - Compile time that are crazy long due to previous reason. - Invasive. Make very hard to integrate with anything else that does not build with Bazel - Make very hard, or sometimes, impossible to tune compiler flags - Just not reproducible. - Unstable, try to build a recent Google package with a 6 month old Bazel and good luck. I strongly advise you to watch this video of the last FOSDEM : https://archive.fosdem.org/2018/schedule/event/how_to_make_package_managers_cry/ https://archive.fosdem.org/2018/schedule/event/how_to_make_p...

haberman 8y ago
I strongly agree with you on the first one. Many of your other ones are byproducts of the fact that Bazel is primarily a build-from-source system. This has some benefits, particularly in a C++ ecosystem where binary compatibility across versions basically doesn't exist. But it also has some big drawbacks when it comes to compile times. I do see Bazel seems to support depending on a prebuilt .so, though I have not tried this: https://docs.bazel.build/versions/master/cpp-use-cases.html#adding-dependencies-on-precompiled-libraries https://docs.bazel.build/versions/master/cpp-use-cases.html#... > Invasive. Make very hard to integrate with anything else that does not build with Bazel I think your main options here are: 1. prebuild the other projects, then depend on the .so from Bazel (https://docs.bazel.build/versions/master/cpp-use-cases.html#adding-dependencies-on-precompiled-libraries https://docs.bazel.build/versions/master/cpp-use-cases.html#...) 2. write a BUILD file for the external project. Here is an example of a project that builds a bunch of non-Bazel deps by writing Bazel BUILD files for each of them: https://github.com/googlecartographer/cartographer/tree/master/bazel/third_party https://github.com/googlecartographer/cartographer/tree/mast... > Make very hard, or sometimes, impossible to tune compiler flags "bazel --copt=<your options here> :target"? > Just not reproducible. What isn't reproducible? I tend to think of reproducibility as a strength of Bazel. Because all of your dependencies are explicit and Bazel fetches a known version, the build is less dependent on your system environment and whatever you happen to have installed there. Disclosure: I am a Googler. I have some gripes with Bazel, but overall I think it gets some important ideas right. You have a BUILD file that is declarative, then any imperative code you need goes into separate .bzl files to define the rules you need.
- adev_ 8y ago
  > Many of your other ones are byproducts of the fact that Bazel is primarily a build-from-source system. This has some benefits, particularly in a C++ ecosystem where binary compatibility across versions basically doesn't exist. But it also has some big drawbacks when it comes to compile times. Nix, Guix and Spack packager managers solved the C++ ABI issue a long time ago already without the crazy needs of Bazel in term of resource consumption, integration and compile time. They even supports binary distributions for some of them. > I think your main options here are: > 1. prebuild the other projects, then depend on the .so from Bazel (https://docs.bazel.build/versions/master/cpp-use-cases.html#... https://docs.bazel.build/versions/master/cpp-use-cases.html#...) > 2. write a BUILD file for the external project. Here is an example of a project that builds a bunch of non-Bazel deps by writing Bazel BUILD files for each of them: https://github.com/googlecartographer/cartographer/tree/mast.. https://github.com/googlecartographer/cartographer/tree/mast.... I know that. But all of them are terrible options. I do not want to depend of SQLite, OpenSSL, libxml or whatever other system library compiled by Bazel, nor I want Bazel to takes 45 minutes to recompile them. Additionally, this will cause diamond dependency problem with other softwares that use Bazel artefact without compiling with Bazel. > "bazel --copt=<your options here> :target"? Can I use that to specify a flag to some target and not some other ? Without having to build sequentially each of them ? Concrete example of the Maddness: SQLITE will not compile if you enable some options that would make tensorflow faster.... Bazel recursively compile both. > What isn't reproducible? I tend to think of reproducibility as a strength of Bazel. Because all of your dependencies are explicit and Bazel fetches a known version, the build is less dependent on your system environment and whatever you happen to have installed there. Bazel try to build in isolated environment but do half of the job. It still depends on system compiler, and do not chroot nor "compiler-wrap" ( c.f Spack ) making the build still very vulnerable of system side effect and update. > Disclosure: I am a Googler. I have some gripes with Bazel, but overall I think it gets some important ideas right. You have a BUILD file that is declarative, then any imperative code you need goes into separate .bzl files to define the rules you need. I can understand that Bazel is very convenient in Google environment with Google resources. But it's a nightmare for everyone I talked to outside of Google.
  - haberman 8y ago

Q6T46nT668w6i3m 8y ago

I’m not the OP (but I share their annoyance), I use a dozen of build systems but TensorFlow is the only project I use that uses Bazel.

solomatov 8y ago
Having experience with bazel, I wouldn't say that tensorflow is the best possible use of bazel. You can use of it in other project, for example, in the bazel itself: https://github.com/bazelbuild/bazel https://github.com/bazelbuild/bazel