5 ms·

> We are talking seasoned build engineers ending up frustrated after literally months of trying to achieve something that is easy as pie in CMake. But it's not

by q3k 6y ago

> We are talking seasoned build engineers ending up frustrated after literally months of trying to achieve something that is easy as pie in CMake. But it's not the same as CMake. A properly set up Bazel project gives you so much more: organization-wide incremental builds, build cache and build farm. Also, actual full hermeticity of builds (no, taking ambient deps from a Docker container doesn't replace that). Comparing CMake to Bazel is like comparing some barely-working bash scripts on a single box to a kubernetes deployment. Maybe you're okay with just bash scripts, but some of us aren't, and that's where Bazel comes in.

simfoo 6y ago
Completely agree that you can't compare Bazel and CMake directly, they are not on the same level. But for compiling C++ Bazel will have to compare itself to CMake, which is plain and simple the defacto standard. All the complex things that Bazel brings into the game (distributed builds and caches primarily) simply don't have any value when the main task that you're trying to make faster is not done well. I would much rather like to see more tooling that does these things but builds upon layers of battle-tested existing software.
keithwinstein 6y ago
I've been confused for a while about the common claim that Bazel gives "full hermeticity" of builds -- it doesn't seem to be true in practice (at least for packages with system dependencies). Maybe you can help me clear it up. E.g. Google's protobuf libraries [1] can be built with Bazel, and they depend heavily on system headers outside the repository, e.g. <iostream> and <stdio.h> and lots more. If those headers subsequently change, Bazel will not pick up on this and will not know to rebuild the parts of the build that depend on them. To reproduce: run `bazel build -c opt //:protoc_lib` and then put random garbage in your /usr/include/stdio.h and /usr/include/c++/<version>/iostream and then rerun the bazel command -- it will not know to invalidate the build cache. If you `bazel clean` and then build again, you'll get different results. Bazel does a lot of really nice things and I can believe that within a google3-like environment (where the source code never references a system header?), it effectively provides hermetic builds, but in practice as used outside Google (or even in Google's public OSS releases) it doesn't seem to really match this description or enforce a hermetic seal. What am I missing? [1] https://github.com/protocolbuffers/protobuf https://github.com/protocolbuffers/protobuf
- kornholi 6y ago
  I wish Bazel was more explicit about things leaking in from outside the workspace, but you can always vendor the toolchain to be more hermetic. For example see https://github.com/grailbio/bazel-toolchain https://github.com/grailbio/bazel-toolchain which sets up LLVM and Clang for you. https://github.com/bazelbuild/bazel/issues/4558 https://github.com/bazelbuild/bazel/issues/4558
  - keithwinstein 6y ago
    Ok, but that decision would have to be made by the project maintainer, in this case Google, not the person using Bazel to compile protobuf. (And not particular to Bazel -- a developer can make any build system effectively hermetic by vendoring everything.) In my view the challenge here is that a dependency changes (e.g. /usr/include/stdio.h is upgraded by the system package manager, or two users sharing a cache have different versions of a system library) and Bazel doesn't realize that it needs to rebuild. It would be a pretty heavy hammer if the way to fix that requires every OSS project to include the whole user-space OS (C++ compiler, system headers, libraries) in the repo or via submodule and then be careful that no include path or any part of the build system accidentally references any header or library outside the repository. And maybe this issue just doesn't need to be fixed (it's not like automake produces build rules that explicitly depend on system headers either!) -- my quibble was with the notion that Bazel, unlike CMake or whatever, provides fully hermetic builds, or tracks dependencies carefully enough to provide an organization-wide build cache across diversely configured/upgraded systems.
    - kornholi 6y ago
      > Ok, but that decision would have to be made by the project maintainer, in this case Google, not the person using Bazel to compile protobuf. No, that's up to the user. If you include protobuf in your Bazel workspace and have a toolchain configured, protobuf will be built with that toolchain (this is also how you would cross-compile). Bazel workspaces are still a little rough around the edges and interactions between them can be very confusing, as Google doesn't need them internally (everything is vendored as part of the same tree). Under the hood there's a default auto-configured toolchain that finds whatever is installed locally in the system. Since it has no way of knowing what files an arbitrary "cc" might depend on, you lose hermeticity by using it. Of course, the whole model falls apart if you want to depend on things outside of Bazel's control. However in practice I've found that writing Bazel build files for third-party dependencies isn't as hard as it seems, and provides tons of benefits as parent mentioned. For example, last week I was tracking down memory corruption in an external dependency. With a simple `bazel run --config asan -c dbg //foo`, I had foo and all of its transitive dependencies re-compiled and running under AddressSanitizer within minutes. How long would that take with other build systems? Probably a solid week.

by q3k 6y ago

simfoo 6y ago

Completely agree that you can't compare Bazel and CMake directly, they are not on the same level. But for compiling C++ Bazel will have to compare itself to CMake, which is plain and simple the defacto standard. All the complex things that Bazel brings into the game (distributed builds and caches primarily) simply don't have any value when the main task that you're trying to make faster is not done well. I would much rather like to see more tooling that does these things but builds upon layers of battle-tested existing software.

keithwinstein 6y ago

I've been confused for a while about the common claim that Bazel gives "full hermeticity" of builds -- it doesn't seem to be true in practice (at least for packages with system dependencies). Maybe you can help me clear it up. E.g. Google's protobuf libraries [1] can be built with Bazel, and they depend heavily on system headers outside the repository, e.g. <iostream> and <stdio.h> and lots more. If those headers subsequently change, Bazel will not pick up on this and will not know to rebuild the parts of the build that depend on them. To reproduce: run `bazel build -c opt //:protoc_lib` and then put random garbage in your /usr/include/stdio.h and /usr/include/c++/<version>/iostream and then rerun the bazel command -- it will not know to invalidate the build cache. If you `bazel clean` and then build again, you'll get different results. Bazel does a lot of really nice things and I can believe that within a google3-like environment (where the source code never references a system header?), it effectively provides hermetic builds, but in practice as used outside Google (or even in Google's public OSS releases) it doesn't seem to really match this description or enforce a hermetic seal. What am I missing? [1] https://github.com/protocolbuffers/protobuf https://github.com/protocolbuffers/protobuf

kornholi 6y ago
I wish Bazel was more explicit about things leaking in from outside the workspace, but you can always vendor the toolchain to be more hermetic. For example see https://github.com/grailbio/bazel-toolchain https://github.com/grailbio/bazel-toolchain which sets up LLVM and Clang for you. https://github.com/bazelbuild/bazel/issues/4558 https://github.com/bazelbuild/bazel/issues/4558
- keithwinstein 6y ago
  Ok, but that decision would have to be made by the project maintainer, in this case Google, not the person using Bazel to compile protobuf. (And not particular to Bazel -- a developer can make any build system effectively hermetic by vendoring everything.) In my view the challenge here is that a dependency changes (e.g. /usr/include/stdio.h is upgraded by the system package manager, or two users sharing a cache have different versions of a system library) and Bazel doesn't realize that it needs to rebuild. It would be a pretty heavy hammer if the way to fix that requires every OSS project to include the whole user-space OS (C++ compiler, system headers, libraries) in the repo or via submodule and then be careful that no include path or any part of the build system accidentally references any header or library outside the repository. And maybe this issue just doesn't need to be fixed (it's not like automake produces build rules that explicitly depend on system headers either!) -- my quibble was with the notion that Bazel, unlike CMake or whatever, provides fully hermetic builds, or tracks dependencies carefully enough to provide an organization-wide build cache across diversely configured/upgraded systems.
  - kornholi 6y ago
    > Ok, but that decision would have to be made by the project maintainer, in this case Google, not the person using Bazel to compile protobuf. No, that's up to the user. If you include protobuf in your Bazel workspace and have a toolchain configured, protobuf will be built with that toolchain (this is also how you would cross-compile). Bazel workspaces are still a little rough around the edges and interactions between them can be very confusing, as Google doesn't need them internally (everything is vendored as part of the same tree). Under the hood there's a default auto-configured toolchain that finds whatever is installed locally in the system. Since it has no way of knowing what files an arbitrary "cc" might depend on, you lose hermeticity by using it. Of course, the whole model falls apart if you want to depend on things outside of Bazel's control. However in practice I've found that writing Bazel build files for third-party dependencies isn't as hard as it seems, and provides tons of benefits as parent mentioned. For example, last week I was tracking down memory corruption in an external dependency. With a simple `bazel run --config asan -c dbg //foo`, I had foo and all of its transitive dependencies re-compiled and running under AddressSanitizer within minutes. How long would that take with other build systems? Probably a solid week.