6 ms·

Hm it seems like the defining feature of Bazel is high level "macros" like cc_library() and proto_library() on top of the low level build graph I looked at th

by chubot 3y ago

Hm it seems like the defining feature of Bazel is high level "macros" like cc_library() and proto_library() on top of the low level build graph I looked at the examples and tutorial and it still seems more low level, in that you mention object files and link them? main_o = compile("src/main.cpp", "build/tut1/src/main.o") util_o = compile("src/util.cpp", "build/tut1/src/util.o") link([main_o, util_o], "build/tut1/app") One reason I like the layering is for build variants -- dbg opt ASAN UBSAN -- something Make does very poorly. IMO you don't want to mention literal object file paths in the build config for this reason. I use a tree layout like _build/obj/cxx-asan/frontend.syntax.o _build/obj/cxx-asan/core/process.o _build/obj/cxx-ubsan/frontend.syntax.o _build/obj/cxx-ubsan/core/process.o and then this is abstracted with the Bazel-like target syntax //frontend/syntax //core/process This works well - you don't have to clean when rebuilding variants, and all the object sharing between test binaries really speeds up the build. IMO this is 100% essential for writing C++ these days -- all tests should be run with ASAN and UBSAN. --- I wrote a mini-bazel on top of Ninja with these features: https://www.oilshell.org/blog/2022/10/garbage-collector.html#declarative-ninja-mini-bazel https://www.oilshell.org/blog/2022/10/garbage-collector.html... So it's ~1700 lines, though some of that is our own logic, and extra features like computing preprocessed size, which I used to improve compile times. You get the build macros like asdl_library() generating C++ and Python (the same as proto_library(), a schema language that generates code) And it also correctly finds dependencies of code generators. So if you change a .py file that is imported by another .py file that is used to generated a C++ header, everything will work. That was one of the trickier bits, with Ninja implicit dependencies. This build file example mixes low level Ninja n.rule() and n.build() with high level r.cc_library() and so forth. I find this layering really does make it scale better for bigger projects https://github.com/oilshell/oil/blob/master/asdl/NINJA_subgraph.py https://github.com/oilshell/oil/blob/master/asdl/NINJA_subgr... Some more description - https://lobste.rs/s/qnb7xt/ninja_is_enough_build_system#c_tutskb https://lobste.rs/s/qnb7xt/ninja_is_enough_build_system#c_tu... Comment about Chrome/Android using the same pattern - https://lobste.rs/s/0qc6vp/arcan_0_6_3_i_pty_fool#c_ahqwky https://lobste.rs/s/0qc6vp/arcan_0_6_3_i_pty_fool#c_ahqwky which is explicitly in the Android docs - https://android.googlesource.com/platform/build/bazel/+/7d96ed30359f12311dd7a78c399e29eb695b2012/docs/concepts.md https://android.googlesource.com/platform/build/bazel/+/7d96... The main thing my system doesn't have as mentioned is the per-action build sandboxing, which Bazel has OS-specific wrappers for. That's really useful for finding missing deps. Right now it's not a big pain point, since the build will break in a slightly less obvious way if you're missing deps.

IshKebab 3y ago
The defining feature of Bazel is sandboxing build steps so that you can't accidentally miss edges in the dependency graph. This has enormous benefits: 1. Incremental/cached builds are reliable. 2. You can reliably avoid rebuilding/testing artifacts that cannot possibly be affected by a PR. Neither of those are possible in build systems that don't do this sandboxing. If you're doing a big project you really need these, hence why Google came up with this technique. I'm not sure there's much value in a new build system that doesn't even get that right. We already have dozens.
- aappleby 3y ago
  True, if you need sandboxing you've already committed yourself to a build system 10x larger than Hancho. If you don't need sandboxing, you can ditch a _ton_ of complexity.
- chubot 3y ago
  Yeah that's what I mentioned in the last sentence of my comment The reason most build systems don't have it is because sandboxing is OS-specific Bazel has sort of a complicated bootstrap/deployment for that reason. I mentioned it a few times, but someone should extract the exec wrappers from Bazel as a separate project :) It's a small amount of code, though fiddly to build and distribute There are few other strategies like bubblewrap, but last I heard bubblewrap is not installed by default on most distros. It needs setuid root so it's a bit fiddly and has security implications But then you also need something totally different for OS X and Windows
- bobsomers 3y ago
  The problem with Bazel is that a sandboxed build environment is useless unless you also have a hermetic runtime environment, and Bazel does nothing to solve this problem. While you may be able to consistently build, your runfiles tree can easily be un-runnable on any system except the one that built it. When I was heavily using Bazel ~3 years ago we had made substantial changes to the Bazel source to try to support this, but ultimately couldn't get it across the finish line without making changes to Linux's dynamic loader. After talking to a few folks familiar with Blaze, it seems that's exactly what they've done inside Google to solve the problem. Perhaps the situation has improved in the last 3 years, but my general experience with Bazel is that it made extremely lofty claims and delivered on very few of them.

by chubot 3y ago

IshKebab 3y ago

The defining feature of Bazel is sandboxing build steps so that you can't accidentally miss edges in the dependency graph. This has enormous benefits: 1. Incremental/cached builds are reliable. 2. You can reliably avoid rebuilding/testing artifacts that cannot possibly be affected by a PR. Neither of those are possible in build systems that don't do this sandboxing. If you're doing a big project you really need these, hence why Google came up with this technique. I'm not sure there's much value in a new build system that doesn't even get that right. We already have dozens.

aappleby 3y ago
True, if you need sandboxing you've already committed yourself to a build system 10x larger than Hancho. If you don't need sandboxing, you can ditch a _ton_ of complexity.
chubot 3y ago
Yeah that's what I mentioned in the last sentence of my comment The reason most build systems don't have it is because sandboxing is OS-specific Bazel has sort of a complicated bootstrap/deployment for that reason. I mentioned it a few times, but someone should extract the exec wrappers from Bazel as a separate project :) It's a small amount of code, though fiddly to build and distribute There are few other strategies like bubblewrap, but last I heard bubblewrap is not installed by default on most distros. It needs setuid root so it's a bit fiddly and has security implications But then you also need something totally different for OS X and Windows
bobsomers 3y ago
The problem with Bazel is that a sandboxed build environment is useless unless you also have a hermetic runtime environment, and Bazel does nothing to solve this problem. While you may be able to consistently build, your runfiles tree can easily be un-runnable on any system except the one that built it. When I was heavily using Bazel ~3 years ago we had made substantial changes to the Bazel source to try to support this, but ultimately couldn't get it across the finish line without making changes to Linux's dynamic loader. After talking to a few folks familiar with Blaze, it seems that's exactly what they've done inside Google to solve the problem. Perhaps the situation has improved in the last 3 years, but my general experience with Bazel is that it made extremely lofty claims and delivered on very few of them.