13 ms·

Bazel Build System Support for LLVM

118 points by mshockwave 6y ago

xvilka 6y ago
While Bazel design seems not bad, could they have used a build system that doesn't require Java? It's especially important for bootstrapping. CMake requires only C++ at least.
- q3k 6y ago
  In practice, this isn't generally noticeable. Bazel is pretty damn fast, and even ships with its own JVM so installing it is pretty much painless. I agree on the bootstrapping issues resulting from that, though.
  - an_d_rew 6y ago
    Unfortunately, for us, Java “out of memory” errors are routine, and for even moderately-sized monorepos the RAM requirements can get insane.
    - dleslie 6y ago
      Precisely this. Advocating for Java as a general purpose tool language sort of overlooks the incredibly broad and prevasive installbase of machines more than ten years old, as well as the rapidly growing installbase of ARM SOC machines. For many, a tool eating up a gigabyte or more of memory is certainly not acceptable.
      - gerash 6y ago
        I'm lost. Can't people build their binaries on a powerful machine instead of an ARM tablet?
        dleslie 6y ago
        Not everyone has a powerful machine to do so with; many folks are still using computers that shipped with WinXP, or even Vista or 98/Me. It's a huge world outside of California. Personally, my primary home/non-work machines are an RPi4 and a Pinebook Pro.
        orf 6y ago
        So Google should ship a potentially inferior tool, slower, because some people run on inappropriate or highly outdated hardware? Hmm.
        brmgb 6y ago
        No, Bazel just fails to satisfy the very real need of building on less powerful machines because of its technological choice, a need served by other tools competing with it, and therefore is not really a truly general, one size fit all, solution. The fact that you personally deem the needs of others illegitimate doesn't preclude them from actually existing.
        dleslie 6y ago
        The topic is whether FOSS developers should use Bazel, not whether Bazel is good for Google. For many it is Bazel that is inferior due to development environment constraints. And "outdated" is a questionable term: if it satisfies their needs, why upgrade? Maybe Google should build better tools.
        oblio 6y ago
        I think you're part of a minority, though. You're talking about computers from 20, 20+ years ago and about Raspberry Pi. It's a personal choice. You can get a good second hand desktop from 2012-2015 for the price of your Raspberry Pi.
        dleslie 6y ago
        Travel to Africa, India and South East Asia some time. The world is a big place outside of California, and the hardware available to many is not what you'll find at a cafe in San Francisco.
    - jsolson 6y ago
      I'm curious what you consider "insane" to be -- and I mean this genuinely. Google has what I think most would consider to be a large monorepo, but also provides developer desktops with ample RAM. I've noticed blaze's startup time, but never really thought about its memory consumption. Where are you running into limits?
      - smiths1999 6y ago
        Doesn't blaze build remotely via citc? I thought files were loaded locally for editing but built remotely.
        jsolson 6y ago
        Blaze itself runs locally, reading BUILD files from citc. The build actions mostly execute remotely.
      - an_d_rew 6y ago
        10,000+ artifacts on a 20 core Intel box with 64 Gb RAM. We have some peculiarities of our Bazel setup that are not common, yet are (supposed to be) fully supported. Building can take hours.
        laurentlb 6y ago
        Are you using remote execution with Bazel? If you have hour long builds, it's often worth investing in remote execution (just remote caching can help too).
  - millstone 6y ago
    The effect is that open source projects avoid Bazel. I'd rather fight with CMake than take a Java dependency in a C++ codebase. It's a pity, feels like a real missed opportunity.
    - klodolph 6y ago
      I think open-source projects avoid Bazel not because of the Java issue. You can install it pretty easily and never realize that it's written in Java. By analogy, I'd say that if Make is a hammer, Bazel is a CNC machine. Most DIYers are going to have hammers, and everyone understands how to use them, but CNC machines are becoming cheaper and more common.
      - millstone 6y ago
        The official .deb is 156MB compressed, which is prohibitive for all but the largest OSS projects. I assume much of that is the bundled JRE, but that only illustrates the point about Java. Most tellingly, Google has gone through two major build system migrations (Android and Chrome), and neither project chose to migrate to Bazel/Blaze. If Google won't eat their own dogfood, it doesn't inspire much confidence.
        q3k 6y ago
        > If Google won't eat their own dogfood, it doesn't inspire much confidence. Chromium's gn started being prototyped in 2013 [1]. Android's soong started being developed in June 2015 [2]. Bazel's first open source release was in September 2015 [3]. In addition, you surely can't be serious about Google not 'dogfooding Blaze' - it's a critical build system internally at Google. And new external projects (like gVisor) are also now built using Bazel. [1] https://chromium.googlesource.com/chromium/src/tools/gn/+/3bd230b9a2b5c011dad077d810642b5c539fcb74 https://chromium.googlesource.com/chromium/src/tools/gn/+/3b... [2] https://android.googlesource.com/platform/build/soong/+/e441b9df9a68595d0dd7b8ed184aecb27c86054b https://android.googlesource.com/platform/build/soong/+/e441... [3] https://github.com/bazelbuild/bazel/releases/tag/0.1.0 https://github.com/bazelbuild/bazel/releases/tag/0.1.0
  - mehrdadn 6y ago
    > I agree on the bootstrapping issues resulting from that, though. What are these "issues"?
    - q3k 6y ago
      To build Bazel from source in an environment where you don't want to use any Internet binaries, you need a Java built from source. Needing a Java is painful. You also need a Bazel to build Bazel. And Bazel's build system itself IIRC wants to download stuff from the Internet on build - which can be pre-fetched, but distro maintainers still don't like having to deal with taht.
      - mehrdadn 6y ago
        Edit: never mind, I think I was confused what the comment was saying.
        SubjectToChange 6y ago
        Look at what Nix or Guix would have to do to build a project using Bazel vs a project using CMake.
      - vips7L 6y ago
        This is nonsense building OpenJDK from source should be as easy as: bash configure && make images
  - Joky 6y ago
    The speed and responsiveness is there, but the price is a tricky management of background daemon and aggressive memory prefetch and caching which may be a bit heavy on memory resources (which is then less available to the compiler processes). This works out well when you run the Bazel daemon locally and distribute the work to a remote build farm, but less when you're building a project like LLVM on your laptop.
- klodolph 6y ago
  As far as implementation languages go, Java is mature, extremely stable, doesn't suffer from the quirks and foibles of older languages like C++, and is slower-moving than some of the newer languages. I know I've complained about running Java services but I think it is a great choice for writing a build system.
- oscargrouch 6y ago
  You might like GN, which is Bazel like design but in C++. Defaults to Ninja and its superfast. https://github.com/timniederhausen/gn https://github.com/timniederhausen/gn
  - sam_bristow 6y ago
    Does GN do all the deterministic build stuff that Bazel does?
    - q3k 6y ago
      Not as far as I understand. It's closer to CMake/Meson than to Bazel.
      - wyldfire 6y ago
        But no feature tests IIRC
- tatskaari 6y ago
  Can I shamelessly plug my build system https://please.build https://please.build? It’s a Bazel-like written in go with a focus on simplicity. I feel your pain, We had a lot of the same complaints with Bazel, Buck, Pants etc.
  - Game_Ender 6y ago
    If you have the cycles porting the LLVM bazel build system to please and comparing performance and usability would be pretty sweet. It seems like a great public use case/benchmark for a large common project to test with.
invokestatic 6y ago
LLVM has the most complicated CMake setup I’ve ever seen. I’m sure there’s a reason for that, but I always loathe hacking around in the source for that reason. Finding which cpp file belongs to which CMake file is an annoying and often frustrating task. Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). To me, the whole value of CMake is to allow you to swap in different toolchains with ease. So I wish someone smarter than me would go clean up the current CMake setup instead of using another build system altogether. But I guess any improvement to build usability is a good thing.
- ihnorton 6y ago
  It’s been a couple years since I looked, but I recall LLVM’s CMake being among the cleanest I’ve ever seen.
  - dfgdghdf 6y ago
    Ouch. I have spent some time digging into LLVM and that is not a complement to CMake.
- nyanpasu64 6y ago
  > Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). As of recently (LLVM/Clang 11), its CMakeLists.txt still doesn't work with Clang as a compiler (and MSVC's headers). Partly because <atomic> doesn't compile in Clang's C++11 mode (which the .cmake files use when testing for atomics), and partly because it can't detect Clang's host/target (don't know) triple. I recall getting build-time failures as well as CMake setup problems.
- mehrdadn 6y ago
  I think the purpose of this project isn't to change the LLVM build system, but rather to integrate LLVM better into Bazel. Bazel offers deterministic builds (which allows things like content-based caching and early stopping), but for that to work well, you often need to build the toolchain itself using Bazel as well—since the object files depend on the compiler too, not just the source files. Currently, the system is kind of broken: if you upgrade your compilers, Bazel doesn't catch the changes, and your build cache ends up potentially breaking as a result. That said, I'm not sure why they don't just hash the compiler and library binaries instead of building them from scratch... but that's a different question.
Game_Ender 6y ago
The world needs a Bazel package management system so the Bazel community can collaborate on projects like this. Buckaroo [0] is the closest I have seen but it’s Buck focused and they have not rolled out Bazel support. I think ultimately it needs a company like npm that can upstream the needed changes to Bazel itself so that it supports these workflows better. With the right focus you could hit cargo levels of ease of use, but for large multi-language projects. 0 - https://github.com/LoopPerfect/buckaroo https://github.com/LoopPerfect/buckaroo
- wesleyy 6y ago
  Github is already a bazel package management system though? If the package is a bazel workspace all you need to do is add a http_archive rule pointing to that github repo
  - q3k 6y ago
    Not every package (especially core system packages, like zlib/openssl/glibc/...) are on GitHub and want to pull in Bazel buildfiles into their source tree. As such, there's no guaranteed canonical-upstream-repo:buildfile-repo mapping, so you need some way to organize, keep track of what's where, and make sure things work well together.
  - gravypod 6y ago
    That would work if, like golang, bazel was the "default" package manager for everyone. Right now it's not easy to get, for example, vulkan or muslc or qt as a bazel package. It's also not easy to publish a version of your package (A) that depends on another package (B). This would create a diamond-problem like situation where your package (C) depends on both packages (A->B, C->A, C->B). So, some code needs to resolve these issues and reproducible identify the exact hashs of everything to pull in to make it a not-manual process. Also, something great about the design docs linked in my other post: there's a presubmit.yaml standard so, pulling in a library, will include tests that bazel will run for whatever arch you're compiling for. For instance, say you pull in sqlite and need to build it for RISC-V. Before you just needed to hope that sqlite worked correctly on your arch, now you'll be able to test those situations in CI with RBE runners for all architectures.
sillysaurusx 6y ago
I was pleasantly surprised by Bazel. It seems to solve pretty much every concern I had, with not much downside. The biggest pain point was not being able to easily put the massive build files on an external hard drive, but you can do it with —disk_cache and —output_base. Unfortunately lots of scripts don’t expose a direct interface to the bazel command, making it necessary to modify those. Still, nice build system.
- mehrdadn 6y ago
  Bazel is like git... it is elegant in a lot of ways but it's also awful in a lot of ways. The nice thing about it is that, overall, it's just than the other stuff you find out there, and (I think) it's because it's very composable—a property many other build systems don't have. But it has lots of rough corners once you get your hands dirty.
- gravypod 6y ago
  You can put things like --disk_cache and --output_base into your ~/.bazelrc to globally apply it to everything without having to pipe it through a script.
Ericson2314 6y ago
no no no no no. Bazel has a some nice ideas, but has shat in my cornflakes too many times. It's really unusable for anything that isn't an all-inclusive monorepo. It's funny because we with Nixpkgs want to be all-inclusive too, except we don't think that requires vendoring the world / perfect synchronicity.
- klodolph 6y ago
  Nix and Bazel solve different problems. I'm not sure exactly what you mean by synchronicity, but that's basically how e.g. package-lock.json works... it's the direction that everything is moving in, and Bazel is a target because it's imposing this on new frontiers rather than just being a new language where "things work that way".
  - Ericson2314 6y ago
    > Nix and Bazel solve different problems. I think both sides want to erase that difference, and I commend them for it. > I'm not sure exactly what you mean by synchronicity, but that's basically how e.g. package-lock.json works... it's the direction that everything is moving in, and Bazel is a target because it's imposing this on new frontiers rather than just being a new language where "things work that way". package-lock.json stuff is fine, and Nix has long required that everything be locked. That is the good synchronicity: it's OK if upstream moves faster, downstream catches up and just pins whatever they are using. You can see how upgrade waves cascade downstream, this is because the sychronizing is local. The internal good model is everything just lives in one repro and must update all at once. This is global synchronicity. ---- It's a lot like 1 giant lock vs Multiversion concurrency control. The MVCC aspect is that there may be a few versions in flight as things cascade down, but old versions should be retired (like committed transactions) as new ones come. The consistency is basically the pinning; nevermind what other transactions/development is going one elsewhere, each package sees a consistent view as structured by the pins. If you go listen to Google people talk about dependency/version management. It's like they think it's global lock vs inconsistency. Monorepo or chaos. This false choice irks me. I guess their dev tools people should talk to their DB people?
anarazel 6y ago
I have never seriously used bazel. Just looking at it, I found the the thought of manually mirroring all #includes to the build configuration in a big project pretty aggravating. Is that not really an issue in practice? Do people automate that away?
- mehrdadn 6y ago
  People use globs (wildcards) and group targets together to make it more manageable. It still does present some friction moving forward, but it's much more pleasant than listing every file individually, and there's a kind of an elegance to the system when you get used to it. And it does warn you if you include files that you didn't depend on. (Pro tip: never use include=...; always depend directly on a cc_library(hdrs=...).) That, said I do wish there was a way to make it easier and avoid the redundancy.
- bertr4nd 6y ago
  By “mirroring the includes” do you just mean listing the .h files in the build files? Don’t most build systems already have that problem for cpp files? Tbh I prefer also being explicit about the headers packaged in a library as well as the source.
  - toast0 6y ago
    I just found this tutorial to get the compiler to list your dependencies for you with (GNU) Make http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/ http://make.mad-scientist.net/papers/advanced-auto-dependenc... So far, it seems to result in a more accurate Makefile (with includes), but my project is pretty small. The Erlang compiler can also output a dependency file, with pretty similar looking arguments, so that helps too.
    - anarazel 6y ago
      > By “mirroring the includes” do you just mean listing the .h files in the build files? It's not just listing them once, so they get installed or such. You need to list them as dependencies for objects that get built. > I just found this tutorial to get the compiler to list your dependencies for you with (GNU) Make Yea, that works very well. A lot of projects sticking with make use that via some makefile hackery (e.g. postgres). Cmake etc do so automatically as well(at least for the common generators). But with bazel you have to basically maintain that manually on a granular basis. So if you add a new #include to some random .c file you often also need to add that as a dependency. There's a bunch of error checking around that to make it easier to get it right, but it's still seems like a lot of work to me.
deleted 6y ago
[deleted]
liuliu 6y ago
Would this allow people to host out-of-tree LLVM-based project easier? Last time I checked (shamefully, probably 2 or 3 years ago), LLVM still heavily prefer in-tree projects (creating a sub-directory and CMakeList.txt file) for source-code level distribution. If I can `deps = ["@llvm-project//:xxx"]` to start write new passes for llvm bitcode, it would be very exciting.
gravypod 6y ago
Ran some benchmarking with cmake+ninja vs these bazel BUILD files on a soyoustart GAME-2. Build from scratch... - cmake: ~2900 files @ 28:21.07 total (https://snapshot.raintank.io/dashboard/snapshot/ijkptI1dlteBvYvIsfernj6DYgiU8GcV?orgId=2 https://snapshot.raintank.io/dashboard/snapshot/ijkptI1dlteB...) - bazel: ~5k targets @ 23:55.13 total (https://snapshot.raintank.io/dashboard/snapshot/TbOP6L814R3qiYJEt6lYimHJW5ksRMFD https://snapshot.raintank.io/dashboard/snapshot/TbOP6L814R3q...) If anyone can recommend a file to modify I'd also be happy to test out incremental build times. Anyway, super glad to see google externalizing some of this stuff! I also have the metrics from this prometheus instance saved so if anyone wants to see any other metrics from node_exporter let me know! fwiw bazel "felt" slower because the number of targets is an estimate and that kept going up while cmake was constant progress to "finish". I'd love to see how the bazel time looks with remote build execution enabled to see how much faster that goes.
- q3k 6y ago
  Another fun thing you can do is to do another clone of the llvm-bazel repo, build that using Bazel again. Things should've cached correctly and the rebuild from a totally different checkout should be pretty much instant, as long as your ~/.cache/bazel is still there.
- Joky 6y ago
  I would double-check that it builds the exact same thing (there are lot of options when building LLVM, and CMake may auto-detect and enable support for things like Go bindings for example). Choice of compiler and compiler flags can make a difference as well.
- siggi 6y ago
  As another data point - I ran the same llvm bazel build using remote build execution on 128 cores. - clean, caching disabled @ 5.93m INFO: Elapsed time: 356.189s, Critical Path: 187.81s INFO: 4235 processes: 4235 remote. - clean, fully remote cached @ 27.33s INFO: Elapsed time: 27.330s, Critical Path: 15.93s INFO: 4235 processes: 4235 remote cache hit. Full disclosure - I co-founded BuildBuddy (YC W20) which provides bazel remote build execution and other open core bazel tooling
chandlerc1024 6y ago
(disclaimer: I'm one of the authors of this) A bunch of comments seem to be comparing and contrasting CMake+Ninja vs. Bazel. Just my two cents, but I think that's missing the real point of this work. It isn't about whether Bazel is a better build system for LLVM. At least, that isn't my motivation. There are users of LLVM's libraries that use Bazel. Whether for good or bad reasons, it is extremely useful to enable them to use LLVM's libraries with a fully native Bazel build. That is my motivation: enabling the users of LLVM libraries that need to use Bazel for whatever reason to have the best possible experience.
ta988 6y ago
Byild systems are not equivalent! https://www.microsoft.com/en-us/research/publication/build-systems-la-carte/ https://www.microsoft.com/en-us/research/publication/build-s... . Also yes, excel is akin to a build system.

118 points by mshockwave 6y ago

While Bazel design seems not bad, could they have used a build system that doesn't require Java? It's especially important for bootstrapping. CMake requires only C++ at least.

q3k 6y ago
In practice, this isn't generally noticeable. Bazel is pretty damn fast, and even ships with its own JVM so installing it is pretty much painless. I agree on the bootstrapping issues resulting from that, though.
- an_d_rew 6y ago
  Unfortunately, for us, Java “out of memory” errors are routine, and for even moderately-sized monorepos the RAM requirements can get insane.
  - dleslie 6y ago
    Precisely this. Advocating for Java as a general purpose tool language sort of overlooks the incredibly broad and prevasive installbase of machines more than ten years old, as well as the rapidly growing installbase of ARM SOC machines. For many, a tool eating up a gigabyte or more of memory is certainly not acceptable.
    - gerash 6y ago
      I'm lost. Can't people build their binaries on a powerful machine instead of an ARM tablet?
      - dleslie 6y ago
        Not everyone has a powerful machine to do so with; many folks are still using computers that shipped with WinXP, or even Vista or 98/Me. It's a huge world outside of California. Personally, my primary home/non-work machines are an RPi4 and a Pinebook Pro.
        orf 6y ago
        So Google should ship a potentially inferior tool, slower, because some people run on inappropriate or highly outdated hardware? Hmm.
        brmgb 6y ago
        No, Bazel just fails to satisfy the very real need of building on less powerful machines because of its technological choice, a need served by other tools competing with it, and therefore is not really a truly general, one size fit all, solution. The fact that you personally deem the needs of others illegitimate doesn't preclude them from actually existing.
        dleslie 6y ago
        The topic is whether FOSS developers should use Bazel, not whether Bazel is good for Google. For many it is Bazel that is inferior due to development environment constraints. And "outdated" is a questionable term: if it satisfies their needs, why upgrade? Maybe Google should build better tools.
        oblio 6y ago
        I think you're part of a minority, though. You're talking about computers from 20, 20+ years ago and about Raspberry Pi. It's a personal choice. You can get a good second hand desktop from 2012-2015 for the price of your Raspberry Pi.
        dleslie 6y ago
        Travel to Africa, India and South East Asia some time. The world is a big place outside of California, and the hardware available to many is not what you'll find at a cafe in San Francisco.
  - jsolson 6y ago
    I'm curious what you consider "insane" to be -- and I mean this genuinely. Google has what I think most would consider to be a large monorepo, but also provides developer desktops with ample RAM. I've noticed blaze's startup time, but never really thought about its memory consumption. Where are you running into limits?
    - smiths1999 6y ago
      Doesn't blaze build remotely via citc? I thought files were loaded locally for editing but built remotely.
      - jsolson 6y ago
        Blaze itself runs locally, reading BUILD files from citc. The build actions mostly execute remotely.
    - an_d_rew 6y ago
      10,000+ artifacts on a 20 core Intel box with 64 Gb RAM. We have some peculiarities of our Bazel setup that are not common, yet are (supposed to be) fully supported. Building can take hours.
      - laurentlb 6y ago
        Are you using remote execution with Bazel? If you have hour long builds, it's often worth investing in remote execution (just remote caching can help too).
- millstone 6y ago
  The effect is that open source projects avoid Bazel. I'd rather fight with CMake than take a Java dependency in a C++ codebase. It's a pity, feels like a real missed opportunity.
  - klodolph 6y ago
    I think open-source projects avoid Bazel not because of the Java issue. You can install it pretty easily and never realize that it's written in Java. By analogy, I'd say that if Make is a hammer, Bazel is a CNC machine. Most DIYers are going to have hammers, and everyone understands how to use them, but CNC machines are becoming cheaper and more common.
    - millstone 6y ago
      The official .deb is 156MB compressed, which is prohibitive for all but the largest OSS projects. I assume much of that is the bundled JRE, but that only illustrates the point about Java. Most tellingly, Google has gone through two major build system migrations (Android and Chrome), and neither project chose to migrate to Bazel/Blaze. If Google won't eat their own dogfood, it doesn't inspire much confidence.
      - q3k 6y ago
        > If Google won't eat their own dogfood, it doesn't inspire much confidence. Chromium's gn started being prototyped in 2013 [1]. Android's soong started being developed in June 2015 [2]. Bazel's first open source release was in September 2015 [3]. In addition, you surely can't be serious about Google not 'dogfooding Blaze' - it's a critical build system internally at Google. And new external projects (like gVisor) are also now built using Bazel. [1] https://chromium.googlesource.com/chromium/src/tools/gn/+/3bd230b9a2b5c011dad077d810642b5c539fcb74 https://chromium.googlesource.com/chromium/src/tools/gn/+/3b... [2] https://android.googlesource.com/platform/build/soong/+/e441b9df9a68595d0dd7b8ed184aecb27c86054b https://android.googlesource.com/platform/build/soong/+/e441... [3] https://github.com/bazelbuild/bazel/releases/tag/0.1.0 https://github.com/bazelbuild/bazel/releases/tag/0.1.0
- mehrdadn 6y ago
  > I agree on the bootstrapping issues resulting from that, though. What are these "issues"?
  - q3k 6y ago
    To build Bazel from source in an environment where you don't want to use any Internet binaries, you need a Java built from source. Needing a Java is painful. You also need a Bazel to build Bazel. And Bazel's build system itself IIRC wants to download stuff from the Internet on build - which can be pre-fetched, but distro maintainers still don't like having to deal with taht.
    - mehrdadn 6y ago
      Edit: never mind, I think I was confused what the comment was saying.
      - SubjectToChange 6y ago
        Look at what Nix or Guix would have to do to build a project using Bazel vs a project using CMake.
    - vips7L 6y ago
      This is nonsense building OpenJDK from source should be as easy as: bash configure && make images
- Joky 6y ago
  The speed and responsiveness is there, but the price is a tricky management of background daemon and aggressive memory prefetch and caching which may be a bit heavy on memory resources (which is then less available to the compiler processes). This works out well when you run the Bazel daemon locally and distribute the work to a remote build farm, but less when you're building a project like LLVM on your laptop.
klodolph 6y ago
As far as implementation languages go, Java is mature, extremely stable, doesn't suffer from the quirks and foibles of older languages like C++, and is slower-moving than some of the newer languages. I know I've complained about running Java services but I think it is a great choice for writing a build system.
oscargrouch 6y ago
You might like GN, which is Bazel like design but in C++. Defaults to Ninja and its superfast. https://github.com/timniederhausen/gn https://github.com/timniederhausen/gn
- sam_bristow 6y ago
  Does GN do all the deterministic build stuff that Bazel does?
  - q3k 6y ago
    Not as far as I understand. It's closer to CMake/Meson than to Bazel.
    - wyldfire 6y ago
      But no feature tests IIRC
tatskaari 6y ago
Can I shamelessly plug my build system https://please.build https://please.build? It’s a Bazel-like written in go with a focus on simplicity. I feel your pain, We had a lot of the same complaints with Bazel, Buck, Pants etc.
- Game_Ender 6y ago
  If you have the cycles porting the LLVM bazel build system to please and comparing performance and usability would be pretty sweet. It seems like a great public use case/benchmark for a large common project to test with.

invokestatic 6y ago

LLVM has the most complicated CMake setup I’ve ever seen. I’m sure there’s a reason for that, but I always loathe hacking around in the source for that reason. Finding which cpp file belongs to which CMake file is an annoying and often frustrating task. Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). To me, the whole value of CMake is to allow you to swap in different toolchains with ease. So I wish someone smarter than me would go clean up the current CMake setup instead of using another build system altogether. But I guess any improvement to build usability is a good thing.

ihnorton 6y ago
It’s been a couple years since I looked, but I recall LLVM’s CMake being among the cleanest I’ve ever seen.
- dfgdghdf 6y ago
  Ouch. I have spent some time digging into LLVM and that is not a complement to CMake.
nyanpasu64 6y ago
> Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). As of recently (LLVM/Clang 11), its CMakeLists.txt still doesn't work with Clang as a compiler (and MSVC's headers). Partly because <atomic> doesn't compile in Clang's C++11 mode (which the .cmake files use when testing for atomics), and partly because it can't detect Clang's host/target (don't know) triple. I recall getting build-time failures as well as CMake setup problems.
mehrdadn 6y ago
I think the purpose of this project isn't to change the LLVM build system, but rather to integrate LLVM better into Bazel. Bazel offers deterministic builds (which allows things like content-based caching and early stopping), but for that to work well, you often need to build the toolchain itself using Bazel as well—since the object files depend on the compiler too, not just the source files. Currently, the system is kind of broken: if you upgrade your compilers, Bazel doesn't catch the changes, and your build cache ends up potentially breaking as a result. That said, I'm not sure why they don't just hash the compiler and library binaries instead of building them from scratch... but that's a different question.

Game_Ender 6y ago

The world needs a Bazel package management system so the Bazel community can collaborate on projects like this. Buckaroo [0] is the closest I have seen but it’s Buck focused and they have not rolled out Bazel support. I think ultimately it needs a company like npm that can upstream the needed changes to Bazel itself so that it supports these workflows better. With the right focus you could hit cargo levels of ease of use, but for large multi-language projects. 0 - https://github.com/LoopPerfect/buckaroo https://github.com/LoopPerfect/buckaroo

wesleyy 6y ago
Github is already a bazel package management system though? If the package is a bazel workspace all you need to do is add a http_archive rule pointing to that github repo
- q3k 6y ago
  Not every package (especially core system packages, like zlib/openssl/glibc/...) are on GitHub and want to pull in Bazel buildfiles into their source tree. As such, there's no guaranteed canonical-upstream-repo:buildfile-repo mapping, so you need some way to organize, keep track of what's where, and make sure things work well together.
- gravypod 6y ago
  That would work if, like golang, bazel was the "default" package manager for everyone. Right now it's not easy to get, for example, vulkan or muslc or qt as a bazel package. It's also not easy to publish a version of your package (A) that depends on another package (B). This would create a diamond-problem like situation where your package (C) depends on both packages (A->B, C->A, C->B). So, some code needs to resolve these issues and reproducible identify the exact hashs of everything to pull in to make it a not-manual process. Also, something great about the design docs linked in my other post: there's a presubmit.yaml standard so, pulling in a library, will include tests that bazel will run for whatever arch you're compiling for. For instance, say you pull in sqlite and need to build it for RISC-V. Before you just needed to hope that sqlite worked correctly on your arch, now you'll be able to test those situations in CI with RBE runners for all architectures.

sillysaurusx 6y ago

I was pleasantly surprised by Bazel. It seems to solve pretty much every concern I had, with not much downside. The biggest pain point was not being able to easily put the massive build files on an external hard drive, but you can do it with —disk_cache and —output_base. Unfortunately lots of scripts don’t expose a direct interface to the bazel command, making it necessary to modify those. Still, nice build system.

mehrdadn 6y ago
Bazel is like git... it is elegant in a lot of ways but it's also awful in a lot of ways. The nice thing about it is that, overall, it's just than the other stuff you find out there, and (I think) it's because it's very composable—a property many other build systems don't have. But it has lots of rough corners once you get your hands dirty.
gravypod 6y ago
You can put things like --disk_cache and --output_base into your ~/.bazelrc to globally apply it to everything without having to pipe it through a script.

Ericson2314 6y ago

no no no no no. Bazel has a some nice ideas, but has shat in my cornflakes too many times. It's really unusable for anything that isn't an all-inclusive monorepo. It's funny because we with Nixpkgs want to be all-inclusive too, except we don't think that requires vendoring the world / perfect synchronicity.

klodolph 6y ago
Nix and Bazel solve different problems. I'm not sure exactly what you mean by synchronicity, but that's basically how e.g. package-lock.json works... it's the direction that everything is moving in, and Bazel is a target because it's imposing this on new frontiers rather than just being a new language where "things work that way".
- Ericson2314 6y ago
  > Nix and Bazel solve different problems. I think both sides want to erase that difference, and I commend them for it. > I'm not sure exactly what you mean by synchronicity, but that's basically how e.g. package-lock.json works... it's the direction that everything is moving in, and Bazel is a target because it's imposing this on new frontiers rather than just being a new language where "things work that way". package-lock.json stuff is fine, and Nix has long required that everything be locked. That is the good synchronicity: it's OK if upstream moves faster, downstream catches up and just pins whatever they are using. You can see how upgrade waves cascade downstream, this is because the sychronizing is local. The internal good model is everything just lives in one repro and must update all at once. This is global synchronicity. ---- It's a lot like 1 giant lock vs Multiversion concurrency control. The MVCC aspect is that there may be a few versions in flight as things cascade down, but old versions should be retired (like committed transactions) as new ones come. The consistency is basically the pinning; nevermind what other transactions/development is going one elsewhere, each package sees a consistent view as structured by the pins. If you go listen to Google people talk about dependency/version management. It's like they think it's global lock vs inconsistency. Monorepo or chaos. This false choice irks me. I guess their dev tools people should talk to their DB people?

anarazel 6y ago

I have never seriously used bazel. Just looking at it, I found the the thought of manually mirroring all #includes to the build configuration in a big project pretty aggravating. Is that not really an issue in practice? Do people automate that away?

mehrdadn 6y ago
People use globs (wildcards) and group targets together to make it more manageable. It still does present some friction moving forward, but it's much more pleasant than listing every file individually, and there's a kind of an elegance to the system when you get used to it. And it does warn you if you include files that you didn't depend on. (Pro tip: never use include=...; always depend directly on a cc_library(hdrs=...).) That, said I do wish there was a way to make it easier and avoid the redundancy.
bertr4nd 6y ago
By “mirroring the includes” do you just mean listing the .h files in the build files? Don’t most build systems already have that problem for cpp files? Tbh I prefer also being explicit about the headers packaged in a library as well as the source.
- toast0 6y ago
  I just found this tutorial to get the compiler to list your dependencies for you with (GNU) Make http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/ http://make.mad-scientist.net/papers/advanced-auto-dependenc... So far, it seems to result in a more accurate Makefile (with includes), but my project is pretty small. The Erlang compiler can also output a dependency file, with pretty similar looking arguments, so that helps too.
  - anarazel 6y ago
    > By “mirroring the includes” do you just mean listing the .h files in the build files? It's not just listing them once, so they get installed or such. You need to list them as dependencies for objects that get built. > I just found this tutorial to get the compiler to list your dependencies for you with (GNU) Make Yea, that works very well. A lot of projects sticking with make use that via some makefile hackery (e.g. postgres). Cmake etc do so automatically as well(at least for the common generators). But with bazel you have to basically maintain that manually on a granular basis. So if you add a new #include to some random .c file you often also need to add that as a dependency. There's a bunch of error checking around that to make it easier to get it right, but it's still seems like a lot of work to me.

deleted 6y ago

[deleted]

liuliu 6y ago

Would this allow people to host out-of-tree LLVM-based project easier? Last time I checked (shamefully, probably 2 or 3 years ago), LLVM still heavily prefer in-tree projects (creating a sub-directory and CMakeList.txt file) for source-code level distribution. If I can `deps = ["@llvm-project//:xxx"]` to start write new passes for llvm bitcode, it would be very exciting.

gravypod 6y ago

Ran some benchmarking with cmake+ninja vs these bazel BUILD files on a soyoustart GAME-2. Build from scratch... - cmake: ~2900 files @ 28:21.07 total (https://snapshot.raintank.io/dashboard/snapshot/ijkptI1dlteBvYvIsfernj6DYgiU8GcV?orgId=2 https://snapshot.raintank.io/dashboard/snapshot/ijkptI1dlteB...) - bazel: ~5k targets @ 23:55.13 total (https://snapshot.raintank.io/dashboard/snapshot/TbOP6L814R3qiYJEt6lYimHJW5ksRMFD https://snapshot.raintank.io/dashboard/snapshot/TbOP6L814R3q...) If anyone can recommend a file to modify I'd also be happy to test out incremental build times. Anyway, super glad to see google externalizing some of this stuff! I also have the metrics from this prometheus instance saved so if anyone wants to see any other metrics from node_exporter let me know! fwiw bazel "felt" slower because the number of targets is an estimate and that kept going up while cmake was constant progress to "finish". I'd love to see how the bazel time looks with remote build execution enabled to see how much faster that goes.

q3k 6y ago
Another fun thing you can do is to do another clone of the llvm-bazel repo, build that using Bazel again. Things should've cached correctly and the rebuild from a totally different checkout should be pretty much instant, as long as your ~/.cache/bazel is still there.
Joky 6y ago
I would double-check that it builds the exact same thing (there are lot of options when building LLVM, and CMake may auto-detect and enable support for things like Go bindings for example). Choice of compiler and compiler flags can make a difference as well.
siggi 6y ago
As another data point - I ran the same llvm bazel build using remote build execution on 128 cores. - clean, caching disabled @ 5.93m INFO: Elapsed time: 356.189s, Critical Path: 187.81s INFO: 4235 processes: 4235 remote. - clean, fully remote cached @ 27.33s INFO: Elapsed time: 27.330s, Critical Path: 15.93s INFO: 4235 processes: 4235 remote cache hit. Full disclosure - I co-founded BuildBuddy (YC W20) which provides bazel remote build execution and other open core bazel tooling

chandlerc1024 6y ago

(disclaimer: I'm one of the authors of this) A bunch of comments seem to be comparing and contrasting CMake+Ninja vs. Bazel. Just my two cents, but I think that's missing the real point of this work. It isn't about whether Bazel is a better build system for LLVM. At least, that isn't my motivation. There are users of LLVM's libraries that use Bazel. Whether for good or bad reasons, it is extremely useful to enable them to use LLVM's libraries with a fully native Bazel build. That is my motivation: enabling the users of LLVM libraries that need to use Bazel for whatever reason to have the best possible experience.

ta988 6y ago

Byild systems are not equivalent! https://www.microsoft.com/en-us/research/publication/build-systems-la-carte/ https://www.microsoft.com/en-us/research/publication/build-s... . Also yes, excel is akin to a build system.