6 ms·

LLVM has the most complicated CMake setup I’ve ever seen. I’m sure there’s a reason for that, but I always loathe hacking around in the source for that reason.

LLVM has the most complicated CMake setup I’ve ever seen. I’m sure there’s a reason for that, but I always loathe hacking around in the source for that reason. Finding which cpp file belongs to which CMake file is an annoying and often frustrating task. Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). To me, the whole value of CMake is to allow you to swap in different toolchains with ease. So I wish someone smarter than me would go clean up the current CMake setup instead of using another build system altogether. But I guess any improvement to build usability is a good thing.

ihnorton 6y ago
It’s been a couple years since I looked, but I recall LLVM’s CMake being among the cleanest I’ve ever seen.
- dfgdghdf 6y ago
  Ouch. I have spent some time digging into LLVM and that is not a complement to CMake.
nyanpasu64 6y ago
> Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). As of recently (LLVM/Clang 11), its CMakeLists.txt still doesn't work with Clang as a compiler (and MSVC's headers). Partly because <atomic> doesn't compile in Clang's C++11 mode (which the .cmake files use when testing for atomics), and partly because it can't detect Clang's host/target (don't know) triple. I recall getting build-time failures as well as CMake setup problems.
mehrdadn 6y ago
I think the purpose of this project isn't to change the LLVM build system, but rather to integrate LLVM better into Bazel. Bazel offers deterministic builds (which allows things like content-based caching and early stopping), but for that to work well, you often need to build the toolchain itself using Bazel as well—since the object files depend on the compiler too, not just the source files. Currently, the system is kind of broken: if you upgrade your compilers, Bazel doesn't catch the changes, and your build cache ends up potentially breaking as a result. That said, I'm not sure why they don't just hash the compiler and library binaries instead of building them from scratch... but that's a different question.
- justicezyx 6y ago
  Right, the Google monorepo is more monolithic than anyone can imagine. The compiler was part of the repo as well. I forgot how does the bootstrap work, but it definitely builds a compiler if needed (most often there is a cached version somewhere already, so no risk of small changes result into gigantic builds)
  - Joky 6y ago
    The bootstrap isn't too much of an issue, a binary release of the compiler is committed inside the repo itself.
  - dleslie 6y ago
    How many dozens of gigabytes is it, excluding art artifacts?
    - gravypod 6y ago
      No idea about excluding binary/artifacts but it was ~80TB [0]. A better comparison is available here: https://www.visualcapitalist.com/wp-content/uploads/2017/02/1276_lines_of_code_sep2015_fb.png https://www.visualcapitalist.com/wp-content/uploads/2017/02/... [0] - https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext#:~:text=Google%2DScale&text=The%20Google%20codebase%20includes%20approximately,Google's%20entire%2018%2Dyear%20existence https://cacm.acm.org/magazines/2016/7/204032-why-google-stor....
      - dleslie 6y ago
        Wild! Thanks for the info.
m0zg 6y ago
Could be that it's harder to "clean up CMake setup" than implement Bazel setup. Google compiles everything from source, so this is probably just a repackaged variant of what they use internally in their monorepo.
deleted 6y ago
[deleted]
Ericson2314 6y ago
The idea having subpackages where one subpackage is used to build another subpackage breaks CMake's (and autotools, and most things') brains. I've been talking to the Meson devs about how it would be really nice to nail this in Meson. Compiler's build systems are usually an absolute mess, and there's no reason it needs to be this way. As a distro maintainer, compiler's terrible build systems are in fact my #1 source of issues.
kingaillas 6y ago
>Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). Any specific info on this? I build llvm and various associated projects, mostly on linux but on windows and macos sometimes, and don't remember issues of this nature. I just kicked off a build of llvm and clang on windows using VS 2019 community and it finished just fine. I used the Ninja generator for speed but "Visual Studio 16 2019" also works. Those are the only two I do. EDIT: ah, using llvm/clang to build llvm/clang. I have not done that, I use VS community. I'm not sure this is actually a CMake issue though.
slimsag 6y ago
It's also not possible to build the Go or OCaml bindings with MSYS2/MinGW because of some silly assumptions in the CMake file.[0] I have an open issue describing exactly what needs to change, as well as a patch for it, but haven't been super successful at getting any attention on the issue and the patch submission process is fairly time consuming for such a small change.[3] [0] https://github.com/hexops/llvm-go-bindings#known-issues-with-go-llvm-bindings https://github.com/hexops/llvm-go-bindings#known-issues-with... [1] https://bugs.llvm.org/show_bug.cgi?id=44551 https://bugs.llvm.org/show_bug.cgi?id=44551 [3] https://llvm.org/docs/Contributing.html#how-to-submit-a-patch https://llvm.org/docs/Contributing.html#how-to-submit-a-patc...
- wallnuss 6y ago
  Arcanist is a but of a bother, but please do submit the patch.

ihnorton 6y ago

It’s been a couple years since I looked, but I recall LLVM’s CMake being among the cleanest I’ve ever seen.

dfgdghdf 6y ago
Ouch. I have spent some time digging into LLVM and that is not a complement to CMake.

nyanpasu64 6y ago

> Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). As of recently (LLVM/Clang 11), its CMakeLists.txt still doesn't work with Clang as a compiler (and MSVC's headers). Partly because <atomic> doesn't compile in Clang's C++11 mode (which the .cmake files use when testing for atomics), and partly because it can't detect Clang's host/target (don't know) triple. I recall getting build-time failures as well as CMake setup problems.

mehrdadn 6y ago

I think the purpose of this project isn't to change the LLVM build system, but rather to integrate LLVM better into Bazel. Bazel offers deterministic builds (which allows things like content-based caching and early stopping), but for that to work well, you often need to build the toolchain itself using Bazel as well—since the object files depend on the compiler too, not just the source files. Currently, the system is kind of broken: if you upgrade your compilers, Bazel doesn't catch the changes, and your build cache ends up potentially breaking as a result. That said, I'm not sure why they don't just hash the compiler and library binaries instead of building them from scratch... but that's a different question.

justicezyx 6y ago
Right, the Google monorepo is more monolithic than anyone can imagine. The compiler was part of the repo as well. I forgot how does the bootstrap work, but it definitely builds a compiler if needed (most often there is a cached version somewhere already, so no risk of small changes result into gigantic builds)
- Joky 6y ago
  The bootstrap isn't too much of an issue, a binary release of the compiler is committed inside the repo itself.
- dleslie 6y ago
  How many dozens of gigabytes is it, excluding art artifacts?
  - gravypod 6y ago
    No idea about excluding binary/artifacts but it was ~80TB [0]. A better comparison is available here: https://www.visualcapitalist.com/wp-content/uploads/2017/02/1276_lines_of_code_sep2015_fb.png https://www.visualcapitalist.com/wp-content/uploads/2017/02/... [0] - https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext#:~:text=Google%2DScale&text=The%20Google%20codebase%20includes%20approximately,Google's%20entire%2018%2Dyear%20existence https://cacm.acm.org/magazines/2016/7/204032-why-google-stor....
    - dleslie 6y ago
      Wild! Thanks for the info.

m0zg 6y ago

Could be that it's harder to "clean up CMake setup" than implement Bazel setup. Google compiles everything from source, so this is probably just a repackaged variant of what they use internally in their monorepo.

deleted 6y ago

[deleted]

Ericson2314 6y ago

The idea having subpackages where one subpackage is used to build another subpackage breaks CMake's (and autotools, and most things') brains. I've been talking to the Meson devs about how it would be really nice to nail this in Meson. Compiler's build systems are usually an absolute mess, and there's no reason it needs to be this way. As a distro maintainer, compiler's terrible build systems are in fact my #1 source of issues.

kingaillas 6y ago

>Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). Any specific info on this? I build llvm and various associated projects, mostly on linux but on windows and macos sometimes, and don't remember issues of this nature. I just kicked off a build of llvm and clang on windows using VS 2019 community and it finished just fine. I used the Ninja generator for speed but "Visual Studio 16 2019" also works. Those are the only two I do. EDIT: ah, using llvm/clang to build llvm/clang. I have not done that, I use VS community. I'm not sure this is actually a CMake issue though.

slimsag 6y ago

It's also not possible to build the Go or OCaml bindings with MSYS2/MinGW because of some silly assumptions in the CMake file.[0] I have an open issue describing exactly what needs to change, as well as a patch for it, but haven't been super successful at getting any attention on the issue and the patch submission process is fairly time consuming for such a small change.[3] [0] https://github.com/hexops/llvm-go-bindings#known-issues-with-go-llvm-bindings https://github.com/hexops/llvm-go-bindings#known-issues-with... [1] https://bugs.llvm.org/show_bug.cgi?id=44551 https://bugs.llvm.org/show_bug.cgi?id=44551 [3] https://llvm.org/docs/Contributing.html#how-to-submit-a-patch https://llvm.org/docs/Contributing.html#how-to-submit-a-patc...

wallnuss 6y ago
Arcanist is a but of a bother, but please do submit the patch.