4 ms·

What's the motivation to avoid LLVM backends?

by MuffinFlavored 1y ago

sakras 1y ago
Likely performance - LLVM is somewhat notorious for being slower than ideal.
deleted 1y ago
[deleted]
mananaysiempre 1y ago
> In exchange [for eliminating the dependency on LLVM], Zig gains these benefits: > All our bugs are belong to us. > The compiler becomes trivial to build from source and to bootstrap with only a C compiler on the host system. > We stop dealing with annoying problems introduced by Linux distributions and package managers such as Homebrew related to LLVM, Clang, and LLD. There have been and continue to be many. > The Zig compiler binary goes from about 150 MiB to 5 MiB. > Compilation speed is increased by orders of magnitude. > [...] https://github.com/ziglang/zig/issues/16270 https://github.com/ziglang/zig/issues/16270
- remindmeagain 1y ago
  Does this mean `zig c++` is going away with LLVM dropped? That would be a shame - so useful.
  - forrestthewoods 1y ago
    As long as Zig builds the glibc shim libraries those can be used in a separate Clang build system. It’d be nice if Zig added a command to emit them more explicitly to use in non-Zig build systems. Would be awesome actually. Plz?
    - remindmeagain 1y ago
      I'm trying to wrap my mind around what you wrote. Do you envision a different binary separate from zig? The closest thing I could find online is: https://stackoverflow.com/questions/78892396/how-to-link-to-external-library-while-compiling-in-zig https://stackoverflow.com/questions/78892396/how-to-link-to-...
      - forrestthewoods 1y ago
        Cross-compiling is super super easy conceptually. All you need is headers and an import library. Windows is trivial to cross compile for because there is a single set of headers. And for DLLs Windows compiler like to generate an import lib. Which is basically a stub function for all exports. Linux is stupid and sucks. Linux doesn’t have a single set of headers because programmers in the 80s didn’t know better. And Linux linking to a shared library expects you to have a full copy of the library you expect to exist at runtime. Which is fucking stupid and ass backwards. But I digress. So. Why is Zig awesome for crosscompiling C++? Because Zig fixed Linux sucking ass. Zig moves mountains to generate thin “import libs” of the half dozen .so files you need to link against to run on Linux. If you want to cross compile C++ for Linux all you need is clang++ executable, headers, and those stub .so files. Zig generates the libs implicitly as part of its zig cc build process. I’m asking for the half dozen (C) or dozen (C++) libs to be explicitly exportable by Zig. So that they can be trivially used in other build systems. I’ve got a custom polyglot build system that leverages Zig. But I have to zig cc a hello_world.cpp and parse the output to get the paths to those libs. Which is annoying and janky. Hopefully that helps?
        remindmeagain 1y ago
        Yes it does. Thank you.
        rstat1 1y ago
        >> Linux doesn’t have a single set of headers because programmers in the 80s didn’t know better. I would argue that it does, but because its apparently illegal (obvious exaggeration alert) to package all the parts of a single entity together in one package, it just seems like it doesn't, where as on Windows there's a single package (the Windows SDK) that contains the majority of the relevant stuff. I do however 100% agree with you on linking to shared libraries. The way Linux compilers handle that is fucking stupid.
  - flohofwoe 1y ago
    AFAIK the current plan for zig cc is outlined here: https://github.com/ziglang/zig/issues/20875 https://github.com/ziglang/zig/issues/20875 E.g. LLVM will still be around one way or another, just not as deeply integrated as it is now.
- donio 1y ago
  The Go toolchain is a nice illustration of this approach working in practice. It fully bootstraps in 90 seconds on my aging laptop and since it's fully self-hosted it doesn't even need a C compiler unless you want cgo support. LLVM takes 2 hours to build on the same host and zig (with the LLVM backend) is another 20 minutes. It will be awesome if that can be brought down to 2 minutes or less.
  - AndyKelley 1y ago
    Is that building Go with Go? Or actual bootstrapping? Check this out... Building Zig with Zig: andy@bark ~/s/zig (master)> time zig build ________________________________________________________ Executed in 11.67 secs fish external Bootstrapping with only a C compiler dependency (not even make or shell!): andy@bark ~/s/zig (master)> time cc -o bootstrap bootstrap.c; and time ./bootstrap ________________________________________________________ Executed in 55.10 millis fish external gcc -o zig-wasm2c stage1/wasm2c.c -O2 -std=c99 ./zig-wasm2c stage1/zig1.wasm zig1.c gcc -o zig1 zig1.c stage1/wasi.c -std=c99 -Os -lm ./zig1 lib build-exe -ofmt=c -lc -OReleaseSmall --name zig2 -femit-bin=zig2.c -target x86_64-linux --dep build_options --dep aro -Mroot=src/main.zig -Mbuild_options=config.zig -Maro=lib/compiler/aro/aro.zig ./zig1 lib build-obj -ofmt=c -OReleaseSmall --name compiler_rt -femit-bin=compiler_rt.c -target x86_64-linux -Mroot=lib/compiler_rt.zig gcc -o zig2 zig2.c compiler_rt.c -std=c99 -O2 -fno-stack-protector -Istage1 -Wl,-z,stack-size=0x10000000 -pthread ________________________________________________________ Executed in 305.06 secs fish external
    - donio 1y ago
      > Is that building Go with Go? Or actual bootstrapping? Normally it's just Go with Go. Besides the Go compiler you need bash if you want to use the normal bootstrap script but not much else. You can build your way up from C by building an old enough version of Go that was still C based but that's not usually done these days. > Executed in 11.67 secs Nice!
  - cxr 1y ago
    Sometime after Minix 3 but before it had attained the critical mass for a self-sustaining community, compilation times went from 10 minutes on low-end hardware to ~3 hours, and the answer to the question "Why?" was "LLVM/clang".
- wiz21c 1y ago
  maybe I'm wrong, but rust is still using LLVM. When I see that list of benefits, I wonder why rust is still on LLVM... (honest question, I use rust everyday and I'm happy with it, except for compilation times :-) )
  - rowanG077 1y ago
    Rust depends heavily on the llvm optimization pipeline afaik. So it would be a heavy investment to write native backends.
  - zozbot234 1y ago
    Rust has cranelift as a natively bootstrapped alternative these days. No different from Golang or Zig.
    - MuffinFlavored 1y ago
      When will it become the default? What does it say about LLVM project that everybody starts off on it and then gets off of it?
tw061023 1y ago
LLVM is basically a resource pool for C++ compiler development. As such, it is highly C++ specific and leaks C++ semantics everywhere. It's especially funny when this happens in Rust, which is marketed as a "safer" alternative. Would you like a segfault out of nowhere in safe Rust? The issue is still open after two years by the way: https://github.com/rust-lang/rust/issues/107975 https://github.com/rust-lang/rust/issues/107975
- pjmlp 1y ago
  Which is why nowadays most frontends have been migrating to MLIR, and there is also ongoing work for clang as well.
  - AndyKelley 1y ago
    How does migrating to MLIR address the problem?
    - pjmlp 1y ago
      The higher abstraction level it provides over the LLVM IR, making language frontends and compiler passes less dependent on its semantics.
      - AndyKelley 1y ago
        Huh?? That can only make frontends' jobs more tricky.
        pjmlp 1y ago
        Yet is being embraced by everyone since its introduction in 2019, with its own organization and conference talks. So maybe all those universities, companies and the LLVM project know kind of what they are doing. - https://mlir.llvm.org/ https://mlir.llvm.org/ - https://llvm.github.io/clangir/ https://llvm.github.io/clangir/ - https://mlir.llvm.org/talks/ https://mlir.llvm.org/talks/
ethan_smith 1y ago
Zig's self-hosted compiler reduces compile times, enables better cross-compilation, allows finer control over codegen optimizations, and eliminates the large LLVM dependency which simplifies distribution and bootstrapping.

What's the motivation to avoid LLVM backends?

sakras 1y ago

Likely performance - LLVM is somewhat notorious for being slower than ideal.

deleted 1y ago

[deleted]

mananaysiempre 1y ago

> In exchange [for eliminating the dependency on LLVM], Zig gains these benefits: > All our bugs are belong to us. > The compiler becomes trivial to build from source and to bootstrap with only a C compiler on the host system. > We stop dealing with annoying problems introduced by Linux distributions and package managers such as Homebrew related to LLVM, Clang, and LLD. There have been and continue to be many. > The Zig compiler binary goes from about 150 MiB to 5 MiB. > Compilation speed is increased by orders of magnitude. > [...] https://github.com/ziglang/zig/issues/16270 https://github.com/ziglang/zig/issues/16270

remindmeagain 1y ago
Does this mean `zig c++` is going away with LLVM dropped? That would be a shame - so useful.
- forrestthewoods 1y ago
  As long as Zig builds the glibc shim libraries those can be used in a separate Clang build system. It’d be nice if Zig added a command to emit them more explicitly to use in non-Zig build systems. Would be awesome actually. Plz?
  - remindmeagain 1y ago
    I'm trying to wrap my mind around what you wrote. Do you envision a different binary separate from zig? The closest thing I could find online is: https://stackoverflow.com/questions/78892396/how-to-link-to-external-library-while-compiling-in-zig https://stackoverflow.com/questions/78892396/how-to-link-to-...
    - forrestthewoods 1y ago
      Cross-compiling is super super easy conceptually. All you need is headers and an import library. Windows is trivial to cross compile for because there is a single set of headers. And for DLLs Windows compiler like to generate an import lib. Which is basically a stub function for all exports. Linux is stupid and sucks. Linux doesn’t have a single set of headers because programmers in the 80s didn’t know better. And Linux linking to a shared library expects you to have a full copy of the library you expect to exist at runtime. Which is fucking stupid and ass backwards. But I digress. So. Why is Zig awesome for crosscompiling C++? Because Zig fixed Linux sucking ass. Zig moves mountains to generate thin “import libs” of the half dozen .so files you need to link against to run on Linux. If you want to cross compile C++ for Linux all you need is clang++ executable, headers, and those stub .so files. Zig generates the libs implicitly as part of its zig cc build process. I’m asking for the half dozen (C) or dozen (C++) libs to be explicitly exportable by Zig. So that they can be trivially used in other build systems. I’ve got a custom polyglot build system that leverages Zig. But I have to zig cc a hello_world.cpp and parse the output to get the paths to those libs. Which is annoying and janky. Hopefully that helps?
      - remindmeagain 1y ago
        Yes it does. Thank you.
      - rstat1 1y ago
        >> Linux doesn’t have a single set of headers because programmers in the 80s didn’t know better. I would argue that it does, but because its apparently illegal (obvious exaggeration alert) to package all the parts of a single entity together in one package, it just seems like it doesn't, where as on Windows there's a single package (the Windows SDK) that contains the majority of the relevant stuff. I do however 100% agree with you on linking to shared libraries. The way Linux compilers handle that is fucking stupid.
- flohofwoe 1y ago
  AFAIK the current plan for zig cc is outlined here: https://github.com/ziglang/zig/issues/20875 https://github.com/ziglang/zig/issues/20875 E.g. LLVM will still be around one way or another, just not as deeply integrated as it is now.
donio 1y ago
The Go toolchain is a nice illustration of this approach working in practice. It fully bootstraps in 90 seconds on my aging laptop and since it's fully self-hosted it doesn't even need a C compiler unless you want cgo support. LLVM takes 2 hours to build on the same host and zig (with the LLVM backend) is another 20 minutes. It will be awesome if that can be brought down to 2 minutes or less.
- AndyKelley 1y ago
  Is that building Go with Go? Or actual bootstrapping? Check this out... Building Zig with Zig: andy@bark ~/s/zig (master)> time zig build ________________________________________________________ Executed in 11.67 secs fish external Bootstrapping with only a C compiler dependency (not even make or shell!): andy@bark ~/s/zig (master)> time cc -o bootstrap bootstrap.c; and time ./bootstrap ________________________________________________________ Executed in 55.10 millis fish external gcc -o zig-wasm2c stage1/wasm2c.c -O2 -std=c99 ./zig-wasm2c stage1/zig1.wasm zig1.c gcc -o zig1 zig1.c stage1/wasi.c -std=c99 -Os -lm ./zig1 lib build-exe -ofmt=c -lc -OReleaseSmall --name zig2 -femit-bin=zig2.c -target x86_64-linux --dep build_options --dep aro -Mroot=src/main.zig -Mbuild_options=config.zig -Maro=lib/compiler/aro/aro.zig ./zig1 lib build-obj -ofmt=c -OReleaseSmall --name compiler_rt -femit-bin=compiler_rt.c -target x86_64-linux -Mroot=lib/compiler_rt.zig gcc -o zig2 zig2.c compiler_rt.c -std=c99 -O2 -fno-stack-protector -Istage1 -Wl,-z,stack-size=0x10000000 -pthread ________________________________________________________ Executed in 305.06 secs fish external
  - donio 1y ago
    > Is that building Go with Go? Or actual bootstrapping? Normally it's just Go with Go. Besides the Go compiler you need bash if you want to use the normal bootstrap script but not much else. You can build your way up from C by building an old enough version of Go that was still C based but that's not usually done these days. > Executed in 11.67 secs Nice!
- cxr 1y ago
  Sometime after Minix 3 but before it had attained the critical mass for a self-sustaining community, compilation times went from 10 minutes on low-end hardware to ~3 hours, and the answer to the question "Why?" was "LLVM/clang".
wiz21c 1y ago
maybe I'm wrong, but rust is still using LLVM. When I see that list of benefits, I wonder why rust is still on LLVM... (honest question, I use rust everyday and I'm happy with it, except for compilation times :-) )
- rowanG077 1y ago
  Rust depends heavily on the llvm optimization pipeline afaik. So it would be a heavy investment to write native backends.
- zozbot234 1y ago
  Rust has cranelift as a natively bootstrapped alternative these days. No different from Golang or Zig.
  - MuffinFlavored 1y ago
    When will it become the default? What does it say about LLVM project that everybody starts off on it and then gets off of it?

tw061023 1y ago

LLVM is basically a resource pool for C++ compiler development. As such, it is highly C++ specific and leaks C++ semantics everywhere. It's especially funny when this happens in Rust, which is marketed as a "safer" alternative. Would you like a segfault out of nowhere in safe Rust? The issue is still open after two years by the way: https://github.com/rust-lang/rust/issues/107975 https://github.com/rust-lang/rust/issues/107975

pjmlp 1y ago
Which is why nowadays most frontends have been migrating to MLIR, and there is also ongoing work for clang as well.
- AndyKelley 1y ago
  How does migrating to MLIR address the problem?
  - pjmlp 1y ago
    The higher abstraction level it provides over the LLVM IR, making language frontends and compiler passes less dependent on its semantics.
    - AndyKelley 1y ago
      Huh?? That can only make frontends' jobs more tricky.
      - pjmlp 1y ago
        Yet is being embraced by everyone since its introduction in 2019, with its own organization and conference talks. So maybe all those universities, companies and the LLVM project know kind of what they are doing. - https://mlir.llvm.org/ https://mlir.llvm.org/ - https://llvm.github.io/clangir/ https://llvm.github.io/clangir/ - https://mlir.llvm.org/talks/ https://mlir.llvm.org/talks/

ethan_smith 1y ago

Zig's self-hosted compiler reduces compile times, enables better cross-compilation, allows finer control over codegen optimizations, and eliminates the large LLVM dependency which simplifies distribution and bootstrapping.