3 ms·

Speaking of cross compilers - I was able to use the zig toolchain in order to cross compile for macos apple silicon, arm Linux, windows x64, windows arm and x86

Speaking of cross compilers - I was able to use the zig toolchain in order to cross compile for macos apple silicon, arm Linux, windows x64, windows arm and x86_64 Linux musl (statically linked). It's super handy and works as a drop in.

vlovich123 3y ago
Interestingly you can use zig within your Rust build to more easily cross-compile: https://github.com/rust-cross/cargo-zigbuild https://github.com/rust-cross/cargo-zigbuild I’m curious what the blockers are for rustc to cross-compile like zig does natively.
- adastra22 3y ago
  Just add another arch with rustup. Am I missing something?
  - weinzierl 3y ago
    That would solve the compilation problem, but there is more to create a working binary. For MacOS the official (and I believe only legal) way is to use a Mac with XCode installed for linking. Now, you can download all the necessary files from a Mac and build a cross-compilation toolchain on your Linux system. I believe you could not legally distribute a project doing this and that is why these projects don't exist or are usually short-lived (with the notable exception of zig). We will see how that goes for OP. So, solutions like osxcross resort to shipping scripts that help you to acquire the necessary files and make that process easier. The OP builds on osxcross, but is "batteries included". zig has an even more difficult problem to solve because it tries to compile and link for many platforms. Shipping all the different requirements in their original form would make a zig distribution huge. So it does some clever magic I do not fully understand to create the necessary files from just the necessary subset of data. This reduces the size because there is overlap between platforms. It also means that they are not shipping the Mac files in their complete and original form and have gotten away with this legally so far. At least that is what I believe is happening. I hope someone with more knowledge about zig could explain it better.
    - adastra22 3y ago
      You don’t need the OSX SDK to make runnable Rust binaries from other OS: https://betterprogramming.pub/cross-compiling-rust-from-mac-to-linux-7fad5a454ab1 https://betterprogramming.pub/cross-compiling-rust-from-mac-...
      - weinzierl 3y ago
        The link is about making Linux binaries on MacOS, which works because almost everything to build for Linux is already on OSX. The other way around it's not true. A Linux system does come neither with the OSX SDK nor the Apple patched llvm linker. I believe at least part of the required files cannot be legally distributed with a Linux system, but I might be wrong on that.
        adastra22 3y ago
        It appears you’re right, thank you! I’m very surprised there isn’t a llvm linker available, minus all the foundation kits that rust doesn’t need.
        weinzierl 3y ago
        The llvm linker is a cross-linker in principle. I think it is more about the specialities that Apple needs, like code signing.
        adastra22 3y ago
        Ah. For what it's worth, those have been reverse engineered and copied elsewhere. I maintain a project that does cross-compiled macOS builds and our own signing on Linux.
  - vlovich123 3y ago
    I could be mistaken, but I believe that installs a completely parallel toolchain which is relatively large compared with zigs approach of having all the platforms generatable from a single toolchain.
  - pornel 3y ago
    Rustup arch only adds support for static libraries (unliked objects), and craps out as soon as you try to build a binary. Rust did not do the legwork that zig did to bundle libc and a working linker, and Cargo is exceptionally naive in its default configuration, so it won't even find a usable cross-linker on the system, nor even try the chronically-unfinished rustc-lld integration there is.
- weinzierl 3y ago
  There is cargo-zigbuild, which uses zig as a linker, but I have yet to try it out.
- shepherdjerred 3y ago
  That's actually exactly how this cross-compiler works for Rust!
- jonathanhrl 3y ago
  AFAIK, there are no blockers really, it's just that Rust does not have its own linker, it delegates linkage to the system linker depending on the target. <https://rust-lang.github.io/rustup/cross-compilation.html?highlight=linker#cross-compilation https://rust-lang.github.io/rustup/cross-compilation.html?hi...> You can specify your own linker if you want, mold is a very popular one, and cargo-zigbuild does the same behind the scenes with zig cc as the linker. I did something similar a couple of months ago (or a year ago? I don't remember exactly). I managed to cross-compile to windows-msvc on Linux using Wine, there's a project that provides the scripts to make this easier, including the linker wrapper: <https://github.com/est31/msvc-wine-rust https://github.com/est31/msvc-wine-rust>. It was just for fun because Rust can already target windows-gnu and it'll use mingw64 linker. Rust's approach to things is normally to provide the basic foundation and let the community build on top of it. I personally like this approach, but it also has this downside of people not knowing they may need an external/community built tool to accomplish what they want.
- wyldfire 3y ago
  > I’m curious what the blockers are for rustc to cross-compile like zig does natively. Zig ships all the bits you need for a C/C++ toolchain including things like the C library in some cases (IIUC). Rust could use libclang to make a "rustc cc" just like zig's. But I get the sense that it is probably not a goal of the project to have this functionality.
- Cloudef 3y ago
  Rust proc macros and the fact you can't really static link if something uses them made rust unsuitable for me.
menduz 3y ago
I'm commenting to add a critical mass around Zig. If you are prisoner of CMake, autoconf, templates, makefiles, etc.. please try the Zig toolchain for your c/c++ projects. It is worth every second of investment
- DragonStrength 3y ago
  I’m intrigued. Do you have a starting point? And how does it handle if I need to integrate with another language as well, like Swift?
  - rockwotj 3y ago
    I was researching this the other day, here is a non trivial example: https://github.com/floooh/sokol-tools/blob/master/build.zig https://github.com/floooh/sokol-tools/blob/master/build.zig Here is a tutorial: https://www.zvm.app/tutorials/zig-build-cpp.html https://www.zvm.app/tutorials/zig-build-cpp.html
- wredue 3y ago
  Zig build is somewhat poorly documented currently, which can lead to some annoyance (also some other annoyances with dependency hashes), but in my experimentation: Dependencies in build.zig.zon are downloaded in to ~/.cache/zig/p/<hash> (incidentally, this means you need to mangle the hash if you are copying and pasting the hash for a dependency, at least currently. Dependency hashes are a sore spot tbh, and needs to be better) Then when you are using said dependency in your build.zig, the function provided will refer to that source artifact in .cache. At least this seems to be the case. I write a decent amount of zig, but haven’t dove too much in to the build system till recently, when I tried (and failed due to translate-c bugs), to get some C libraries added to zig using only zig build.
  - Cloudef 3y ago
    zig fetch fills the hash for you, or simply omitting .hash in zon and the compiler tells it for you. I'm writing backend in zig and I have zstd directly from zon as a dependency. .dependencies = .{ .zstd = .{ .url = "git+https://github.com/facebook/zstd.git#v1.5.5", .hash = "1220185ad79a437fd9f148d1422ff756287534c79a0712105039b4034031480e41a9", }, }, You then refer to that with dep = b.dependency(...) and can get paths to the unpacked source with dep.path(...)

vlovich123 3y ago

Interestingly you can use zig within your Rust build to more easily cross-compile: https://github.com/rust-cross/cargo-zigbuild https://github.com/rust-cross/cargo-zigbuild I’m curious what the blockers are for rustc to cross-compile like zig does natively.

adastra22 3y ago
Just add another arch with rustup. Am I missing something?
- weinzierl 3y ago
  That would solve the compilation problem, but there is more to create a working binary. For MacOS the official (and I believe only legal) way is to use a Mac with XCode installed for linking. Now, you can download all the necessary files from a Mac and build a cross-compilation toolchain on your Linux system. I believe you could not legally distribute a project doing this and that is why these projects don't exist or are usually short-lived (with the notable exception of zig). We will see how that goes for OP. So, solutions like osxcross resort to shipping scripts that help you to acquire the necessary files and make that process easier. The OP builds on osxcross, but is "batteries included". zig has an even more difficult problem to solve because it tries to compile and link for many platforms. Shipping all the different requirements in their original form would make a zig distribution huge. So it does some clever magic I do not fully understand to create the necessary files from just the necessary subset of data. This reduces the size because there is overlap between platforms. It also means that they are not shipping the Mac files in their complete and original form and have gotten away with this legally so far. At least that is what I believe is happening. I hope someone with more knowledge about zig could explain it better.
  - adastra22 3y ago
    You don’t need the OSX SDK to make runnable Rust binaries from other OS: https://betterprogramming.pub/cross-compiling-rust-from-mac-to-linux-7fad5a454ab1 https://betterprogramming.pub/cross-compiling-rust-from-mac-...
    - weinzierl 3y ago
      The link is about making Linux binaries on MacOS, which works because almost everything to build for Linux is already on OSX. The other way around it's not true. A Linux system does come neither with the OSX SDK nor the Apple patched llvm linker. I believe at least part of the required files cannot be legally distributed with a Linux system, but I might be wrong on that.
      - adastra22 3y ago
        It appears you’re right, thank you! I’m very surprised there isn’t a llvm linker available, minus all the foundation kits that rust doesn’t need.
        weinzierl 3y ago
        The llvm linker is a cross-linker in principle. I think it is more about the specialities that Apple needs, like code signing.
        adastra22 3y ago
        Ah. For what it's worth, those have been reverse engineered and copied elsewhere. I maintain a project that does cross-compiled macOS builds and our own signing on Linux.
- vlovich123 3y ago
  I could be mistaken, but I believe that installs a completely parallel toolchain which is relatively large compared with zigs approach of having all the platforms generatable from a single toolchain.
- pornel 3y ago
  Rustup arch only adds support for static libraries (unliked objects), and craps out as soon as you try to build a binary. Rust did not do the legwork that zig did to bundle libc and a working linker, and Cargo is exceptionally naive in its default configuration, so it won't even find a usable cross-linker on the system, nor even try the chronically-unfinished rustc-lld integration there is.
weinzierl 3y ago
There is cargo-zigbuild, which uses zig as a linker, but I have yet to try it out.
shepherdjerred 3y ago
That's actually exactly how this cross-compiler works for Rust!
jonathanhrl 3y ago
AFAIK, there are no blockers really, it's just that Rust does not have its own linker, it delegates linkage to the system linker depending on the target. <https://rust-lang.github.io/rustup/cross-compilation.html?highlight=linker#cross-compilation https://rust-lang.github.io/rustup/cross-compilation.html?hi...> You can specify your own linker if you want, mold is a very popular one, and cargo-zigbuild does the same behind the scenes with zig cc as the linker. I did something similar a couple of months ago (or a year ago? I don't remember exactly). I managed to cross-compile to windows-msvc on Linux using Wine, there's a project that provides the scripts to make this easier, including the linker wrapper: <https://github.com/est31/msvc-wine-rust https://github.com/est31/msvc-wine-rust>. It was just for fun because Rust can already target windows-gnu and it'll use mingw64 linker. Rust's approach to things is normally to provide the basic foundation and let the community build on top of it. I personally like this approach, but it also has this downside of people not knowing they may need an external/community built tool to accomplish what they want.
wyldfire 3y ago
> I’m curious what the blockers are for rustc to cross-compile like zig does natively. Zig ships all the bits you need for a C/C++ toolchain including things like the C library in some cases (IIUC). Rust could use libclang to make a "rustc cc" just like zig's. But I get the sense that it is probably not a goal of the project to have this functionality.
Cloudef 3y ago
Rust proc macros and the fact you can't really static link if something uses them made rust unsuitable for me.

menduz 3y ago

I'm commenting to add a critical mass around Zig. If you are prisoner of CMake, autoconf, templates, makefiles, etc.. please try the Zig toolchain for your c/c++ projects. It is worth every second of investment

DragonStrength 3y ago
I’m intrigued. Do you have a starting point? And how does it handle if I need to integrate with another language as well, like Swift?
- rockwotj 3y ago
  I was researching this the other day, here is a non trivial example: https://github.com/floooh/sokol-tools/blob/master/build.zig https://github.com/floooh/sokol-tools/blob/master/build.zig Here is a tutorial: https://www.zvm.app/tutorials/zig-build-cpp.html https://www.zvm.app/tutorials/zig-build-cpp.html
wredue 3y ago
Zig build is somewhat poorly documented currently, which can lead to some annoyance (also some other annoyances with dependency hashes), but in my experimentation: Dependencies in build.zig.zon are downloaded in to ~/.cache/zig/p/<hash> (incidentally, this means you need to mangle the hash if you are copying and pasting the hash for a dependency, at least currently. Dependency hashes are a sore spot tbh, and needs to be better) Then when you are using said dependency in your build.zig, the function provided will refer to that source artifact in .cache. At least this seems to be the case. I write a decent amount of zig, but haven’t dove too much in to the build system till recently, when I tried (and failed due to translate-c bugs), to get some C libraries added to zig using only zig build.
- Cloudef 3y ago
  zig fetch fills the hash for you, or simply omitting .hash in zon and the compiler tells it for you. I'm writing backend in zig and I have zstd directly from zon as a dependency. .dependencies = .{ .zstd = .{ .url = "git+https://github.com/facebook/zstd.git#v1.5.5", .hash = "1220185ad79a437fd9f148d1422ff756287534c79a0712105039b4034031480e41a9", }, }, You then refer to that with dep = b.dependency(...) and can get paths to the unpacked source with dep.path(...)