12 ms·

I recently did some experiments with creating small static Rust binaries, custom linking, no_std et cetera. A lot of stuff around that kind of thing is unstable

I recently did some experiments with creating small static Rust binaries, custom linking, no_std et cetera. A lot of stuff around that kind of thing is unstable or unfinished, which might be somewhat expected. But I’ve also come to the conclusion that Rust relies on libc way too much. That might be fine on Linux, where GNU’s libc is well-maintained, is a bit questionable on MacOS (as seen in this article) and is a a complete distribution nightmare on Windows (in no small part due to a series of very questionable decisions by Microsoft). My understanding is that Go doesn’t use the libc at all and makes system calls directly, which IMO is the correct decision in a modern systems programming language that doesn’t want to be limited by 40 years of cruft.

fluffything 6y ago
> But I’ve also come to the conclusion that Rust relies on libc way too much. How did you come to this conclusion? People using Rust rely on libc a lot. For example, #![no_std] means "no standard-library", but it doesn't mean "no libc, no libunwind, etc.". So a lot of people like to advertise their crates as "#![no_std]" compatible, because they compile with #![no_std], but then the first thing the crate does is linking against libc, against liballoc, against libm, .... or even the standard library itself... So... if you are trying to build a binary that does not depend on libc or libstd, then there is no language feature to help you there. #[no_std] binaries are not only unstable, but also probably not what you want, since that won't prevent you from linking any library that links libc, or the standard library, etc. If you want to avoid libc, you have to enforce that, e.g., by checking in your build system that your binary doesn't contain any libc, libstd, etc. symbols (I just use nm for this). #![no_std] helps a bit here, but making sure you don't link any library that violates this is up to you.
- codeflo 6y ago
  That’s true. But going the no_std route is very hard (the ecosystem isn’t big, and relying cargo and crates.io you’re almost guaranteed to link in the std or liballoc by accident at some point). Even when using the std intentionally, I really wouldn’t have expected that basic std functions like println require the libc.
  - geofft 6y ago
    I would not necessarily expect it but I appreciate it in a "systems" language where "systems" is defined as compatible with the existing traditional systems software on a machine. For example, it's nice if the stdbuf(1) command works on Rust binaries on glibc systems, and it's nice if a Rust binary calling a C library that writes with printf (or vice versa) don't maintain two separate output buffers. To me, Go is the systems programming language for a world untethered by existing platform compatibility (and so, for instance, writing Go libraries to be called from C is awkward, calling C libraries from Go incurs overhead, Go has its own concurrency model, etc.) and Rust is the systems programming language for use cases where you'd otherwise want to use C (really "the platform's native systems language," but that's C on all the major platforms) but you want a better language. I appreciate that they both exist and target these different use cases.
    - BuckRogers 6y ago
      Is Go widely used compared to the others as a systems language at this point? Over the years I’ve gathered that it’s more of a competitor for C# & Java rather than Rust & C(++).
      - geofft 6y ago
        Depends what you mean by "systems language" (many common definitions turn out to be equivalent to "drop-in replacement for the platform language," which makes the argument circular), but the choice of Go as an implementation language for Docker and Kubernetes puts it pretty firmly in the "systems language" space IMO. Container management requires more systems-level functionality than C# and Java are really geared towards (although you certainly could use them, perhaps with a tablespoon of FFI).
        apta 6y ago
        The original version of Docker was written in Python, and the original version of Kubernetes was written in Java, so your argument doesn't hold.
- geofft 6y ago
  To me, a #![no_std] library is useful for contexts where there isn't a platform libc (embedded systems, kernels, etc.) and you can't assume things like the existence of stdin or processes. On those systems, there may still be a dynamic allocator, because that's a super common feature in all but the most spartan environments. For those use cases, a #![no_std] library that links liballoc (and documents that it needs it) is totally okay, and often better than a more-limited library that only does static allocation - or conversely implementing all of libstd with runtime panics for most of it, just because your library needs to pass a Vec around. It's a balance. The only case I think I've seen where a #![no_std] library ends up pulling in libc is if you haven't added a custom allocator and your platform's default allocator uses libc (and so you could switch to a libc-free allocator if you want). Are there other cases?
  - codeflo 6y ago
    There are lots of tiny practical annoyances you notice, starting with having to vet dependencies and transitive dependency upgrades extremely carefully. A major one is that dev-dependencies are very broken in a no_std environment. It’s obviously useful to have the std in your build.rs script, but unfortunately, Cargo merges dependencies and devDependencies in a way that’s simply incorrect. The good news is that every problem I found had an actively discussed GitHub issue and the community is active, so there will be progress.
    - geofft 6y ago
      > unfortunately, Cargo merges dependencies and devDependencies in a way that’s simply incorrect. Yeah, I've run into that, but IIRC this got solved super recently/is being solved. Probably https://github.com/rust-lang/cargo/pull/7820 https://github.com/rust-lang/cargo/pull/7820 ?
  - fluffything 6y ago
    > To me, a #![no_std] library is useful for contexts where there isn't a platform libc (embedded systems, kernels, etc.) and you can't assume things like the existence of stdin or processes. "Could be useful for" (FTFY). Unfortunately, for the reasons mentioned above, most #![no_std] libraries aren't useful for that, because they link libc and other libraries. Most people don't do this intentionally. The compiler doesn't complain about: #![no_std] extern "C" { fn malloc(...) -> ...; } so when somebody does it accidentally, everything works and they get no feedback. When you then go and build a #![no_std] binary, and run it in a platform when libc is not available, only then you get an error. And at that point, good luck figuring out which of your 200 dependencies has the error. In particular, if you are running `#[test]`s, doing that links standard, so if your program implicitly depends on libc somewhere, tests won't reveal that, because while testing, libc will be linked.
ekidd 6y ago
As far as I know, the official system interface on Windows and several Unix systems is via the standard library, not via direct syscalls. I don't know about the MacOS. But in general, you may be required to dynamically link the standard library on many platforms. Linux guarantees syscalls are stable. And on Linux, you have the option of telling Rust to cross-compile using a statically-linked musl-libc. (If you also need to statically link OpenSSL or a few other common libraries, I maintain https://github.com/emk/rust-musl-builder https://github.com/emk/rust-musl-builder, and there's at least one similar image out there.)
- majewsky 6y ago
  macOS requires you to make syscalls through libSystem if you want a stable interface. Go binaries used to make direct syscalls until 1.10. Since this caused major breakage on most new macOS releases, they have since switched to using libSystem as well in 1.11: > On macOS and iOS, the runtime now uses libSystem.dylib instead of calling the kernel directly. This should make Go binaries more compatible with future versions of macOS and iOS. The syscall package still makes direct system calls; fixing this is planned for a future release. Source: https://golang.org/doc/go1.11 https://golang.org/doc/go1.11
- codeflo 6y ago
  That’s partly true, but the stable interface on Windows is not the libc, but the kernel32/user32 functions like VirtualAlloc, CreateFileW etc. Those are stable since the NT days. The libc functions like malloc and fopen are a compatibility layer above that and unfortunately switch places every few years. Currently they are delivered by a combination of a Windows update and a redistributable package, which makes it a nightmare to ship (on pre-Windows 10 even more so).
  - jfkebwjsbx 6y ago
    Do you really need to ship them? I thought libc (CRT?) in Windows was a given, and what used to be redistributed was only the C++ ones. Is not that the case?
    - 6y ago
steveklabnik 6y ago
The problem with this is that not every system makes their system call ABI stable. You have two choices here: use the interface that is stable (which is libc), or track changes and fix things up when they break.
- codeflo 6y ago
  The only stable interface on Windows are the documented functions from kernel32.dll, user32.dll etc. Libc is a compatibility layer above that, that Microsoft invents a new incompatible distribution mechanism for every 3-5 years. It’s pure DLL hell unfortunately. Edit: Not even the DLL name of Microsoft’s libc is stable (msvcrt140.dll etc.), leading to all kinds of wild goose chases when trying to run old binaries.
  - steveklabnik 6y ago
    Yes, I was being imprecise wrt Windows, thanks :)
    - codeflo 6y ago
      Yeah, I saw your username too late or I would have known that you know that. ;) But it’s a common misunderstanding among many Unix programmers, so I feel it was good to clarify.
      - steveklabnik 6y ago
        Nah I think it's helpful! Comments are read by a lot more folks than just the person you're replying to. I knew it wasn't exactly libc but I didn't know the full hierarchy involved.
weff_ 6y ago
I think Go tries to reduce its dependence on libc but, by default, it will still link to it. For instance, this code: package main import "net" func main(){ net.Dial("tcp", "golang.org:80") } When compiled with go build main.go does link: linux-vdso.so.1 (0x00007ffe3d7f0000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc7ac05a000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc7abc69000) /lib64/ld-linux-x86-64.so.2 (0x00007fc7ac279000) There are of course compiler options to truly statically compile.
fortran77 6y ago
This is simply false. Windows APIs are stable. It's one of the best platforms for back compatibility. The kernel32/user32 API has been stable for 20 years. Rust has some work to do.
- codeflo 6y ago
  That’s exactly my point: The problem is that Rust doesn’t use the stable kernel32/user32 functions (VirtualAlloc etc.), but the libc ones, which don’t have a stable location on Windows. Without looking it up: Which DLL do you have to redistribute this week to get “malloc”?
  - fortran77 6y ago
    I just tell Visual Studio to create an installer, I select the set of target platforms, and it just works. See, for example: https://docs.microsoft.com/en-us/cpp/ide/walkthrough-deploying-your-program-cpp?view=vs-2019 https://docs.microsoft.com/en-us/cpp/ide/walkthrough-deployi...
rhinoceraptor 6y ago
Go’s pathological NIH syndrome does come with downsides. For example, there was an infamous memory corruption bug in the way they called into vDSO.
- adev_ 6y ago
  > Go’s pathological NIH syndrome does come with downsides. Yes. And you can add the inability to use the glibc's nss modules under Linux. Making it unable to use sssd properly and authenticate a posix user on a machine with LDAP authentication. Getting completely independent from OS sys lib has consequences
  - alecthomas 6y ago
    This is not accurate. [1] When compiled on Linux for Linux, Go will use libc and natively call NSS. When cross-compiling to Linux from another system, Go requires (mostly) CGO to be disabled and a subset of NSS will be implemented in Go. Native NSS modules will not work. [1] https://github.com/golang/go/issues/24083 https://github.com/golang/go/issues/24083
cesarb 6y ago
> [...] relies on libc way too much. That might be fine on Linux [...] > My understanding is that Go doesn’t use the libc at all and makes system calls directly Actually, the only system on which it's fine to "not use the libc at all and make system calls directly" is Linux. On MacOS, Windows, and most non-Linux Unix-like systems, you must to go through the libc or its equivalent (which on Windows is kernel32.dll and/or ntdll.dll), since the system call interface is unstable (the libc or its equivalent is distributed together with the kernel, so it's updated whenever the system call interface changes). AFAIK, Go tried for a while to use system calls directly on MacOS; after being broken several times by operating system updates, they gave up and now go through the shared libraries like everyone else. They still insist on using direct system calls on Linux, where it works mostly fine (except for things like the DNS resolver, in which AFAIK they try to guess whether they can do directly network DNS requests, or have to go through libc; and that one time in which Go's small stacks conflicted with a misoptimized kernel VDSO using too much stack).

fluffything 6y ago

> But I’ve also come to the conclusion that Rust relies on libc way too much. How did you come to this conclusion? People using Rust rely on libc a lot. For example, #![no_std] means "no standard-library", but it doesn't mean "no libc, no libunwind, etc.". So a lot of people like to advertise their crates as "#![no_std]" compatible, because they compile with #![no_std], but then the first thing the crate does is linking against libc, against liballoc, against libm, .... or even the standard library itself... So... if you are trying to build a binary that does not depend on libc or libstd, then there is no language feature to help you there. #[no_std] binaries are not only unstable, but also probably not what you want, since that won't prevent you from linking any library that links libc, or the standard library, etc. If you want to avoid libc, you have to enforce that, e.g., by checking in your build system that your binary doesn't contain any libc, libstd, etc. symbols (I just use nm for this). #![no_std] helps a bit here, but making sure you don't link any library that violates this is up to you.

codeflo 6y ago
That’s true. But going the no_std route is very hard (the ecosystem isn’t big, and relying cargo and crates.io you’re almost guaranteed to link in the std or liballoc by accident at some point). Even when using the std intentionally, I really wouldn’t have expected that basic std functions like println require the libc.
- geofft 6y ago
  I would not necessarily expect it but I appreciate it in a "systems" language where "systems" is defined as compatible with the existing traditional systems software on a machine. For example, it's nice if the stdbuf(1) command works on Rust binaries on glibc systems, and it's nice if a Rust binary calling a C library that writes with printf (or vice versa) don't maintain two separate output buffers. To me, Go is the systems programming language for a world untethered by existing platform compatibility (and so, for instance, writing Go libraries to be called from C is awkward, calling C libraries from Go incurs overhead, Go has its own concurrency model, etc.) and Rust is the systems programming language for use cases where you'd otherwise want to use C (really "the platform's native systems language," but that's C on all the major platforms) but you want a better language. I appreciate that they both exist and target these different use cases.
  - BuckRogers 6y ago
    Is Go widely used compared to the others as a systems language at this point? Over the years I’ve gathered that it’s more of a competitor for C# & Java rather than Rust & C(++).
    - geofft 6y ago
      Depends what you mean by "systems language" (many common definitions turn out to be equivalent to "drop-in replacement for the platform language," which makes the argument circular), but the choice of Go as an implementation language for Docker and Kubernetes puts it pretty firmly in the "systems language" space IMO. Container management requires more systems-level functionality than C# and Java are really geared towards (although you certainly could use them, perhaps with a tablespoon of FFI).
      - apta 6y ago
        The original version of Docker was written in Python, and the original version of Kubernetes was written in Java, so your argument doesn't hold.
geofft 6y ago
To me, a #![no_std] library is useful for contexts where there isn't a platform libc (embedded systems, kernels, etc.) and you can't assume things like the existence of stdin or processes. On those systems, there may still be a dynamic allocator, because that's a super common feature in all but the most spartan environments. For those use cases, a #![no_std] library that links liballoc (and documents that it needs it) is totally okay, and often better than a more-limited library that only does static allocation - or conversely implementing all of libstd with runtime panics for most of it, just because your library needs to pass a Vec around. It's a balance. The only case I think I've seen where a #![no_std] library ends up pulling in libc is if you haven't added a custom allocator and your platform's default allocator uses libc (and so you could switch to a libc-free allocator if you want). Are there other cases?
- codeflo 6y ago
  There are lots of tiny practical annoyances you notice, starting with having to vet dependencies and transitive dependency upgrades extremely carefully. A major one is that dev-dependencies are very broken in a no_std environment. It’s obviously useful to have the std in your build.rs script, but unfortunately, Cargo merges dependencies and devDependencies in a way that’s simply incorrect. The good news is that every problem I found had an actively discussed GitHub issue and the community is active, so there will be progress.
  - geofft 6y ago
    > unfortunately, Cargo merges dependencies and devDependencies in a way that’s simply incorrect. Yeah, I've run into that, but IIRC this got solved super recently/is being solved. Probably https://github.com/rust-lang/cargo/pull/7820 https://github.com/rust-lang/cargo/pull/7820 ?
- fluffything 6y ago
  > To me, a #![no_std] library is useful for contexts where there isn't a platform libc (embedded systems, kernels, etc.) and you can't assume things like the existence of stdin or processes. "Could be useful for" (FTFY). Unfortunately, for the reasons mentioned above, most #![no_std] libraries aren't useful for that, because they link libc and other libraries. Most people don't do this intentionally. The compiler doesn't complain about: #![no_std] extern "C" { fn malloc(...) -> ...; } so when somebody does it accidentally, everything works and they get no feedback. When you then go and build a #![no_std] binary, and run it in a platform when libc is not available, only then you get an error. And at that point, good luck figuring out which of your 200 dependencies has the error. In particular, if you are running `#[test]`s, doing that links standard, so if your program implicitly depends on libc somewhere, tests won't reveal that, because while testing, libc will be linked.

ekidd 6y ago

As far as I know, the official system interface on Windows and several Unix systems is via the standard library, not via direct syscalls. I don't know about the MacOS. But in general, you may be required to dynamically link the standard library on many platforms. Linux guarantees syscalls are stable. And on Linux, you have the option of telling Rust to cross-compile using a statically-linked musl-libc. (If you also need to statically link OpenSSL or a few other common libraries, I maintain https://github.com/emk/rust-musl-builder https://github.com/emk/rust-musl-builder, and there's at least one similar image out there.)

majewsky 6y ago
macOS requires you to make syscalls through libSystem if you want a stable interface. Go binaries used to make direct syscalls until 1.10. Since this caused major breakage on most new macOS releases, they have since switched to using libSystem as well in 1.11: > On macOS and iOS, the runtime now uses libSystem.dylib instead of calling the kernel directly. This should make Go binaries more compatible with future versions of macOS and iOS. The syscall package still makes direct system calls; fixing this is planned for a future release. Source: https://golang.org/doc/go1.11 https://golang.org/doc/go1.11
codeflo 6y ago
That’s partly true, but the stable interface on Windows is not the libc, but the kernel32/user32 functions like VirtualAlloc, CreateFileW etc. Those are stable since the NT days. The libc functions like malloc and fopen are a compatibility layer above that and unfortunately switch places every few years. Currently they are delivered by a combination of a Windows update and a redistributable package, which makes it a nightmare to ship (on pre-Windows 10 even more so).
- jfkebwjsbx 6y ago
  Do you really need to ship them? I thought libc (CRT?) in Windows was a given, and what used to be redistributed was only the C++ ones. Is not that the case?
  - 6y ago

steveklabnik 6y ago

The problem with this is that not every system makes their system call ABI stable. You have two choices here: use the interface that is stable (which is libc), or track changes and fix things up when they break.

codeflo 6y ago
The only stable interface on Windows are the documented functions from kernel32.dll, user32.dll etc. Libc is a compatibility layer above that, that Microsoft invents a new incompatible distribution mechanism for every 3-5 years. It’s pure DLL hell unfortunately. Edit: Not even the DLL name of Microsoft’s libc is stable (msvcrt140.dll etc.), leading to all kinds of wild goose chases when trying to run old binaries.
- steveklabnik 6y ago
  Yes, I was being imprecise wrt Windows, thanks :)
  - codeflo 6y ago
    Yeah, I saw your username too late or I would have known that you know that. ;) But it’s a common misunderstanding among many Unix programmers, so I feel it was good to clarify.
    - steveklabnik 6y ago
      Nah I think it's helpful! Comments are read by a lot more folks than just the person you're replying to. I knew it wasn't exactly libc but I didn't know the full hierarchy involved.

weff_ 6y ago

I think Go tries to reduce its dependence on libc but, by default, it will still link to it. For instance, this code: package main import "net" func main(){ net.Dial("tcp", "golang.org:80") } When compiled with go build main.go does link: linux-vdso.so.1 (0x00007ffe3d7f0000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc7ac05a000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc7abc69000) /lib64/ld-linux-x86-64.so.2 (0x00007fc7ac279000) There are of course compiler options to truly statically compile.

fortran77 6y ago

This is simply false. Windows APIs are stable. It's one of the best platforms for back compatibility. The kernel32/user32 API has been stable for 20 years. Rust has some work to do.

codeflo 6y ago
That’s exactly my point: The problem is that Rust doesn’t use the stable kernel32/user32 functions (VirtualAlloc etc.), but the libc ones, which don’t have a stable location on Windows. Without looking it up: Which DLL do you have to redistribute this week to get “malloc”?
- fortran77 6y ago
  I just tell Visual Studio to create an installer, I select the set of target platforms, and it just works. See, for example: https://docs.microsoft.com/en-us/cpp/ide/walkthrough-deploying-your-program-cpp?view=vs-2019 https://docs.microsoft.com/en-us/cpp/ide/walkthrough-deployi...

rhinoceraptor 6y ago

Go’s pathological NIH syndrome does come with downsides. For example, there was an infamous memory corruption bug in the way they called into vDSO.

adev_ 6y ago
> Go’s pathological NIH syndrome does come with downsides. Yes. And you can add the inability to use the glibc's nss modules under Linux. Making it unable to use sssd properly and authenticate a posix user on a machine with LDAP authentication. Getting completely independent from OS sys lib has consequences
- alecthomas 6y ago
  This is not accurate. [1] When compiled on Linux for Linux, Go will use libc and natively call NSS. When cross-compiling to Linux from another system, Go requires (mostly) CGO to be disabled and a subset of NSS will be implemented in Go. Native NSS modules will not work. [1] https://github.com/golang/go/issues/24083 https://github.com/golang/go/issues/24083

cesarb 6y ago

> [...] relies on libc way too much. That might be fine on Linux [...] > My understanding is that Go doesn’t use the libc at all and makes system calls directly Actually, the only system on which it's fine to "not use the libc at all and make system calls directly" is Linux. On MacOS, Windows, and most non-Linux Unix-like systems, you must to go through the libc or its equivalent (which on Windows is kernel32.dll and/or ntdll.dll), since the system call interface is unstable (the libc or its equivalent is distributed together with the kernel, so it's updated whenever the system call interface changes). AFAIK, Go tried for a while to use system calls directly on MacOS; after being broken several times by operating system updates, they gave up and now go through the shared libraries like everyone else. They still insist on using direct system calls on Linux, where it works mostly fine (except for things like the DNS resolver, in which AFAIK they try to guess whether they can do directly network DNS requests, or have to go through libc; and that one time in which Go's small stacks conflicted with a misoptimized kernel VDSO using too much stack).