6 ms·

What does “LLVM implementation of Libc” mean here? Does that just mean an implementation of libc maintained by the LLVM project?

fourier_mode 7y ago
Yeah, I am also confused. Isn't there Clang already? (Please correct me if I am wrong)
- wyldfire 7y ago
  The C library is generally considered a part of the OS and the overall toolchain. Clang is "merely" a C compiler. The C library provides implementations for the platform-neutral language functions like "open/read/write/close/ioctl/dup/dup2" and more.
  - fourier_mode 7y ago
    So implementations of those subroutines are written in assembly lang?
    - wyldfire 7y ago
      No, it's very common for them to be written in primarily C. But I'm obligated to mention (as a member of the Rust Evangelism Strike Force) that you don't have to write them in C. ;) Many implementations do have performance-critical parts be target-dependent and implemented in assembly. IIRC glibc has more of this than musl/newlib/bionic, but it's been a while since I last looked.
      - fourier_mode 7y ago
        > No, it's very common for them to be written in primarily C Then can't you just get the LLVM IR out of it using Clang?
        wyldfire 7y ago
        I think that's immaterial but you're right that you could dump the IR. For the most part, C is a much easier language to program in than the IR. The IR is also very unstable: it changes from release to release.
        ploxiln 7y ago
        The root of the issue is that, while you can write most of the code in C (but not all), it has to be different for different operating system kernels. It will use different syscall numbers, argument size and order and "extra options" will be different, some kernels will have a more general syscall with options that enable it to be used for a handful of similar standard libc wrapper functions ... On macOS, you're really supposed to always dynamically link to the libSystem.dylib libc implementation, because the macOS kernel has subtle backwards-incompatible changes in the raw syscall interfaces on every macOS release (maybe even in point-release updates, I'm not sure). Go tried to have its own internal static libc replacement for macOS like it does for Linux, but just a year or two ago Go gave that up, and now dynamically links the macOS libc.
        asveikau 7y ago
        Somebody has to write the library to begin with. The typical Unix thing is for the OS to have one everybody links to. Which means, among other things, variation between libcs is a factor in portability of C programs from one OS to another.
    - deleted 7y ago
      [deleted]
  - asveikau 7y ago
    > platform-neutral language functions like "open/read/write/close/ioctl/dup/dup2" These are all syscall wrappers. They are present in libc but they are going to be very boring stubs that merely call into the kernel. (Also I don't think you can call them platform neutral or language functions as they are POSIX rather than being from the C standard...) The more "implementation-heavy" parts of a libc... Things like stdio, string calls, malloc, ...
    - caf 7y ago
      Those stubs have to correctly handle POSIX thread cancellation, which is "exciting" in the worst possible way. Some of them also have to translate between the userspace version of particular argument types and the kernel version, which don't always line up.
      - murderfs 7y ago
        pthread cancellation was a terrible idea when it was added, and it's even worse today. It's an API that provides no actual value (you can implement it yourself via signals), is a giant minefield if you care about not leaking or deadlocking, and infects the entirety of libc with pointless checks for functionality that no one in their right mind should use.
        yc12340 7y ago
        > pthread cancellation was a terrible idea when it was added, and it's even worse today > you can implement it yourself via signals It appears, that you are contradicting yourself. Either it is "bad idea" or it is easy to correctly implement by oneself, not both. musl has well though-out implementation of pthread cancellation, so it is clearly doable, even if glibc developers have failed at it.
        murderfs 7y ago
        The problem with pthread cancellation is that it is fundamentally broken when used on a thread that can ever acquire resources (opening an fd, taking a mutex, mallocing a buffer, etc.). If it were just "call pthread_testcancel to figure out if you were cancelled", that would be fine, but no, you either get blown up immediately, or you get blown up at arbitrary function call boundaries (that don't really make any sense; e.g. why is strerror_r allowed to be a cancellation point?) It's easy to correctly implement the parts that aren't just adding a call to pthread_testcancel to every single syscall wrapper, just reserve an RT signal and do your thread teardown when you receive it, using pthread_sigmask to implement enable/disable. It's just that it's just a terrible idea.
  - pjmlp 7y ago
    That is an UNIX thing, due to their symbiotic relationship, in every other OS out there the C library is whatever comes with the C compiler.
- Pyxl101 7y ago
  Clang is a C language family frontend for LLVM. That is, it's a compiler for C and C++ and related languages. Libc is the C standard library, and there are multiple implementations of it, e.g. glibc and musl. Clang does not include an implementation of libc, though it does have an implementation of the C++ standard library called libc++: https://libcxx.llvm.org/ https://libcxx.llvm.org/ It's not immediately clear to me why an LLVM implementation of the C++ standard library would be desirable while an implementation of the C standard library would be undesirable.
  - asveikau 7y ago
    > It's not immediately clear to me why an LLVM implementation of the C++ standard library would be desirable while an implementation of the C standard library would be undesirable. Worth noting the c++ standard library has had a lot of changes in recent years. libc has been hugely more stable in the same timeframe. (Notably there was C11, but the changes since C99 are not nearly what has been going on in c++ land.) The odds that whatever libc you already have is good enough are very high.
noncoml 7y ago
Sounds like an LLVM "recommended" libc? Loved the third point in Rich's reply: "there is tremendous value in non-monoculture of libc implementations, or implementations of any important library interfaces or language runtimes. Likewise there's tremendous value in non-monoculture of tooling (compilers, linkers, etc.). Avoiding monoculture preserves the motivation for consensus-based standards processes rather than single-party control (see also: Chrome and what it's done to the web) and the motivation for people writing software to write to the standards rather than to a particular implementation. A big part of making that possible is clear delineation of roles between parts of the toolchain and runtime, with well-defined interface boundaries. Some folks have told me that I should press LLVM to make musl the "LLVM libc" instead of whatever Google wants to do, but that misses the point: there shouldn't be a "LLVM libc", or any one library implementation that's "first class" for use with LLVM while others are only "second class"."
- wyldfire 7y ago
  > there is tremendous value in non-monoculture of libc implementations > any one library implementation that's "first class" for use with LLVM while others are only "second class"." I agree with Zachary here. LLVM's really good at mixing and matching w/target libs IMO (and linkers too, while we're at it). FWIW lld and libc++/abi have been a part of the llvm project for a long time now and BFD ld and libstdc++ are still the defaults for the linux clang driver.
  - bigcheesegs 7y ago
    Correct. clang is always going to default to targeting whatever the platform ecosystem is. I don't see linux distros moving away from glibc any time soon. The notion that a LLVM libc would in some way make other libc's "second class" is completely unfounded. libc++ vs. libstdc++ is proof of this.
- taeric 7y ago
  The benefits of not having a monoculture resonate heavily with me. I'm convinced it is in line with antifragility. Odd that so much of the goals folks make involve building monocultures.

fourier_mode 7y ago

Yeah, I am also confused. Isn't there Clang already? (Please correct me if I am wrong)

wyldfire 7y ago
The C library is generally considered a part of the OS and the overall toolchain. Clang is "merely" a C compiler. The C library provides implementations for the platform-neutral language functions like "open/read/write/close/ioctl/dup/dup2" and more.
- fourier_mode 7y ago
  So implementations of those subroutines are written in assembly lang?
  - wyldfire 7y ago
    No, it's very common for them to be written in primarily C. But I'm obligated to mention (as a member of the Rust Evangelism Strike Force) that you don't have to write them in C. ;) Many implementations do have performance-critical parts be target-dependent and implemented in assembly. IIRC glibc has more of this than musl/newlib/bionic, but it's been a while since I last looked.
    - fourier_mode 7y ago
      > No, it's very common for them to be written in primarily C Then can't you just get the LLVM IR out of it using Clang?
      - wyldfire 7y ago
        I think that's immaterial but you're right that you could dump the IR. For the most part, C is a much easier language to program in than the IR. The IR is also very unstable: it changes from release to release.
      - ploxiln 7y ago
        The root of the issue is that, while you can write most of the code in C (but not all), it has to be different for different operating system kernels. It will use different syscall numbers, argument size and order and "extra options" will be different, some kernels will have a more general syscall with options that enable it to be used for a handful of similar standard libc wrapper functions ... On macOS, you're really supposed to always dynamically link to the libSystem.dylib libc implementation, because the macOS kernel has subtle backwards-incompatible changes in the raw syscall interfaces on every macOS release (maybe even in point-release updates, I'm not sure). Go tried to have its own internal static libc replacement for macOS like it does for Linux, but just a year or two ago Go gave that up, and now dynamically links the macOS libc.
      - asveikau 7y ago
        Somebody has to write the library to begin with. The typical Unix thing is for the OS to have one everybody links to. Which means, among other things, variation between libcs is a factor in portability of C programs from one OS to another.
  - deleted 7y ago
    [deleted]
- asveikau 7y ago
  > platform-neutral language functions like "open/read/write/close/ioctl/dup/dup2" These are all syscall wrappers. They are present in libc but they are going to be very boring stubs that merely call into the kernel. (Also I don't think you can call them platform neutral or language functions as they are POSIX rather than being from the C standard...) The more "implementation-heavy" parts of a libc... Things like stdio, string calls, malloc, ...
  - caf 7y ago
    Those stubs have to correctly handle POSIX thread cancellation, which is "exciting" in the worst possible way. Some of them also have to translate between the userspace version of particular argument types and the kernel version, which don't always line up.
    - murderfs 7y ago
      pthread cancellation was a terrible idea when it was added, and it's even worse today. It's an API that provides no actual value (you can implement it yourself via signals), is a giant minefield if you care about not leaking or deadlocking, and infects the entirety of libc with pointless checks for functionality that no one in their right mind should use.
      - yc12340 7y ago
        > pthread cancellation was a terrible idea when it was added, and it's even worse today > you can implement it yourself via signals It appears, that you are contradicting yourself. Either it is "bad idea" or it is easy to correctly implement by oneself, not both. musl has well though-out implementation of pthread cancellation, so it is clearly doable, even if glibc developers have failed at it.
        murderfs 7y ago
        The problem with pthread cancellation is that it is fundamentally broken when used on a thread that can ever acquire resources (opening an fd, taking a mutex, mallocing a buffer, etc.). If it were just "call pthread_testcancel to figure out if you were cancelled", that would be fine, but no, you either get blown up immediately, or you get blown up at arbitrary function call boundaries (that don't really make any sense; e.g. why is strerror_r allowed to be a cancellation point?) It's easy to correctly implement the parts that aren't just adding a call to pthread_testcancel to every single syscall wrapper, just reserve an RT signal and do your thread teardown when you receive it, using pthread_sigmask to implement enable/disable. It's just that it's just a terrible idea.
- pjmlp 7y ago
  That is an UNIX thing, due to their symbiotic relationship, in every other OS out there the C library is whatever comes with the C compiler.
Pyxl101 7y ago
Clang is a C language family frontend for LLVM. That is, it's a compiler for C and C++ and related languages. Libc is the C standard library, and there are multiple implementations of it, e.g. glibc and musl. Clang does not include an implementation of libc, though it does have an implementation of the C++ standard library called libc++: https://libcxx.llvm.org/ https://libcxx.llvm.org/ It's not immediately clear to me why an LLVM implementation of the C++ standard library would be desirable while an implementation of the C standard library would be undesirable.
- asveikau 7y ago
  > It's not immediately clear to me why an LLVM implementation of the C++ standard library would be desirable while an implementation of the C standard library would be undesirable. Worth noting the c++ standard library has had a lot of changes in recent years. libc has been hugely more stable in the same timeframe. (Notably there was C11, but the changes since C99 are not nearly what has been going on in c++ land.) The odds that whatever libc you already have is good enough are very high.

noncoml 7y ago

Sounds like an LLVM "recommended" libc? Loved the third point in Rich's reply: "there is tremendous value in non-monoculture of libc implementations, or implementations of any important library interfaces or language runtimes. Likewise there's tremendous value in non-monoculture of tooling (compilers, linkers, etc.). Avoiding monoculture preserves the motivation for consensus-based standards processes rather than single-party control (see also: Chrome and what it's done to the web) and the motivation for people writing software to write to the standards rather than to a particular implementation. A big part of making that possible is clear delineation of roles between parts of the toolchain and runtime, with well-defined interface boundaries. Some folks have told me that I should press LLVM to make musl the "LLVM libc" instead of whatever Google wants to do, but that misses the point: there shouldn't be a "LLVM libc", or any one library implementation that's "first class" for use with LLVM while others are only "second class"."

wyldfire 7y ago
> there is tremendous value in non-monoculture of libc implementations > any one library implementation that's "first class" for use with LLVM while others are only "second class"." I agree with Zachary here. LLVM's really good at mixing and matching w/target libs IMO (and linkers too, while we're at it). FWIW lld and libc++/abi have been a part of the llvm project for a long time now and BFD ld and libstdc++ are still the defaults for the linux clang driver.
- bigcheesegs 7y ago
  Correct. clang is always going to default to targeting whatever the platform ecosystem is. I don't see linux distros moving away from glibc any time soon. The notion that a LLVM libc would in some way make other libc's "second class" is completely unfounded. libc++ vs. libstdc++ is proof of this.
taeric 7y ago
The benefits of not having a monoculture resonate heavily with me. I'm convinced it is in line with antifragility. Odd that so much of the goals folks make involve building monocultures.