10 ms·

A proposal to start “llvm-libc” (2020)

TheBrokenRail 5y ago
I see this as nothing but a good thing. Especially since this seems like it can delegate to a system libc for necessary functions.
eatonphil 5y ago
When was this published? I can't tell if the last edit date at the bottom is for the entire site or this page.
- Arnavion 5y ago
  https://github.com/llvm/llvm-project/commits/main/llvm/docs/Proposals/LLVMLibC.rst https://github.com/llvm/llvm-project/commits/main/llvm/docs/... Merged in 2019-08, and hasn't been edited since. As linked elsewhere in the thread, https://github.com/llvm/llvm-project/tree/main/libc https://github.com/llvm/llvm-project/tree/main/libc indicates it's alive and well.
  - andyferris 5y ago
    16 August 2019
- deleted 5y ago
  [deleted]
Falell 5y ago
Sounds complicated, but the goal of a fuzzable/sanitizable/etc libc sounds nice. Lack of ABI stability sounds terrifying as an application developer. My other immediate thought was "how will this interact with systems where the OS-provided libc is the only stable way to e.g. make syscalls", and "Layering Over Another libc" addresses this. I guess the idea is you'd link an application against llvm-libc and the system libc, and ship llvm-libc with your application?
- jart 5y ago
  Cosmopolitan Libc is sanitizable. I believe it's currently the only one where you can use Address Sanitizer at all layers of the software stack.
- Arnavion 5y ago
  If your software is in the distro repos, the maintainer of said package queues it to be rebuilt when the distro's llvm-libc package is updated. If you're providing the packages yourself, it's up to you to do that yourself. Or yes, you can vendor libc in your package. Not something everyone will like, but it depends on who your users are. It's not like this is unusual. Binaries compiled against today's glibc can fail to run on a machine that hasn't been updated since last week because they rely on a new / different symbol. Rebuilding the distro's packages when their deps are updated is standard fare.
  - mananaysiempre 5y ago
    > Binaries compiled against today's glibc can fail to run on a machine that hasn't been updated since last week because they rely on a new / different symbol. Note, however, that it is a Glibc bug (modulo Drepper’s temper) if the reverse happens: Glibc symbol versioning ensures that binaries depending on an old Glibc (only) will run on a new one. So the proper way to build a maximally-compatible Linux executable would be to build a cross toolchain targeting an old Glibc and compile your code with it. Unfortunately, the build system is hell and old Glibcs doesn’t compile without backported patches, so while I did try to follow in the footsteps of a couple of people[1–5], I did not succeed. Mass rebuilds still happen with other ecosystems, though. GHC-compiled Haskell libraries are fine-grained and not ABI-stable across compiler versions, so my Arch box regularly gets hit with a deluge of teensy Haskell library updates, and Arch is currently undergoing a massive Python rebuild (blocking all other Python package updates) behind the scenes as well. [1]: https://github.com/wheybags/glibc_version_header https://github.com/wheybags/glibc_version_header (hack but easy and will probably work most of the time) [2]: https://www.lordaro.co.uk/posts/2018-08-26-compiling-glibc.html https://www.lordaro.co.uk/posts/2018-08-26-compiling-glibc.h... (someone’s mostly-nonhackish effort) [3]: https://github.com/pypa/manylinux https://github.com/pypa/manylinux (what Python manylinux wheels use, more modern than absolutely necessary) [4]: https://github.com/FooBarWidget/holy-build-box https://github.com/FooBarWidget/holy-build-box (ditto and is also a complete opaque cross-toolchain build recipe, but apparently people use that) [5]: https://casualhacking.io/blog/2018/12/25/create-highly-portable-elf-binaries-using-the-build-anywhere-toolchain https://casualhacking.io/blog/2018/12/25/create-highly-porta... (missed it last time, so can’t say much)
bch 5y ago
In the absence any any mention of licensing, I presume this is Apache-2… I don’t know if I’ve ever seen such a core piece of infra licensed that way (am used to BSD or MIT) - others might know better than I: is this a sane way forward?
- jcranmer 5y ago
  The license is here: <https://github.com/llvm/llvm-project/blob/main/libc/LICENSE.TXT https://github.com/llvm/llvm-project/blob/main/libc/LICENSE....>. It's Apache v2 with a compiler exception--i.e., the compiler linking bits of itself into you doesn't count for license purposes.
- Someone 5y ago
  It’s Apache licensed with LLVM exceptions (https://github.com/llvm/llvm-project/blob/main/libc/LICENSE.TXT https://github.com/llvm/llvm-project/blob/main/libc/LICENSE....) LibC++ is licensed that way, too (https://github.com/llvm/llvm-project/blob/main/libcxx/LICENSE.TXT https://github.com/llvm/llvm-project/blob/main/libcxx/LICENS...)
- mananaysiempre 5y ago
  The OpenBSD people in general and De Raadt specifically are probably the best-known objectors to Apache-2[1–4], among other things because they hold that Apache-2 is too broad to be a copyright license and thus has to be interpreted as a contract instead. I don’t know if they’re right, but their arguments did shift my opinion in their direction. Corporate lawyers seem to love it, though, because of the mutually-assured-patent-destruction clause. [1]: http://www.openbsd.org/policy.html http://www.openbsd.org/policy.html (see corresponding section) [2]: https://marc.info/?i=91077.1475036864%20()%20cvs%20!%20openbsd%20!%20org https://marc.info/?i=91077.1475036864%20()%20cvs%20!%20openb... (De Raadt rants on openbsd-misc, discussed on HN at https://news.ycombinator.com/item?id=126178810 https://news.ycombinator.com/item?id=126178810) [3]: https://lists.llvm.org/pipermail/llvm-dev/2017-April/112300.html https://lists.llvm.org/pipermail/llvm-dev/2017-April/112300.... (Kettenis objects on behalf of OpenBSD on llvm-dev) [4]: https://www.cambus.net/the-state-of-toolchains-in-openbsd/ https://www.cambus.net/the-state-of-toolchains-in-openbsd/ (OpenBSD gives up on staying with old LLVM)
flohofwoe 5y ago
> but take advantage and use C++ language facilities for the core implementation. This point sticks out. Would be nice to get some more details why it would make sense to use C++ for the implementation instead of C.
- maccard 5y ago
  Raii is enough of a reason IMO, but templates, type safety, actual arrays, constexpr, and various modern changes to the language that allow for the compiler to optimise code (move semantics and RVO)
  - dralley 5y ago
    Those are all fantastic reasons to use relibc, the libc written in Rust, which has the additional benefit of already existing. https://gitlab.redox-os.org/redox-os/relibc https://gitlab.redox-os.org/redox-os/relibc
    - maccard 5y ago
      Sorry I missed this reply. I do'nt disagree, but honestly at this point anything is better than using C for libraries like this.
  - mhh__ 5y ago
    Function overloading and more places to use default values was enough to make me go mental last time I had to use strict C as opposed to limited C++
- mhh__ 5y ago
  Other than platform support I think C is practically only a hindrance compared to C++ in the hands of competent developers.
- zemoo 5y ago
  The Microsoft C standard library implementation also takes this approach - C++ implementation with C exports, for the reasons called out by the sibling comments.
spullara 5y ago
Since this is now over 2 years old is there any progress?
- jcranmer 5y ago
  Yes, there is progress. But it is very far from anything you might consider complete or arguably even usable--there's no implementation of malloc or basic stdio, for example.
  - cclearwater 5y ago
    Wouldn't they just use Google's tcmalloc?
    - fyrn- 5y ago
      Probably not the ideal pick for an everuone facing library like this, tcmalloc focuses on reducing core thread contention, at the cost of compactness and often debugability. I've not been following the ptoject for some time though, so maybe things have changed. Still a great allocator if those tradoffs are what you want. Something more like mimalloc would probably be a better general purpose allocator for something like this
    - dxf 5y ago
      The plan is to use Scudo, which is the allocator that lives in compiler-rt. Edited to include the link to Scudo: https://github.com/llvm/llvm-project/tree/main/compiler-rt/lib/scudo https://github.com/llvm/llvm-project/tree/main/compiler-rt/l...
      - dxf 5y ago
        Edited again: I am reminded that you can already get scudo today by using a CMake option to include scudo when building LLVM-libc.
    - scrb 5y ago
      Users wanting allocator functions (malloc and friends) from LLVM libc will get the SCUDO allocator. They need to use a special CMake option to include SCUDO when building LLVM libc.
po1nt 5y ago
I've expected some plot twist like "It will be coded in Rust"
- guerrilla 5y ago
  Close... but take advantage and use C++ language facilities for the core implementation.
- kryptiskt 5y ago
  If you want that Redox has a libc coded in Rust: https://gitlab.redox-os.org/redox-os/relibc https://gitlab.redox-os.org/redox-os/relibc There is a bunch of Redox ports of C programs, so it is apparently good enough for a range of software. https://gitlab.redox-os.org/redox-os/cookbook/-/tree/master/recipes https://gitlab.redox-os.org/redox-os/cookbook/-/tree/master/...
ndesaulniers 5y ago
I hope one day that I can statically link against my libc easily one day. Not just statically link but even LTO my libc into my binary. I understand the "security" concerns, but I feel there are cases where they don't matter. Case in point; my C compiler. It already was not designed to handle untrusted input well enough and I plan to run multiple instances in parallel, so my kernel should handle COW.
- yjftsjthsd-h 5y ago
  You can do this today with musl, FWIW.
  - ndesaulniers 5y ago
    You can, and I have done it. It's easy for hello world. But for larger projects it's more difficult. If anyone has even done a build of clang with this, I'd love to know!
    - boogies 5y ago
      > If anyone has even done a build of clang with this, I'd love to know! Clang's in Alpine Linux's repo's, so I guess they have and this is how: https://git.alpinelinux.org/aports/tree/main/clang https://git.alpinelinux.org/aports/tree/main/clang
      - Aissen 5y ago
        They built clang against musl, just not statically.
    - Ericson2314 5y ago
      I made https://github.com/NixOS/nixpkgs/pull/149523 https://github.com/NixOS/nixpkgs/pull/149523 to kick off nix-build -A pkgsStatic.llvmPackages.clang-unwrapped Which built all the other deps just fine, but failed buildling LLVM with CONFIGURE_LLVM_NATIVE. That seems easy enough to fix, as that's the regular part, though. No guarantee there wouldn't be other more serious issues lurking beneath, but we've attempted quite a lot of static builds of complicated thing, augmented Musl with various bits and scraps to make it more featureful. You gave a very nice intro for my talk, saying Nixpkgs could well be the future so check it out. Well, I hope that future can arive more widely soon :).
pabs3 5y ago
I wonder if it will support versioned symbols for ABI stability like glibc does.
exikyut 5y ago
Hmm, my one question is: what platforms will this explicitly support? Linux, macOS, Windows, FreeBSD, and probably OpenBSD seem shoo-in table stakes. I'm more curious about - iOS/iPadOS: already have a libc, but... maybe? - Android: already has bionic/NDK; alternative useful? - NetBSD: rump kernel/unikernel applications? - VMS: has x86_64 support now; suddenly less irrelevant than before? - QNX: IIUC still the best deterministic/hard real time POSIX OS... - Illumos: not dead yet? - HP-UX/AIX: still used in certain industrial applications...? - Serenity: ...oh wait, just realized this isn't full POSIX, woops (would that be a prerequisite?) - (what obvious thing did I forget? :P) I ask this question mostly to update my understanding of "the state of kernel/OS interestingness, ~2022", since the process of deciding what targets a new major libc should consider relevant is going to be both well-informed and carefully considered given the anticipated (hoped) timescale of such a project.
- jcranmer 5y ago
  As of right now, it supports... Linux x86-64 and maybe aarch64. Although as of right now, the only OS-specific things it supports are a very stub loader for main, a thread library, and signal handling.
- indrora 5y ago
  Embedded targets: ARM6-ARM8, even older (STM32) in some cases. Consider the places where libiberty and the now very long in the tooth RedHat Newlib is in use. There's a lot of places where llvm is used but where a good libc is basically gone or is mostly implemented as a bunch of messy assembly routines.
- RantyDave 5y ago
  > Illumos: not dead yet? Not quite: https://omnios.org https://omnios.org
account42 5y ago
> Ability to layer this libc over the system libc if possible and desired for a platform. This sounds nice for being able to use newer C/C++ library functions when targeting an older system libc by statically linking the llvm-libc implementation of the missing functions.

I see this as nothing but a good thing. Especially since this seems like it can delegate to a system libc for necessary functions.

eatonphil 5y ago

When was this published? I can't tell if the last edit date at the bottom is for the entire site or this page.

Arnavion 5y ago
https://github.com/llvm/llvm-project/commits/main/llvm/docs/Proposals/LLVMLibC.rst https://github.com/llvm/llvm-project/commits/main/llvm/docs/... Merged in 2019-08, and hasn't been edited since. As linked elsewhere in the thread, https://github.com/llvm/llvm-project/tree/main/libc https://github.com/llvm/llvm-project/tree/main/libc indicates it's alive and well.
- andyferris 5y ago
  16 August 2019
deleted 5y ago
[deleted]

Falell 5y ago

Sounds complicated, but the goal of a fuzzable/sanitizable/etc libc sounds nice. Lack of ABI stability sounds terrifying as an application developer. My other immediate thought was "how will this interact with systems where the OS-provided libc is the only stable way to e.g. make syscalls", and "Layering Over Another libc" addresses this. I guess the idea is you'd link an application against llvm-libc and the system libc, and ship llvm-libc with your application?

jart 5y ago
Cosmopolitan Libc is sanitizable. I believe it's currently the only one where you can use Address Sanitizer at all layers of the software stack.
Arnavion 5y ago
If your software is in the distro repos, the maintainer of said package queues it to be rebuilt when the distro's llvm-libc package is updated. If you're providing the packages yourself, it's up to you to do that yourself. Or yes, you can vendor libc in your package. Not something everyone will like, but it depends on who your users are. It's not like this is unusual. Binaries compiled against today's glibc can fail to run on a machine that hasn't been updated since last week because they rely on a new / different symbol. Rebuilding the distro's packages when their deps are updated is standard fare.
- mananaysiempre 5y ago
  > Binaries compiled against today's glibc can fail to run on a machine that hasn't been updated since last week because they rely on a new / different symbol. Note, however, that it is a Glibc bug (modulo Drepper’s temper) if the reverse happens: Glibc symbol versioning ensures that binaries depending on an old Glibc (only) will run on a new one. So the proper way to build a maximally-compatible Linux executable would be to build a cross toolchain targeting an old Glibc and compile your code with it. Unfortunately, the build system is hell and old Glibcs doesn’t compile without backported patches, so while I did try to follow in the footsteps of a couple of people[1–5], I did not succeed. Mass rebuilds still happen with other ecosystems, though. GHC-compiled Haskell libraries are fine-grained and not ABI-stable across compiler versions, so my Arch box regularly gets hit with a deluge of teensy Haskell library updates, and Arch is currently undergoing a massive Python rebuild (blocking all other Python package updates) behind the scenes as well. [1]: https://github.com/wheybags/glibc_version_header https://github.com/wheybags/glibc_version_header (hack but easy and will probably work most of the time) [2]: https://www.lordaro.co.uk/posts/2018-08-26-compiling-glibc.html https://www.lordaro.co.uk/posts/2018-08-26-compiling-glibc.h... (someone’s mostly-nonhackish effort) [3]: https://github.com/pypa/manylinux https://github.com/pypa/manylinux (what Python manylinux wheels use, more modern than absolutely necessary) [4]: https://github.com/FooBarWidget/holy-build-box https://github.com/FooBarWidget/holy-build-box (ditto and is also a complete opaque cross-toolchain build recipe, but apparently people use that) [5]: https://casualhacking.io/blog/2018/12/25/create-highly-portable-elf-binaries-using-the-build-anywhere-toolchain https://casualhacking.io/blog/2018/12/25/create-highly-porta... (missed it last time, so can’t say much)

bch 5y ago

In the absence any any mention of licensing, I presume this is Apache-2… I don’t know if I’ve ever seen such a core piece of infra licensed that way (am used to BSD or MIT) - others might know better than I: is this a sane way forward?

jcranmer 5y ago
The license is here: <https://github.com/llvm/llvm-project/blob/main/libc/LICENSE.TXT https://github.com/llvm/llvm-project/blob/main/libc/LICENSE....>. It's Apache v2 with a compiler exception--i.e., the compiler linking bits of itself into you doesn't count for license purposes.
Someone 5y ago
It’s Apache licensed with LLVM exceptions (https://github.com/llvm/llvm-project/blob/main/libc/LICENSE.TXT https://github.com/llvm/llvm-project/blob/main/libc/LICENSE....) LibC++ is licensed that way, too (https://github.com/llvm/llvm-project/blob/main/libcxx/LICENSE.TXT https://github.com/llvm/llvm-project/blob/main/libcxx/LICENS...)
mananaysiempre 5y ago
The OpenBSD people in general and De Raadt specifically are probably the best-known objectors to Apache-2[1–4], among other things because they hold that Apache-2 is too broad to be a copyright license and thus has to be interpreted as a contract instead. I don’t know if they’re right, but their arguments did shift my opinion in their direction. Corporate lawyers seem to love it, though, because of the mutually-assured-patent-destruction clause. [1]: http://www.openbsd.org/policy.html http://www.openbsd.org/policy.html (see corresponding section) [2]: https://marc.info/?i=91077.1475036864%20()%20cvs%20!%20openbsd%20!%20org https://marc.info/?i=91077.1475036864%20()%20cvs%20!%20openb... (De Raadt rants on openbsd-misc, discussed on HN at https://news.ycombinator.com/item?id=126178810 https://news.ycombinator.com/item?id=126178810) [3]: https://lists.llvm.org/pipermail/llvm-dev/2017-April/112300.html https://lists.llvm.org/pipermail/llvm-dev/2017-April/112300.... (Kettenis objects on behalf of OpenBSD on llvm-dev) [4]: https://www.cambus.net/the-state-of-toolchains-in-openbsd/ https://www.cambus.net/the-state-of-toolchains-in-openbsd/ (OpenBSD gives up on staying with old LLVM)

flohofwoe 5y ago

> but take advantage and use C++ language facilities for the core implementation. This point sticks out. Would be nice to get some more details why it would make sense to use C++ for the implementation instead of C.

maccard 5y ago
Raii is enough of a reason IMO, but templates, type safety, actual arrays, constexpr, and various modern changes to the language that allow for the compiler to optimise code (move semantics and RVO)
- dralley 5y ago
  Those are all fantastic reasons to use relibc, the libc written in Rust, which has the additional benefit of already existing. https://gitlab.redox-os.org/redox-os/relibc https://gitlab.redox-os.org/redox-os/relibc
  - maccard 5y ago
    Sorry I missed this reply. I do'nt disagree, but honestly at this point anything is better than using C for libraries like this.
- mhh__ 5y ago
  Function overloading and more places to use default values was enough to make me go mental last time I had to use strict C as opposed to limited C++
mhh__ 5y ago
Other than platform support I think C is practically only a hindrance compared to C++ in the hands of competent developers.
zemoo 5y ago
The Microsoft C standard library implementation also takes this approach - C++ implementation with C exports, for the reasons called out by the sibling comments.

spullara 5y ago

Since this is now over 2 years old is there any progress?

jcranmer 5y ago
Yes, there is progress. But it is very far from anything you might consider complete or arguably even usable--there's no implementation of malloc or basic stdio, for example.
- cclearwater 5y ago
  Wouldn't they just use Google's tcmalloc?
  - fyrn- 5y ago
    Probably not the ideal pick for an everuone facing library like this, tcmalloc focuses on reducing core thread contention, at the cost of compactness and often debugability. I've not been following the ptoject for some time though, so maybe things have changed. Still a great allocator if those tradoffs are what you want. Something more like mimalloc would probably be a better general purpose allocator for something like this
  - dxf 5y ago
    The plan is to use Scudo, which is the allocator that lives in compiler-rt. Edited to include the link to Scudo: https://github.com/llvm/llvm-project/tree/main/compiler-rt/lib/scudo https://github.com/llvm/llvm-project/tree/main/compiler-rt/l...
    - dxf 5y ago
      Edited again: I am reminded that you can already get scudo today by using a CMake option to include scudo when building LLVM-libc.
  - scrb 5y ago
    Users wanting allocator functions (malloc and friends) from LLVM libc will get the SCUDO allocator. They need to use a special CMake option to include SCUDO when building LLVM libc.

po1nt 5y ago

I've expected some plot twist like "It will be coded in Rust"

guerrilla 5y ago
Close... but take advantage and use C++ language facilities for the core implementation.
kryptiskt 5y ago
If you want that Redox has a libc coded in Rust: https://gitlab.redox-os.org/redox-os/relibc https://gitlab.redox-os.org/redox-os/relibc There is a bunch of Redox ports of C programs, so it is apparently good enough for a range of software. https://gitlab.redox-os.org/redox-os/cookbook/-/tree/master/recipes https://gitlab.redox-os.org/redox-os/cookbook/-/tree/master/...

ndesaulniers 5y ago

I hope one day that I can statically link against my libc easily one day. Not just statically link but even LTO my libc into my binary. I understand the "security" concerns, but I feel there are cases where they don't matter. Case in point; my C compiler. It already was not designed to handle untrusted input well enough and I plan to run multiple instances in parallel, so my kernel should handle COW.

yjftsjthsd-h 5y ago
You can do this today with musl, FWIW.
- ndesaulniers 5y ago
  You can, and I have done it. It's easy for hello world. But for larger projects it's more difficult. If anyone has even done a build of clang with this, I'd love to know!
  - boogies 5y ago
    > If anyone has even done a build of clang with this, I'd love to know! Clang's in Alpine Linux's repo's, so I guess they have and this is how: https://git.alpinelinux.org/aports/tree/main/clang https://git.alpinelinux.org/aports/tree/main/clang
    - Aissen 5y ago
      They built clang against musl, just not statically.
  - Ericson2314 5y ago
    I made https://github.com/NixOS/nixpkgs/pull/149523 https://github.com/NixOS/nixpkgs/pull/149523 to kick off nix-build -A pkgsStatic.llvmPackages.clang-unwrapped Which built all the other deps just fine, but failed buildling LLVM with CONFIGURE_LLVM_NATIVE. That seems easy enough to fix, as that's the regular part, though. No guarantee there wouldn't be other more serious issues lurking beneath, but we've attempted quite a lot of static builds of complicated thing, augmented Musl with various bits and scraps to make it more featureful. You gave a very nice intro for my talk, saying Nixpkgs could well be the future so check it out. Well, I hope that future can arive more widely soon :).

pabs3 5y ago

I wonder if it will support versioned symbols for ABI stability like glibc does.

exikyut 5y ago

Hmm, my one question is: what platforms will this explicitly support? Linux, macOS, Windows, FreeBSD, and probably OpenBSD seem shoo-in table stakes. I'm more curious about - iOS/iPadOS: already have a libc, but... maybe? - Android: already has bionic/NDK; alternative useful? - NetBSD: rump kernel/unikernel applications? - VMS: has x86_64 support now; suddenly less irrelevant than before? - QNX: IIUC still the best deterministic/hard real time POSIX OS... - Illumos: not dead yet? - HP-UX/AIX: still used in certain industrial applications...? - Serenity: ...oh wait, just realized this isn't full POSIX, woops (would that be a prerequisite?) - (what obvious thing did I forget? :P) I ask this question mostly to update my understanding of "the state of kernel/OS interestingness, ~2022", since the process of deciding what targets a new major libc should consider relevant is going to be both well-informed and carefully considered given the anticipated (hoped) timescale of such a project.

jcranmer 5y ago
As of right now, it supports... Linux x86-64 and maybe aarch64. Although as of right now, the only OS-specific things it supports are a very stub loader for main, a thread library, and signal handling.
indrora 5y ago
Embedded targets: ARM6-ARM8, even older (STM32) in some cases. Consider the places where libiberty and the now very long in the tooth RedHat Newlib is in use. There's a lot of places where llvm is used but where a good libc is basically gone or is mostly implemented as a bunch of messy assembly routines.
RantyDave 5y ago
> Illumos: not dead yet? Not quite: https://omnios.org https://omnios.org

account42 5y ago

> Ability to layer this libc over the system libc if possible and desired for a platform. This sounds nice for being able to use newer C/C++ library functions when targeting an older system libc by statically linking the llvm-libc implementation of the missing functions.