4 ms·

Many developers seem to have a cultish belief that the systems libc is the Only True Libc, missing the point (or being incapable of understanding) that alternat

Many developers seem to have a cultish belief that the systems libc is the Only True Libc, missing the point (or being incapable of understanding) that alternatives are possible.

iainmerrick 4y ago
Is that really any more cultish than the idea that the kernel interface is the only true interface, and everything else should be replaceable? It’s just a different approach. What’s so crazy about an OS vendor wanting to exercise control over that userspace shim layer, rather than assuming that user programs will be poking directly at the kernel? You can argue that having multiple libc implementations is good, sure, but it’s not the only way of doing things and it’s not without tradeoffs.
- mananaysiempre 4y ago
  The argument in the first paragraph doesn’t really work, I think: the kernel-user boundary is a security one, while the libc-app one is not (OpenBSD is something of an outlier in this respect, and even its case is a bit murky). So it is a tad more natural to draw the ABI boundary at the former point than at the latter, though of course this is not at all the only design consideration. Looking at this through a KERNEL32 lens, I can actually somewhat see the alternative point of view: in NT < 4 it was mostly an in-process RPC proxy for the subsystem server in CSRSS, and one can argue that libc on Unix-likes is in the same position (cf the vDSO on Linux). You can build RPC to a trusted server either way: define a stable wire protocol; or require the client to load proxy code into its address space. COM on Windows and hardware-accelerated graphics on Linux both take the second approach. But then why the hell does the proxy also have opinions about memory allocation, assignment of TLS slots, or floating-point formatting? (KERNEL32 has them on the first two points as well, mind you.) History[1] aside, does this really look like good engineering? (KERNEL32 makes a bit of a point there, but given how much HeapAlloc sucks as an allocator compared to the interoperability benefits it brings, I’m not sure it should be taken too seriously.) [1] https://utcc.utoronto.ca/~cks/space/blog/unix/UnixAPIAndCRuntime https://utcc.utoronto.ca/~cks/space/blog/unix/UnixAPIAndCRun...
  - comex 4y ago
    It's a bit of a paradox, isn't it? In theory it seems quite elegant to avoid tightly coupling the memory allocator to the kernel. But in practice, the fact that every pointer has to be freed by the same DLL that allocated it is famous for causing problems for Windows developers. As is, more generally, the inability to pass C and C++ standard library types across DLL boundaries. I think my ideal system would look like Linux in some ways. Any given Linux system has a 'standard' shared library ecosystem, with a single shared libc, and usually a shared C++ standard library as well. Executables and libraries that participate in that ecosystem can assume they'll use the same standard libraries and so can pass those libraries' types across module boundaries. Alongside them are 'hermit' executables, like Go programs or C programs statically linking to musl, that avoid the standard libc. But those executables typically also avoid all other system shared libraries (or greatly constrain their usage), so the difficulty in using their APIs (due to mismatched standard library types) isn't a big deal. Now, Linux sort of forces this approach by having the dynamic linker itself be part of libc. But that has some downsides, such as forcing the vDSO to work differently from every other library. I think I'd prefer to have a universal dynamic linker, like on Windows, but to still have 'ecosystem shares a libc, while hermits stick to themselves' as a convention.
twic 4y ago
This is incorrect. On most platforms, the libc is part of the operating system, and the libc API is the supported system call interface; user programs aren't allowed to make system calls directly to the kernel, any more than they're allowed to jump directly into the middle of kernel routines. As described elsewhere in this thread, that is the policy of MacOS X and Windows. It is also a policy which OpenBSD is moving towards: https://lwn.net/Articles/806863/ https://lwn.net/Articles/806863/ According to the Go devs, this is also the policy on Solaris-family OSs (Go uses libc there): https://github.com/golang/go/issues/24357#issuecomment-373007875 https://github.com/golang/go/issues/24357#issuecomment-37300... AFAIK, it is only Linux which explicitly supports the kernel system call interface, and so where alternative libcs are possible.
- nibbleshifter 4y ago
  > AFAIK, it is only Linux which explicitly supports the kernel system call interface, and so where alternative libcs are possible. You can do this on Windows. The syscall numbers change sometimes. I use it to evade EDR hooks :)
- bogomipz 4y ago
  >"AFAIK, it is only Linux which explicitly supports the kernel system call interface.: Could you elaborate on this? Specifically I'm not understanding what exactly is "explicit" with regards system calls on Linux compared to other OSes.
  - nibbleshifter 4y ago
    I think by this they mean the policy of the Linux kernel "never breaking user land". Libc is seen as user land by the Linux kernel, it provides an API for programs to abstract them away from syscalls. Libc dies syscalls for programs. By "never breaking userland", Linux promises to never break libc by never changing syscall numbers/args/types - new syscalls are added in ways that don't break older syscalls. Other platforms don't make this explicit promise and reserve the right to shuffle syscalls around, change them, remove them, etc as desired - making maintaining a libc for them a pain in the ass, and programmers are told to only use the libc as syscalls don't come with guarantees. You can (and I often do) program on windows using only syscalls, but your program tends to need rebuilding across major versions as the syscall interface isn't stable. Fuck me typing that on a phone was painful.

Many developers seem to have a cultish belief that the systems libc is the Only True Libc, missing the point (or being incapable of understanding) that alternatives are possible.

iainmerrick 4y ago

Is that really any more cultish than the idea that the kernel interface is the only true interface, and everything else should be replaceable? It’s just a different approach. What’s so crazy about an OS vendor wanting to exercise control over that userspace shim layer, rather than assuming that user programs will be poking directly at the kernel? You can argue that having multiple libc implementations is good, sure, but it’s not the only way of doing things and it’s not without tradeoffs.

mananaysiempre 4y ago
The argument in the first paragraph doesn’t really work, I think: the kernel-user boundary is a security one, while the libc-app one is not (OpenBSD is something of an outlier in this respect, and even its case is a bit murky). So it is a tad more natural to draw the ABI boundary at the former point than at the latter, though of course this is not at all the only design consideration. Looking at this through a KERNEL32 lens, I can actually somewhat see the alternative point of view: in NT < 4 it was mostly an in-process RPC proxy for the subsystem server in CSRSS, and one can argue that libc on Unix-likes is in the same position (cf the vDSO on Linux). You can build RPC to a trusted server either way: define a stable wire protocol; or require the client to load proxy code into its address space. COM on Windows and hardware-accelerated graphics on Linux both take the second approach. But then why the hell does the proxy also have opinions about memory allocation, assignment of TLS slots, or floating-point formatting? (KERNEL32 has them on the first two points as well, mind you.) History[1] aside, does this really look like good engineering? (KERNEL32 makes a bit of a point there, but given how much HeapAlloc sucks as an allocator compared to the interoperability benefits it brings, I’m not sure it should be taken too seriously.) [1] https://utcc.utoronto.ca/~cks/space/blog/unix/UnixAPIAndCRuntime https://utcc.utoronto.ca/~cks/space/blog/unix/UnixAPIAndCRun...
- comex 4y ago
  It's a bit of a paradox, isn't it? In theory it seems quite elegant to avoid tightly coupling the memory allocator to the kernel. But in practice, the fact that every pointer has to be freed by the same DLL that allocated it is famous for causing problems for Windows developers. As is, more generally, the inability to pass C and C++ standard library types across DLL boundaries. I think my ideal system would look like Linux in some ways. Any given Linux system has a 'standard' shared library ecosystem, with a single shared libc, and usually a shared C++ standard library as well. Executables and libraries that participate in that ecosystem can assume they'll use the same standard libraries and so can pass those libraries' types across module boundaries. Alongside them are 'hermit' executables, like Go programs or C programs statically linking to musl, that avoid the standard libc. But those executables typically also avoid all other system shared libraries (or greatly constrain their usage), so the difficulty in using their APIs (due to mismatched standard library types) isn't a big deal. Now, Linux sort of forces this approach by having the dynamic linker itself be part of libc. But that has some downsides, such as forcing the vDSO to work differently from every other library. I think I'd prefer to have a universal dynamic linker, like on Windows, but to still have 'ecosystem shares a libc, while hermits stick to themselves' as a convention.

twic 4y ago

This is incorrect. On most platforms, the libc is part of the operating system, and the libc API is the supported system call interface; user programs aren't allowed to make system calls directly to the kernel, any more than they're allowed to jump directly into the middle of kernel routines. As described elsewhere in this thread, that is the policy of MacOS X and Windows. It is also a policy which OpenBSD is moving towards: https://lwn.net/Articles/806863/ https://lwn.net/Articles/806863/ According to the Go devs, this is also the policy on Solaris-family OSs (Go uses libc there): https://github.com/golang/go/issues/24357#issuecomment-373007875 https://github.com/golang/go/issues/24357#issuecomment-37300... AFAIK, it is only Linux which explicitly supports the kernel system call interface, and so where alternative libcs are possible.

nibbleshifter 4y ago
> AFAIK, it is only Linux which explicitly supports the kernel system call interface, and so where alternative libcs are possible. You can do this on Windows. The syscall numbers change sometimes. I use it to evade EDR hooks :)
bogomipz 4y ago
>"AFAIK, it is only Linux which explicitly supports the kernel system call interface.: Could you elaborate on this? Specifically I'm not understanding what exactly is "explicit" with regards system calls on Linux compared to other OSes.
- nibbleshifter 4y ago
  I think by this they mean the policy of the Linux kernel "never breaking user land". Libc is seen as user land by the Linux kernel, it provides an API for programs to abstract them away from syscalls. Libc dies syscalls for programs. By "never breaking userland", Linux promises to never break libc by never changing syscall numbers/args/types - new syscalls are added in ways that don't break older syscalls. Other platforms don't make this explicit promise and reserve the right to shuffle syscalls around, change them, remove them, etc as desired - making maintaining a libc for them a pain in the ass, and programmers are told to only use the libc as syscalls don't come with guarantees. You can (and I often do) program on windows using only syscalls, but your program tends to need rebuilding across major versions as the syscall interface isn't stable. Fuck me typing that on a phone was painful.