9 ms·

I just don't understand why, what are the benefits to simply offering syscall interface and let developers decide if they want to link to libc

catiopatio 4y ago
The main benefit is that you can implement complex logic, performance optimizations, and compatibility shims on the userspace side of the kernel/userspace divide, where it's generally much easier to do, and bugs won't result in full compromise of the kernel. There's not really a strong argument for not linking libc. Even if you want to implement your own libc, you can still do so with linker tricks and calling through to the supported syscall wrappers in the real libc.
- JonChesterfield 4y ago
  Your options are poor if you don't want to build on libc though.
  - catiopatio 4y ago
    Outside unsupported edge cases (e.g. shell code), there's really no material difference between jumping to the address of syscall wrapper in libSystem.dylib, versus trapping to the kernel via a given syscall number — other than the latter being a supported interface, while the former is explicitly not.
    - JonChesterfield 4y ago
      Edge cases also include using a different libc, working around bugs in the supported libc, trying to minimise how much of your stack is implemented in C.
      - catiopatio 4y ago
        Alternative libc implementations just aren't a thing outside of Linux. However, if, for some reason, you really want to re-implement libc, you still can, but your library will have to link against the system-provided libc for the syscall entry points, because that's the only stable interface to the kernel. There's no reason that should be a problem for a hypothetical libc-reimplementor.
- matheusmoreira 4y ago
  > There's not really a strong argument for not linking libc. Sure there is. The libc sucks. Freestanding C actually turned out to be a superior language because there's not as much legacy weighing it down. There's many systems languages out there, nobody should be forced to link to C stuff.
  - wongarsu 4y ago
    I think Windows solved this much better than libc by separating the concerns: there's user32.dll providing a thin abstraction over syscalls, a couple libraries providing higher-level interfaces like wintrust.dll, and thirdly the C runtime library providing all the stuff mandated by C. You can easily ditch the latter, while still keeping the benefits of user32.dll (which cosmopolitan is doing).
    - lgg 4y ago
      macOS essentially has the same distinction internally (libsystem_kernel.dylib), but that is not directly linkable and is instead re-exported via the libSystem umbrella which also exports libsystem_c.dylib. While you cannot link directly to libsystem_kernel.dylib, if you choose to ignore everything in libsystem_c.dylib and only use the syscall wrappers reexported from libsystem_kernel.dylib via libSystem it has practically the same effect on macOS (in fact, the resulting binary will be identical to what a hypothetical binary linked to just a single libsystem_kernel.dylib would be except for a single `LC_LOAD_DYLIB` command). Such a binary would have the system lib c initialized sitting in its address space, but for the rare binary that really wants its own libc that doesn't seem unreasonable.
  - twic 4y ago
    What's at issue here is linking to the platform libc for syscalls. If you don't want to use the platform libc's qsort implementation, by all means, don't. But if you want to call gettimeofday, you should do that by calling the gettimeofday function in the libc, not by trying to set up a system call on your own. The former is a stable interface, the latter isn't. It's annoying that libc combines two completely different things - userspace utility functions, system calls - but unfortunately, that's where history has brought us.
kjeetgill 4y ago
My take: Like with any interface the definer needs to balance api user ergonomics, your developer's ergonomics, and internal developer's needs. If your api can get a little toe-hold on the remote side there's all sorts of advantages (and compromises) you can leverage. In this case that means the api designer gets a little bit of wiggle room in the process userspace where it might be more appropriate to say shim some calls so that they're no longer 1:1 with a syscall. The obvious example is that malloc() can call syscalls for you and is the "default" way to allocate memory you might want to provide but It's more of it's own little runtime. My favorite example was in my time at LinkedIn. Because the internal Kafka team had thick client libraries, making the company wide change to enable encryption was as simple as pushing out new libraries and deprecating old ones.
arghwhat 4y ago
It significantly simplifies OS development. E.g., syscalls can be modified and improved or entirely deprecated without concern. If OpenBSD wanted to, they could implement something like io_uring with support for all kernel functionality, port libc to use that and ditch conventional syscalls entirely (simulating blocking where needed), without user-space knowing anything changed. Under Linux, which is actually rather unusual in providing ABI guarantees, you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written, and cannot change user-space no matter how good that change would be for either or both.
- ithinkso 4y ago
  > you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). Also, I understand catiopatio's argument in the sibling comment about doing more in user-space than in kernel in case of a bug, but it breaks down the moment thin wrappers around syscalls exist there. You link to libc/libSystem and use every thin wrapper in existence - now no syscall can be changed (if I understand correctly how macOS works, never used one)
  - simias 4y ago
    I'd much rather export this complexity to userland code than privileged kernel mode code though. It reduces the potential consequence for a mistake pretty drastically.
  - arghwhat 4y ago
    > But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. It is not instead. With syscall ABI you need to maintain both, without it you only need to maintain libc (a majority of which is dictated by POSIX anyway). > In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). As above, maintaining two contracts is harder than one. Proprietary parts aside, most OS's have their kernel and user-space developed together, and that is exactly what allows them to work this way. This is used to adopt new features, change or deprecate old features with a brutal efficiency that Linux cannot compete with - e.g., when OpenBSD implemented pledge in both kernel and all relevant tools. This is one of the reasons that these projects can keep up or in some cases surpass Linux (FreeBSD networking is seen as superior, and you used to get better performance from running Linux binaries on FreeBSD through its compatibility layer) despite having much smaller groups of maintainers and users.

catiopatio 4y ago

The main benefit is that you can implement complex logic, performance optimizations, and compatibility shims on the userspace side of the kernel/userspace divide, where it's generally much easier to do, and bugs won't result in full compromise of the kernel. There's not really a strong argument for not linking libc. Even if you want to implement your own libc, you can still do so with linker tricks and calling through to the supported syscall wrappers in the real libc.

JonChesterfield 4y ago
Your options are poor if you don't want to build on libc though.
- catiopatio 4y ago
  Outside unsupported edge cases (e.g. shell code), there's really no material difference between jumping to the address of syscall wrapper in libSystem.dylib, versus trapping to the kernel via a given syscall number — other than the latter being a supported interface, while the former is explicitly not.
  - JonChesterfield 4y ago
    Edge cases also include using a different libc, working around bugs in the supported libc, trying to minimise how much of your stack is implemented in C.
    - catiopatio 4y ago
      Alternative libc implementations just aren't a thing outside of Linux. However, if, for some reason, you really want to re-implement libc, you still can, but your library will have to link against the system-provided libc for the syscall entry points, because that's the only stable interface to the kernel. There's no reason that should be a problem for a hypothetical libc-reimplementor.
matheusmoreira 4y ago
> There's not really a strong argument for not linking libc. Sure there is. The libc sucks. Freestanding C actually turned out to be a superior language because there's not as much legacy weighing it down. There's many systems languages out there, nobody should be forced to link to C stuff.
- wongarsu 4y ago
  I think Windows solved this much better than libc by separating the concerns: there's user32.dll providing a thin abstraction over syscalls, a couple libraries providing higher-level interfaces like wintrust.dll, and thirdly the C runtime library providing all the stuff mandated by C. You can easily ditch the latter, while still keeping the benefits of user32.dll (which cosmopolitan is doing).
  - lgg 4y ago
    macOS essentially has the same distinction internally (libsystem_kernel.dylib), but that is not directly linkable and is instead re-exported via the libSystem umbrella which also exports libsystem_c.dylib. While you cannot link directly to libsystem_kernel.dylib, if you choose to ignore everything in libsystem_c.dylib and only use the syscall wrappers reexported from libsystem_kernel.dylib via libSystem it has practically the same effect on macOS (in fact, the resulting binary will be identical to what a hypothetical binary linked to just a single libsystem_kernel.dylib would be except for a single `LC_LOAD_DYLIB` command). Such a binary would have the system lib c initialized sitting in its address space, but for the rare binary that really wants its own libc that doesn't seem unreasonable.
- twic 4y ago
  What's at issue here is linking to the platform libc for syscalls. If you don't want to use the platform libc's qsort implementation, by all means, don't. But if you want to call gettimeofday, you should do that by calling the gettimeofday function in the libc, not by trying to set up a system call on your own. The former is a stable interface, the latter isn't. It's annoying that libc combines two completely different things - userspace utility functions, system calls - but unfortunately, that's where history has brought us.

kjeetgill 4y ago

My take: Like with any interface the definer needs to balance api user ergonomics, your developer's ergonomics, and internal developer's needs. If your api can get a little toe-hold on the remote side there's all sorts of advantages (and compromises) you can leverage. In this case that means the api designer gets a little bit of wiggle room in the process userspace where it might be more appropriate to say shim some calls so that they're no longer 1:1 with a syscall. The obvious example is that malloc() can call syscalls for you and is the "default" way to allocate memory you might want to provide but It's more of it's own little runtime. My favorite example was in my time at LinkedIn. Because the internal Kafka team had thick client libraries, making the company wide change to enable encryption was as simple as pushing out new libraries and deprecating old ones.

arghwhat 4y ago

It significantly simplifies OS development. E.g., syscalls can be modified and improved or entirely deprecated without concern. If OpenBSD wanted to, they could implement something like io_uring with support for all kernel functionality, port libc to use that and ditch conventional syscalls entirely (simulating blocking where needed), without user-space knowing anything changed. Under Linux, which is actually rather unusual in providing ABI guarantees, you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written, and cannot change user-space no matter how good that change would be for either or both.

ithinkso 4y ago
> you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). Also, I understand catiopatio's argument in the sibling comment about doing more in user-space than in kernel in case of a bug, but it breaks down the moment thin wrappers around syscalls exist there. You link to libc/libSystem and use every thin wrapper in existence - now no syscall can be changed (if I understand correctly how macOS works, never used one)
- simias 4y ago
  I'd much rather export this complexity to userland code than privileged kernel mode code though. It reduces the potential consequence for a mistake pretty drastically.
- arghwhat 4y ago
  > But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. It is not instead. With syscall ABI you need to maintain both, without it you only need to maintain libc (a majority of which is dictated by POSIX anyway). > In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). As above, maintaining two contracts is harder than one. Proprietary parts aside, most OS's have their kernel and user-space developed together, and that is exactly what allows them to work this way. This is used to adopt new features, change or deprecate old features with a brutal efficiency that Linux cannot compete with - e.g., when OpenBSD implemented pledge in both kernel and all relevant tools. This is one of the reasons that these projects can keep up or in some cases surpass Linux (FreeBSD networking is seen as superior, and you used to get better performance from running Linux binaries on FreeBSD through its compatibility layer) despite having much smaller groups of maintainers and users.