7 ms·

Obviously, do whatever you want if you’re just playing around, but please do be aware that the syscall interface on macOS is private, subject to change at any t

Obviously, do whatever you want if you’re just playing around, but please do be aware that the syscall interface on macOS is private, subject to change at any time, and should never be used directly.

deleted 4y ago
[deleted]
arghwhat 4y ago
Same goes for OpenBSD. It is quite normal that the only defined API is the provided libc or similar. This is of course a little annoying at times.
- ithinkso 4y ago
  I just don't understand why, what are the benefits to simply offering syscall interface and let developers decide if they want to link to libc
  - catiopatio 4y ago
    The main benefit is that you can implement complex logic, performance optimizations, and compatibility shims on the userspace side of the kernel/userspace divide, where it's generally much easier to do, and bugs won't result in full compromise of the kernel. There's not really a strong argument for not linking libc. Even if you want to implement your own libc, you can still do so with linker tricks and calling through to the supported syscall wrappers in the real libc.
    - JonChesterfield 4y ago
      Your options are poor if you don't want to build on libc though.
      - catiopatio 4y ago
        Outside unsupported edge cases (e.g. shell code), there's really no material difference between jumping to the address of syscall wrapper in libSystem.dylib, versus trapping to the kernel via a given syscall number — other than the latter being a supported interface, while the former is explicitly not.
        JonChesterfield 4y ago
        Edge cases also include using a different libc, working around bugs in the supported libc, trying to minimise how much of your stack is implemented in C.
        catiopatio 4y ago
        Alternative libc implementations just aren't a thing outside of Linux. However, if, for some reason, you really want to re-implement libc, you still can, but your library will have to link against the system-provided libc for the syscall entry points, because that's the only stable interface to the kernel. There's no reason that should be a problem for a hypothetical libc-reimplementor.
    - matheusmoreira 4y ago
      > There's not really a strong argument for not linking libc. Sure there is. The libc sucks. Freestanding C actually turned out to be a superior language because there's not as much legacy weighing it down. There's many systems languages out there, nobody should be forced to link to C stuff.
      - wongarsu 4y ago
        I think Windows solved this much better than libc by separating the concerns: there's user32.dll providing a thin abstraction over syscalls, a couple libraries providing higher-level interfaces like wintrust.dll, and thirdly the C runtime library providing all the stuff mandated by C. You can easily ditch the latter, while still keeping the benefits of user32.dll (which cosmopolitan is doing).
        lgg 4y ago
        macOS essentially has the same distinction internally (libsystem_kernel.dylib), but that is not directly linkable and is instead re-exported via the libSystem umbrella which also exports libsystem_c.dylib. While you cannot link directly to libsystem_kernel.dylib, if you choose to ignore everything in libsystem_c.dylib and only use the syscall wrappers reexported from libsystem_kernel.dylib via libSystem it has practically the same effect on macOS (in fact, the resulting binary will be identical to what a hypothetical binary linked to just a single libsystem_kernel.dylib would be except for a single `LC_LOAD_DYLIB` command). Such a binary would have the system lib c initialized sitting in its address space, but for the rare binary that really wants its own libc that doesn't seem unreasonable.
      - twic 4y ago
        What's at issue here is linking to the platform libc for syscalls. If you don't want to use the platform libc's qsort implementation, by all means, don't. But if you want to call gettimeofday, you should do that by calling the gettimeofday function in the libc, not by trying to set up a system call on your own. The former is a stable interface, the latter isn't. It's annoying that libc combines two completely different things - userspace utility functions, system calls - but unfortunately, that's where history has brought us.
  - kjeetgill 4y ago
    My take: Like with any interface the definer needs to balance api user ergonomics, your developer's ergonomics, and internal developer's needs. If your api can get a little toe-hold on the remote side there's all sorts of advantages (and compromises) you can leverage. In this case that means the api designer gets a little bit of wiggle room in the process userspace where it might be more appropriate to say shim some calls so that they're no longer 1:1 with a syscall. The obvious example is that malloc() can call syscalls for you and is the "default" way to allocate memory you might want to provide but It's more of it's own little runtime. My favorite example was in my time at LinkedIn. Because the internal Kafka team had thick client libraries, making the company wide change to enable encryption was as simple as pushing out new libraries and deprecating old ones.
  - arghwhat 4y ago
    It significantly simplifies OS development. E.g., syscalls can be modified and improved or entirely deprecated without concern. If OpenBSD wanted to, they could implement something like io_uring with support for all kernel functionality, port libc to use that and ditch conventional syscalls entirely (simulating blocking where needed), without user-space knowing anything changed. Under Linux, which is actually rather unusual in providing ABI guarantees, you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written, and cannot change user-space no matter how good that change would be for either or both.
    - ithinkso 4y ago
      > you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). Also, I understand catiopatio's argument in the sibling comment about doing more in user-space than in kernel in case of a bug, but it breaks down the moment thin wrappers around syscalls exist there. You link to libc/libSystem and use every thin wrapper in existence - now no syscall can be changed (if I understand correctly how macOS works, never used one)
      - simias 4y ago
        I'd much rather export this complexity to userland code than privileged kernel mode code though. It reduces the potential consequence for a mistake pretty drastically.
      - arghwhat 4y ago
        > But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. It is not instead. With syscall ABI you need to maintain both, without it you only need to maintain libc (a majority of which is dictated by POSIX anyway). > In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). As above, maintaining two contracts is harder than one. Proprietary parts aside, most OS's have their kernel and user-space developed together, and that is exactly what allows them to work this way. This is used to adopt new features, change or deprecate old features with a brutal efficiency that Linux cannot compete with - e.g., when OpenBSD implemented pledge in both kernel and all relevant tools. This is one of the reasons that these projects can keep up or in some cases surpass Linux (FreeBSD networking is seen as superior, and you used to get better performance from running Linux binaries on FreeBSD through its compatibility layer) despite having much smaller groups of maintainers and users.
- weinzierl 4y ago
  What do alternative libc implementations like musl or dietlib do on those systems? Do their maintainers manually track the unstable syscall interface or are developers forced to use the system provided libc because the alternatives just don't work completely?
  - brynet 4y ago
    musl/dietlibc target the Linux kernel. There are no alternative libc implementations for *BSD, unlike Linux the system provided libc is the only libc that matters.
  - deleted 4y ago
    [deleted]
  - arghwhat 4y ago
    On such systems, user-space is developed together with the kernel. The libc is as much part of OpenBSD as the kernel is.
    - jart 4y ago
      Just because no one's done it before doesn't mean we're not allowed to do it. What Cosmopolitan Libc does is no different from any statically-linked binary on BSD, because they too use the SYSCALL instruction directly in a way that can't be changed at will by the operating system maintainers. The BSD operating systems do not forbid static linking. If you look at their syscalls.master files, a lot of time, particularly with ones like FreeBSD, you'll notice a large number of system calls which are only there to preserve backwards compatibility with statically linked binaries. It's really nice of them to do that. When people talk about the supposed requirement to depend on the platform libc dso, what they're actually talking about is Apple frowning upon statically linked binaries. https://developer.apple.com/library/archive/qa/qa1118/_index.html https://developer.apple.com/library/archive/qa/qa1118/_index... They don't forbid it though, like Microsoft and Fuschia do. Fuschia for instance uses RIP origin detection. Windows does it by changing the RAX ordinals fortnightly. Apple simply asks that we say, hey, if you use Cosmo there's some risk Apple might break our binaries. We take proactive steps to avoid that happening with Cosmo. For example, we don't do some of the things Go did, like reverse engineering the memory layout of Apple's time functions. Cosmo sticks to the APIs that are shared by UNIXes in general, e.g. gettimeofday(), rather than depending on Apple's own internal designs, e.g. Mach system calls. I don't believe Apple can rightfully claim APIs that aren't their own, as being their own implementation detail which they can change at will. We do our best to respect Apple's boundaries, so I believe the risk of breakage with Cosmo on Apple should be minimal.
vonwoodson 4y ago
That sounds like using a Mac in general: “Do what you want if your just playing around, but be aware that [if you’re not a blessed developer] macOS is private, subject to change at any time, and should never be used [professionally]”
- catiopatio 4y ago
  This isn't an Apple thing, it's an "almost every OS but Linux" thing.
  - ithkuil 4y ago
    are you sure? for all things bad about windows, one cannot say that they didn't take backward compatibility quite seriously
    - tinalumfoil 4y ago
      They do change over time, and programs will break between updates on different windows versions of you make syscalls directly. https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/ (click show all)
    - matheusmoreira 4y ago
      Yes. Windows has changed system call numbers before. User space is supposed to use the Microsoft libraries.
Jolter 4y ago
Well, isn’t that why this web page exists? To advertise the nice libc implementation so users don’t have to make syscalls directly to the OS? Maybe I’ve misunderstood something…
euclaise 4y ago
Same goes for Windows, I believe?

deleted 4y ago

[deleted]

arghwhat 4y ago

Same goes for OpenBSD. It is quite normal that the only defined API is the provided libc or similar. This is of course a little annoying at times.

ithinkso 4y ago
I just don't understand why, what are the benefits to simply offering syscall interface and let developers decide if they want to link to libc
- catiopatio 4y ago
  The main benefit is that you can implement complex logic, performance optimizations, and compatibility shims on the userspace side of the kernel/userspace divide, where it's generally much easier to do, and bugs won't result in full compromise of the kernel. There's not really a strong argument for not linking libc. Even if you want to implement your own libc, you can still do so with linker tricks and calling through to the supported syscall wrappers in the real libc.
  - JonChesterfield 4y ago
    Your options are poor if you don't want to build on libc though.
    - catiopatio 4y ago
      Outside unsupported edge cases (e.g. shell code), there's really no material difference between jumping to the address of syscall wrapper in libSystem.dylib, versus trapping to the kernel via a given syscall number — other than the latter being a supported interface, while the former is explicitly not.
      - JonChesterfield 4y ago
        Edge cases also include using a different libc, working around bugs in the supported libc, trying to minimise how much of your stack is implemented in C.
        catiopatio 4y ago
        Alternative libc implementations just aren't a thing outside of Linux. However, if, for some reason, you really want to re-implement libc, you still can, but your library will have to link against the system-provided libc for the syscall entry points, because that's the only stable interface to the kernel. There's no reason that should be a problem for a hypothetical libc-reimplementor.
  - matheusmoreira 4y ago
    > There's not really a strong argument for not linking libc. Sure there is. The libc sucks. Freestanding C actually turned out to be a superior language because there's not as much legacy weighing it down. There's many systems languages out there, nobody should be forced to link to C stuff.
    - wongarsu 4y ago
      I think Windows solved this much better than libc by separating the concerns: there's user32.dll providing a thin abstraction over syscalls, a couple libraries providing higher-level interfaces like wintrust.dll, and thirdly the C runtime library providing all the stuff mandated by C. You can easily ditch the latter, while still keeping the benefits of user32.dll (which cosmopolitan is doing).
      - lgg 4y ago
        macOS essentially has the same distinction internally (libsystem_kernel.dylib), but that is not directly linkable and is instead re-exported via the libSystem umbrella which also exports libsystem_c.dylib. While you cannot link directly to libsystem_kernel.dylib, if you choose to ignore everything in libsystem_c.dylib and only use the syscall wrappers reexported from libsystem_kernel.dylib via libSystem it has practically the same effect on macOS (in fact, the resulting binary will be identical to what a hypothetical binary linked to just a single libsystem_kernel.dylib would be except for a single `LC_LOAD_DYLIB` command). Such a binary would have the system lib c initialized sitting in its address space, but for the rare binary that really wants its own libc that doesn't seem unreasonable.
    - twic 4y ago
      What's at issue here is linking to the platform libc for syscalls. If you don't want to use the platform libc's qsort implementation, by all means, don't. But if you want to call gettimeofday, you should do that by calling the gettimeofday function in the libc, not by trying to set up a system call on your own. The former is a stable interface, the latter isn't. It's annoying that libc combines two completely different things - userspace utility functions, system calls - but unfortunately, that's where history has brought us.
- kjeetgill 4y ago
  My take: Like with any interface the definer needs to balance api user ergonomics, your developer's ergonomics, and internal developer's needs. If your api can get a little toe-hold on the remote side there's all sorts of advantages (and compromises) you can leverage. In this case that means the api designer gets a little bit of wiggle room in the process userspace where it might be more appropriate to say shim some calls so that they're no longer 1:1 with a syscall. The obvious example is that malloc() can call syscalls for you and is the "default" way to allocate memory you might want to provide but It's more of it's own little runtime. My favorite example was in my time at LinkedIn. Because the internal Kafka team had thick client libraries, making the company wide change to enable encryption was as simple as pushing out new libraries and deprecating old ones.
- arghwhat 4y ago
  It significantly simplifies OS development. E.g., syscalls can be modified and improved or entirely deprecated without concern. If OpenBSD wanted to, they could implement something like io_uring with support for all kernel functionality, port libc to use that and ditch conventional syscalls entirely (simulating blocking where needed), without user-space knowing anything changed. Under Linux, which is actually rather unusual in providing ABI guarantees, you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written, and cannot change user-space no matter how good that change would be for either or both.
  - ithinkso 4y ago
    > you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). Also, I understand catiopatio's argument in the sibling comment about doing more in user-space than in kernel in case of a bug, but it breaks down the moment thin wrappers around syscalls exist there. You link to libc/libSystem and use every thin wrapper in existence - now no syscall can be changed (if I understand correctly how macOS works, never used one)
    - simias 4y ago
      I'd much rather export this complexity to userland code than privileged kernel mode code though. It reduces the potential consequence for a mistake pretty drastically.
    - arghwhat 4y ago
      > But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. It is not instead. With syscall ABI you need to maintain both, without it you only need to maintain libc (a majority of which is dictated by POSIX anyway). > In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). As above, maintaining two contracts is harder than one. Proprietary parts aside, most OS's have their kernel and user-space developed together, and that is exactly what allows them to work this way. This is used to adopt new features, change or deprecate old features with a brutal efficiency that Linux cannot compete with - e.g., when OpenBSD implemented pledge in both kernel and all relevant tools. This is one of the reasons that these projects can keep up or in some cases surpass Linux (FreeBSD networking is seen as superior, and you used to get better performance from running Linux binaries on FreeBSD through its compatibility layer) despite having much smaller groups of maintainers and users.
weinzierl 4y ago
What do alternative libc implementations like musl or dietlib do on those systems? Do their maintainers manually track the unstable syscall interface or are developers forced to use the system provided libc because the alternatives just don't work completely?
- brynet 4y ago
  musl/dietlibc target the Linux kernel. There are no alternative libc implementations for *BSD, unlike Linux the system provided libc is the only libc that matters.
- deleted 4y ago
  [deleted]
- arghwhat 4y ago
  On such systems, user-space is developed together with the kernel. The libc is as much part of OpenBSD as the kernel is.
  - jart 4y ago
    Just because no one's done it before doesn't mean we're not allowed to do it. What Cosmopolitan Libc does is no different from any statically-linked binary on BSD, because they too use the SYSCALL instruction directly in a way that can't be changed at will by the operating system maintainers. The BSD operating systems do not forbid static linking. If you look at their syscalls.master files, a lot of time, particularly with ones like FreeBSD, you'll notice a large number of system calls which are only there to preserve backwards compatibility with statically linked binaries. It's really nice of them to do that. When people talk about the supposed requirement to depend on the platform libc dso, what they're actually talking about is Apple frowning upon statically linked binaries. https://developer.apple.com/library/archive/qa/qa1118/_index.html https://developer.apple.com/library/archive/qa/qa1118/_index... They don't forbid it though, like Microsoft and Fuschia do. Fuschia for instance uses RIP origin detection. Windows does it by changing the RAX ordinals fortnightly. Apple simply asks that we say, hey, if you use Cosmo there's some risk Apple might break our binaries. We take proactive steps to avoid that happening with Cosmo. For example, we don't do some of the things Go did, like reverse engineering the memory layout of Apple's time functions. Cosmo sticks to the APIs that are shared by UNIXes in general, e.g. gettimeofday(), rather than depending on Apple's own internal designs, e.g. Mach system calls. I don't believe Apple can rightfully claim APIs that aren't their own, as being their own implementation detail which they can change at will. We do our best to respect Apple's boundaries, so I believe the risk of breakage with Cosmo on Apple should be minimal.

vonwoodson 4y ago

That sounds like using a Mac in general: “Do what you want if your just playing around, but be aware that [if you’re not a blessed developer] macOS is private, subject to change at any time, and should never be used [professionally]”

catiopatio 4y ago
This isn't an Apple thing, it's an "almost every OS but Linux" thing.
- ithkuil 4y ago
  are you sure? for all things bad about windows, one cannot say that they didn't take backward compatibility quite seriously
  - tinalumfoil 4y ago
    They do change over time, and programs will break between updates on different windows versions of you make syscalls directly. https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/ (click show all)
  - matheusmoreira 4y ago
    Yes. Windows has changed system call numbers before. User space is supposed to use the Microsoft libraries.

Jolter 4y ago

Well, isn’t that why this web page exists? To advertise the nice libc implementation so users don’t have to make syscalls directly to the OS? Maybe I’ve misunderstood something…

euclaise 4y ago

Same goes for Windows, I believe?