12 ms·

Can I use a system call?

578 points by signa11 4y ago

catiopatio 4y ago
Obviously, do whatever you want if you’re just playing around, but please do be aware that the syscall interface on macOS is private, subject to change at any time, and should never be used directly.
- deleted 4y ago
  [deleted]
- arghwhat 4y ago
  Same goes for OpenBSD. It is quite normal that the only defined API is the provided libc or similar. This is of course a little annoying at times.
  - ithinkso 4y ago
    I just don't understand why, what are the benefits to simply offering syscall interface and let developers decide if they want to link to libc
    - catiopatio 4y ago
      The main benefit is that you can implement complex logic, performance optimizations, and compatibility shims on the userspace side of the kernel/userspace divide, where it's generally much easier to do, and bugs won't result in full compromise of the kernel. There's not really a strong argument for not linking libc. Even if you want to implement your own libc, you can still do so with linker tricks and calling through to the supported syscall wrappers in the real libc.
      - JonChesterfield 4y ago
        Your options are poor if you don't want to build on libc though.
        catiopatio 4y ago
        Outside unsupported edge cases (e.g. shell code), there's really no material difference between jumping to the address of syscall wrapper in libSystem.dylib, versus trapping to the kernel via a given syscall number — other than the latter being a supported interface, while the former is explicitly not.
        JonChesterfield 4y ago
        Edge cases also include using a different libc, working around bugs in the supported libc, trying to minimise how much of your stack is implemented in C.
        catiopatio 4y ago
        Alternative libc implementations just aren't a thing outside of Linux. However, if, for some reason, you really want to re-implement libc, you still can, but your library will have to link against the system-provided libc for the syscall entry points, because that's the only stable interface to the kernel. There's no reason that should be a problem for a hypothetical libc-reimplementor.
      - matheusmoreira 4y ago
        > There's not really a strong argument for not linking libc. Sure there is. The libc sucks. Freestanding C actually turned out to be a superior language because there's not as much legacy weighing it down. There's many systems languages out there, nobody should be forced to link to C stuff.
        wongarsu 4y ago
        I think Windows solved this much better than libc by separating the concerns: there's user32.dll providing a thin abstraction over syscalls, a couple libraries providing higher-level interfaces like wintrust.dll, and thirdly the C runtime library providing all the stuff mandated by C. You can easily ditch the latter, while still keeping the benefits of user32.dll (which cosmopolitan is doing).
        lgg 4y ago
        macOS essentially has the same distinction internally (libsystem_kernel.dylib), but that is not directly linkable and is instead re-exported via the libSystem umbrella which also exports libsystem_c.dylib. While you cannot link directly to libsystem_kernel.dylib, if you choose to ignore everything in libsystem_c.dylib and only use the syscall wrappers reexported from libsystem_kernel.dylib via libSystem it has practically the same effect on macOS (in fact, the resulting binary will be identical to what a hypothetical binary linked to just a single libsystem_kernel.dylib would be except for a single `LC_LOAD_DYLIB` command). Such a binary would have the system lib c initialized sitting in its address space, but for the rare binary that really wants its own libc that doesn't seem unreasonable.
        twic 4y ago
        What's at issue here is linking to the platform libc for syscalls. If you don't want to use the platform libc's qsort implementation, by all means, don't. But if you want to call gettimeofday, you should do that by calling the gettimeofday function in the libc, not by trying to set up a system call on your own. The former is a stable interface, the latter isn't. It's annoying that libc combines two completely different things - userspace utility functions, system calls - but unfortunately, that's where history has brought us.
    - kjeetgill 4y ago
      My take: Like with any interface the definer needs to balance api user ergonomics, your developer's ergonomics, and internal developer's needs. If your api can get a little toe-hold on the remote side there's all sorts of advantages (and compromises) you can leverage. In this case that means the api designer gets a little bit of wiggle room in the process userspace where it might be more appropriate to say shim some calls so that they're no longer 1:1 with a syscall. The obvious example is that malloc() can call syscalls for you and is the "default" way to allocate memory you might want to provide but It's more of it's own little runtime. My favorite example was in my time at LinkedIn. Because the internal Kafka team had thick client libraries, making the company wide change to enable encryption was as simple as pushing out new libraries and deprecating old ones.
    - arghwhat 4y ago
      It significantly simplifies OS development. E.g., syscalls can be modified and improved or entirely deprecated without concern. If OpenBSD wanted to, they could implement something like io_uring with support for all kernel functionality, port libc to use that and ditch conventional syscalls entirely (simulating blocking where needed), without user-space knowing anything changed. Under Linux, which is actually rather unusual in providing ABI guarantees, you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written, and cannot change user-space no matter how good that change would be for either or both.
      - ithinkso 4y ago
        > you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). Also, I understand catiopatio's argument in the sibling comment about doing more in user-space than in kernel in case of a bug, but it breaks down the moment thin wrappers around syscalls exist there. You link to libc/libSystem and use every thin wrapper in existence - now no syscall can be changed (if I understand correctly how macOS works, never used one)
        simias 4y ago
        I'd much rather export this complexity to userland code than privileged kernel mode code though. It reduces the potential consequence for a mistake pretty drastically.
        arghwhat 4y ago
        > But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. It is not instead. With syscall ABI you need to maintain both, without it you only need to maintain libc (a majority of which is dictated by POSIX anyway). > In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). As above, maintaining two contracts is harder than one. Proprietary parts aside, most OS's have their kernel and user-space developed together, and that is exactly what allows them to work this way. This is used to adopt new features, change or deprecate old features with a brutal efficiency that Linux cannot compete with - e.g., when OpenBSD implemented pledge in both kernel and all relevant tools. This is one of the reasons that these projects can keep up or in some cases surpass Linux (FreeBSD networking is seen as superior, and you used to get better performance from running Linux binaries on FreeBSD through its compatibility layer) despite having much smaller groups of maintainers and users.
  - weinzierl 4y ago
    What do alternative libc implementations like musl or dietlib do on those systems? Do their maintainers manually track the unstable syscall interface or are developers forced to use the system provided libc because the alternatives just don't work completely?
    - brynet 4y ago
      musl/dietlibc target the Linux kernel. There are no alternative libc implementations for *BSD, unlike Linux the system provided libc is the only libc that matters.
    - deleted 4y ago
      [deleted]
    - arghwhat 4y ago
      On such systems, user-space is developed together with the kernel. The libc is as much part of OpenBSD as the kernel is.
      - jart 4y ago
        Just because no one's done it before doesn't mean we're not allowed to do it. What Cosmopolitan Libc does is no different from any statically-linked binary on BSD, because they too use the SYSCALL instruction directly in a way that can't be changed at will by the operating system maintainers. The BSD operating systems do not forbid static linking. If you look at their syscalls.master files, a lot of time, particularly with ones like FreeBSD, you'll notice a large number of system calls which are only there to preserve backwards compatibility with statically linked binaries. It's really nice of them to do that. When people talk about the supposed requirement to depend on the platform libc dso, what they're actually talking about is Apple frowning upon statically linked binaries. https://developer.apple.com/library/archive/qa/qa1118/_index.html https://developer.apple.com/library/archive/qa/qa1118/_index... They don't forbid it though, like Microsoft and Fuschia do. Fuschia for instance uses RIP origin detection. Windows does it by changing the RAX ordinals fortnightly. Apple simply asks that we say, hey, if you use Cosmo there's some risk Apple might break our binaries. We take proactive steps to avoid that happening with Cosmo. For example, we don't do some of the things Go did, like reverse engineering the memory layout of Apple's time functions. Cosmo sticks to the APIs that are shared by UNIXes in general, e.g. gettimeofday(), rather than depending on Apple's own internal designs, e.g. Mach system calls. I don't believe Apple can rightfully claim APIs that aren't their own, as being their own implementation detail which they can change at will. We do our best to respect Apple's boundaries, so I believe the risk of breakage with Cosmo on Apple should be minimal.
- vonwoodson 4y ago
  That sounds like using a Mac in general: “Do what you want if your just playing around, but be aware that [if you’re not a blessed developer] macOS is private, subject to change at any time, and should never be used [professionally]”
  - catiopatio 4y ago
    This isn't an Apple thing, it's an "almost every OS but Linux" thing.
    - ithkuil 4y ago
      are you sure? for all things bad about windows, one cannot say that they didn't take backward compatibility quite seriously
      - tinalumfoil 4y ago
        They do change over time, and programs will break between updates on different windows versions of you make syscalls directly. https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/ (click show all)
      - matheusmoreira 4y ago
        Yes. Windows has changed system call numbers before. User space is supposed to use the Microsoft libraries.
- Jolter 4y ago
  Well, isn’t that why this web page exists? To advertise the nice libc implementation so users don’t have to make syscalls directly to the OS? Maybe I’ve misunderstood something…
- euclaise 4y ago
  Same goes for Windows, I believe?
Areading314 4y ago
one of the most cringe things I'll see in "high performance" code is ad hoc syscalls in the middle of loops and such. it should be much more obvious imo in environments like python notebooks that these less scalable operations are happening
ericpruitt 4y ago
I think the title would benefit from mentioning that this is for cosmopolitan libc. I was expecting this to be about syscall portability in general.
- dvratil 4y ago
  Syscalls are provided by the operating system, it doesn't matter which C library you use. The C library usually just provides a nice API to call the syscall. And it it doesn't, you can still call it directly via syscall(1).
  - baq 4y ago
    ...unless the system libc is the syscall interface...
  - kentonv 4y ago
    Yes but that's not what this table is describing. The table appears to be describing cosmopolitan libc's wrappers. Certainly most of these syscalls do not exist on Windows, so the "Windows" column couldn't possibly be describing raw syscalls.
  - deleted 4y ago
    [deleted]
ridiculous_fish 4y ago
The coverage and the attention to detail here is incredible. Hats off to Justine. I would love to know how this table is generated and what the test process is. What's the best way to run tests like this across such a wide variety of OSes? Maybe vagrant images?
- jart 4y ago
  Justine here. I have a fleet of VMs for various OSes running the runitd.com daemon. I then write unit tests in Emacs and when I press CTRL-C CTRL-_ it builds the test, gzips it, and uses runit.com to deploy and run the executable to every VM, and reports back the output within milliseconds. Here's a screenshot of me testing the sendfile() system call: https://justine.lol/cosmopolitan/sendfile-testing.png https://justine.lol/cosmopolitan/sendfile-testing.png When I run `make test` it deploys and runs all 465 test executables to all seven test VMs currently in the fleet, and it takes about 10 seconds. Here's a video: https://storage.googleapis.com/justine/sizetricks/runit.mp4 https://storage.googleapis.com/justine/sizetricks/runit.mp4 All runit.com and runitd.com do (their sources are in tool/build/ of the cosmo github repo) is basically `scp program host: && ssh host ./program` which is what I used to do, except SSH was unacceptably slow, so I wrote runit, which use PSK TLS and DEFLATE to transfer files over the network.
  - davedx 4y ago
    Love your naming of the magic numbers headers. https://github.com/jart/cosmopolitan/blob/1.0/libc/sysv/consts.sh#L2714 https://github.com/jart/cosmopolitan/blob/1.0/libc/sysv/cons...
    - serbrech 4y ago
      that file is incredible :)
      - smnscu 4y ago
        100%, and also how TIL about the Swedish Rhapsody https://www.youtube.com/watch?v=GUQUD3IMbb4&t=85s https://www.youtube.com/watch?v=GUQUD3IMbb4&t=85s -- link at the bottom of the file https://en.wikipedia.org/wiki/Swedish_Rhapsody_(numbers_station) https://en.wikipedia.org/wiki/Swedish_Rhapsody_(numbers_stat...
userbinator 4y ago
Most of the cell labels are self-explanatory, but some of them are a bit puzzling (as is the intended ordering) and a bit amusing: "hairy"? "no cow"?
- corndoge 4y ago
  copy on write
- llanowarelves 4y ago
  If you roll your mouse over an instance of hairy (meaning technical/complicated), the tooltip will give more detail
  - matheusmoreira 4y ago
    What about people using smartphones?
    - Akronymus 4y ago
      Long press?
      - kortex 4y ago
        Doesn't work for me on android, tries to just select the text.
        Akronymus 4y ago
        Oh, its not implemented as alt text but rather some JS. My bad. Thats what I get for assuming.
- sbierwagen 4y ago
  http://www.catb.org/jargon/html/H/hairy.html http://www.catb.org/jargon/html/H/hairy.html "Annoyingly complicated".
zxcvbn4038 4y ago
Not long ago golang switched from doing direct system calls to libc on *BSD systems. The explanation I believe was that the BSD systems fix a lot of kernel problems and compatibility issues in libc and it was just easier to use libc then try to rewrite all of that in the golang standard libraries. On Linux it uses direct system calls for everything.
- catiopatio 4y ago
  I believe the impetus for Golang was macOS and Windows, rather than the BSDs. It’s not that libc is used to fix kernel problems/compatibility issues, but rather, the kernel ABI is not a supported, stable public API. On macOS, there are cases where using the syscall ABI directly will result in your code breaking when interacting with any other code that does correctly use libc, due to out-of-sync userspace state maintained by libc. For example, the fork(2) implementation in macOS’ libSystem will invalidate the cached copy of the current process’ pid used by getpid(2). If you fork(2) by directly trapping to a syscall, the cached pid won’t be invalidated; any future calls to getpid(2) through libSystem will return the stale pid.
  - glandium 4y ago
    Things like this are true on Linux too. Don't call the clone syscall manually because it will mess up with glibc internals.
    - iainmerrick 4y ago
      But only if you’re also using glibc, right? If you’re only using syscalls and no libc at all, you’re fine (on Linux, and only Linux).
      - glandium 4y ago
        If you're only using syscalls on macOS, you're fine too, as far as libSystem is involved. You're not fine if newer kernels change the syscall ABI, though.
      - 4y ago
zx8080 4y ago
Could not find what Metal is (bare-metal?). Can anyone explain it or point out to a relevant docs page?
- sai_c 4y ago
  From the front page: "Your program will also boot on bare metal too. In other words, you've written a normal textbook C program, and thanks to Cosmopolitan's low-level linker magic, you've effectively created your own operating system which happens to run on all the existing ones as well."
  - stevewatson301 4y ago
    Does it have basic networking support (via Ethernet)? That should make for a cool way to build unikernels.
    - Tepix 4y ago
      Looking at the table, it looks as if networking is not yet supported on Metal (socket, bind, connect, accept, etc.)
goddstream 4y ago
Justine's posts are always worth reading, even if their subject is outside your normal area of interest.
- beebmam 4y ago
  - ninjin 4y ago
    Given my (rough) understanding of Justine’s political leanings (active in Occupy [1]), I think (and hope) that you are mistaken. The following searches turn up empty [2,3] as well. [1]: https://www.thenation.com/article/archive/breaking-occupy/ https://www.thenation.com/article/archive/breaking-occupy/ [2]: https://duckduckgo.com/?q=site%3Ajustine.lol+dark https://duckduckgo.com/?q=site%3Ajustine.lol+dark [3]: https://duckduckgo.com/?q=site%3Ajustine.lol+moldbug https://duckduckgo.com/?q=site%3Ajustine.lol+moldbug It is perfectly possible that you are still right. But I would be darn careful to make statements like you made without backing them up with facts. The Internet is a chilling enough place without even more vague rumours going around. Edit: Well, apparently your memory is on to something after all [4,5]. [4]: https://crooksandliars.com/2022/07/donald-j-moldbug-lurks-around-corner https://crooksandliars.com/2022/07/donald-j-moldbug-lurks-ar... [5]: https://www.thedailybeast.com/occupying-the-throne-justine-tunney-neoreactionaries-and-the-new-1 https://www.thedailybeast.com/occupying-the-throne-justine-t...
    - erichocean 4y ago
      - xupybd 4y ago
        It's an interesting article about tech. I don't care about the author's crazy political views but enjoyed the article. Please don't ban something because of an unrelated aspect of the author.
chrisseaton 4y ago
I’m not an expert but I was under the impression these we’re library calls on macOS, and the system calls were unstable and not supposed to be used directly?
deleted 4y ago
[deleted]
Test0129 4y ago
In case the author is reading these comments MalwareBytes blocks justine.lol as riskware for some reason.
- fragmede 4y ago
  They also blocked Google this morning so I'd take that with a grain of salt. https://twitter.com/malwarebytes/status/1572607583093161984 https://twitter.com/malwarebytes/status/1572607583093161984
m4r1k 4y ago
I wonder how would compare a more traditional Unix system like Solaris with all the rest here
trebot 4y ago
What is the "metal" platform listed in the rightmost column? Also what does "zipos" mean, in some of its cells?
- vkoskiv 4y ago
  Cosmopolitan/APE binaries can run on bare metal x86 with no operating system, so that column shows syscall coverage for that use case.
  - trebot 4y ago
    Interesting, so it packages its own minimal OS in the binary? Does it interface with the EFI API surface, or similar?
    - jart 4y ago
      EFI is supported. But in order to use it, you have to trade away the ability to run your binary on Windows.
simplify 4y ago
Incredible output like this is why Justine is my proudest developer sponsorship https://github.com/sponsors/jart https://github.com/sponsors/jart
- soheil 4y ago
  So an HTML table is where you draw the line for deciding which developer you're most proud of? People should study the difference between libc and glibc and history of it. We've been down this path before.
lbotos 4y ago
When they say metal here, what does that actually mean?
- jjice 4y ago
  Bare metal, used to refer to a machine without a traditional operating system on it.
  - nkozyra 4y ago
    Is the state of this still "experimental" as it was ~ year ago or so?
jacobkg 4y ago
I used to work with someone who was an expert in writing cryptography libraries. He insisted on never using any system calls for maximum portability (he also wrote his own memory allocators and optimized for small binary size). Seemed quirky to me but he was proven right multiple times as our ultraportable library let us book some big deals that would have been otherwise infeasible without a big rewrite
- l33t233372 4y ago
  How can you write your own memory allocator in a way that works on Linux and Windows? Don’t you need to request heap space from operating system via system call?
  - ShroudedNight 4y ago
    I suppose if you're willing to malloc big chunks and then potentially never have them released until the process terminates (as free often doesn't do this), one might call that writing one's own memory allocator despite not using mmap / VirtualAlloc.
  - ablob 4y ago
    If you know how much memory you need, you can just have a section of zero-initialized-data for that, which the allocator then handles. The allocator serves also as a proxy, as in: only the allocator needs to know how to request heap space. Portable in that case probably means that you have little to no code change required for a port. Something else to consider is that there may not even be an operating system the program was run on.
    - l33t233372 4y ago
      > there may not even be an operating system the program was run on. Sure, in this case you can just implement malloc the same way a kernel implements me. My failure of understanding is when there _is_ a kernel that your program needs to interact with.
Dwedit 4y ago
Some of the Win32 things mention "millisecond precision only", but this isn't quite correct. Waitable Timers on Win32 let you request time values in units of 200 nanoseconds. See `CreateWaitableTimer`, and `SetWaitableTimer`. `Sleep` and `SleepEx` can be implemented using a waitable timer, just use `WaitForSingleObjectEx`. If you need to wait for an object (semaphore, etc) using a time unit other than milliseconds, you can use `WaitForMultipleObjectsEx` with one of them being a waitable timer. --- The next question is if Win32 can actually deliver those requests for precise times or not. From the testing I did a while ago where I was simulating Sleep, I got actual sleep times rounded to about 4ms. So much for requesting nanosecond level precision.

578 points by signa11 4y ago

Obviously, do whatever you want if you’re just playing around, but please do be aware that the syscall interface on macOS is private, subject to change at any time, and should never be used directly.

deleted 4y ago
[deleted]
arghwhat 4y ago
Same goes for OpenBSD. It is quite normal that the only defined API is the provided libc or similar. This is of course a little annoying at times.
- ithinkso 4y ago
  I just don't understand why, what are the benefits to simply offering syscall interface and let developers decide if they want to link to libc
  - catiopatio 4y ago
    The main benefit is that you can implement complex logic, performance optimizations, and compatibility shims on the userspace side of the kernel/userspace divide, where it's generally much easier to do, and bugs won't result in full compromise of the kernel. There's not really a strong argument for not linking libc. Even if you want to implement your own libc, you can still do so with linker tricks and calling through to the supported syscall wrappers in the real libc.
    - JonChesterfield 4y ago
      Your options are poor if you don't want to build on libc though.
      - catiopatio 4y ago
        Outside unsupported edge cases (e.g. shell code), there's really no material difference between jumping to the address of syscall wrapper in libSystem.dylib, versus trapping to the kernel via a given syscall number — other than the latter being a supported interface, while the former is explicitly not.
        JonChesterfield 4y ago
        Edge cases also include using a different libc, working around bugs in the supported libc, trying to minimise how much of your stack is implemented in C.
        catiopatio 4y ago
        Alternative libc implementations just aren't a thing outside of Linux. However, if, for some reason, you really want to re-implement libc, you still can, but your library will have to link against the system-provided libc for the syscall entry points, because that's the only stable interface to the kernel. There's no reason that should be a problem for a hypothetical libc-reimplementor.
    - matheusmoreira 4y ago
      > There's not really a strong argument for not linking libc. Sure there is. The libc sucks. Freestanding C actually turned out to be a superior language because there's not as much legacy weighing it down. There's many systems languages out there, nobody should be forced to link to C stuff.
      - wongarsu 4y ago
        I think Windows solved this much better than libc by separating the concerns: there's user32.dll providing a thin abstraction over syscalls, a couple libraries providing higher-level interfaces like wintrust.dll, and thirdly the C runtime library providing all the stuff mandated by C. You can easily ditch the latter, while still keeping the benefits of user32.dll (which cosmopolitan is doing).
        lgg 4y ago
        macOS essentially has the same distinction internally (libsystem_kernel.dylib), but that is not directly linkable and is instead re-exported via the libSystem umbrella which also exports libsystem_c.dylib. While you cannot link directly to libsystem_kernel.dylib, if you choose to ignore everything in libsystem_c.dylib and only use the syscall wrappers reexported from libsystem_kernel.dylib via libSystem it has practically the same effect on macOS (in fact, the resulting binary will be identical to what a hypothetical binary linked to just a single libsystem_kernel.dylib would be except for a single `LC_LOAD_DYLIB` command). Such a binary would have the system lib c initialized sitting in its address space, but for the rare binary that really wants its own libc that doesn't seem unreasonable.
      - twic 4y ago
        What's at issue here is linking to the platform libc for syscalls. If you don't want to use the platform libc's qsort implementation, by all means, don't. But if you want to call gettimeofday, you should do that by calling the gettimeofday function in the libc, not by trying to set up a system call on your own. The former is a stable interface, the latter isn't. It's annoying that libc combines two completely different things - userspace utility functions, system calls - but unfortunately, that's where history has brought us.
  - kjeetgill 4y ago
    My take: Like with any interface the definer needs to balance api user ergonomics, your developer's ergonomics, and internal developer's needs. If your api can get a little toe-hold on the remote side there's all sorts of advantages (and compromises) you can leverage. In this case that means the api designer gets a little bit of wiggle room in the process userspace where it might be more appropriate to say shim some calls so that they're no longer 1:1 with a syscall. The obvious example is that malloc() can call syscalls for you and is the "default" way to allocate memory you might want to provide but It's more of it's own little runtime. My favorite example was in my time at LinkedIn. Because the internal Kafka team had thick client libraries, making the company wide change to enable encryption was as simple as pushing out new libraries and deprecating old ones.
  - arghwhat 4y ago
    It significantly simplifies OS development. E.g., syscalls can be modified and improved or entirely deprecated without concern. If OpenBSD wanted to, they could implement something like io_uring with support for all kernel functionality, port libc to use that and ditch conventional syscalls entirely (simulating blocking where needed), without user-space knowing anything changed. Under Linux, which is actually rather unusual in providing ABI guarantees, you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written, and cannot change user-space no matter how good that change would be for either or both.
    - ithinkso 4y ago
      > you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). Also, I understand catiopatio's argument in the sibling comment about doing more in user-space than in kernel in case of a bug, but it breaks down the moment thin wrappers around syscalls exist there. You link to libc/libSystem and use every thin wrapper in existence - now no syscall can be changed (if I understand correctly how macOS works, never used one)
      - simias 4y ago
        I'd much rather export this complexity to userland code than privileged kernel mode code though. It reduces the potential consequence for a mistake pretty drastically.
      - arghwhat 4y ago
        > But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. It is not instead. With syscall ABI you need to maintain both, without it you only need to maintain libc (a majority of which is dictated by POSIX anyway). > In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho). As above, maintaining two contracts is harder than one. Proprietary parts aside, most OS's have their kernel and user-space developed together, and that is exactly what allows them to work this way. This is used to adopt new features, change or deprecate old features with a brutal efficiency that Linux cannot compete with - e.g., when OpenBSD implemented pledge in both kernel and all relevant tools. This is one of the reasons that these projects can keep up or in some cases surpass Linux (FreeBSD networking is seen as superior, and you used to get better performance from running Linux binaries on FreeBSD through its compatibility layer) despite having much smaller groups of maintainers and users.
- weinzierl 4y ago
  What do alternative libc implementations like musl or dietlib do on those systems? Do their maintainers manually track the unstable syscall interface or are developers forced to use the system provided libc because the alternatives just don't work completely?
  - brynet 4y ago
    musl/dietlibc target the Linux kernel. There are no alternative libc implementations for *BSD, unlike Linux the system provided libc is the only libc that matters.
  - deleted 4y ago
    [deleted]
  - arghwhat 4y ago
    On such systems, user-space is developed together with the kernel. The libc is as much part of OpenBSD as the kernel is.
    - jart 4y ago
      Just because no one's done it before doesn't mean we're not allowed to do it. What Cosmopolitan Libc does is no different from any statically-linked binary on BSD, because they too use the SYSCALL instruction directly in a way that can't be changed at will by the operating system maintainers. The BSD operating systems do not forbid static linking. If you look at their syscalls.master files, a lot of time, particularly with ones like FreeBSD, you'll notice a large number of system calls which are only there to preserve backwards compatibility with statically linked binaries. It's really nice of them to do that. When people talk about the supposed requirement to depend on the platform libc dso, what they're actually talking about is Apple frowning upon statically linked binaries. https://developer.apple.com/library/archive/qa/qa1118/_index.html https://developer.apple.com/library/archive/qa/qa1118/_index... They don't forbid it though, like Microsoft and Fuschia do. Fuschia for instance uses RIP origin detection. Windows does it by changing the RAX ordinals fortnightly. Apple simply asks that we say, hey, if you use Cosmo there's some risk Apple might break our binaries. We take proactive steps to avoid that happening with Cosmo. For example, we don't do some of the things Go did, like reverse engineering the memory layout of Apple's time functions. Cosmo sticks to the APIs that are shared by UNIXes in general, e.g. gettimeofday(), rather than depending on Apple's own internal designs, e.g. Mach system calls. I don't believe Apple can rightfully claim APIs that aren't their own, as being their own implementation detail which they can change at will. We do our best to respect Apple's boundaries, so I believe the risk of breakage with Cosmo on Apple should be minimal.
vonwoodson 4y ago
That sounds like using a Mac in general: “Do what you want if your just playing around, but be aware that [if you’re not a blessed developer] macOS is private, subject to change at any time, and should never be used [professionally]”
- catiopatio 4y ago
  This isn't an Apple thing, it's an "almost every OS but Linux" thing.
  - ithkuil 4y ago
    are you sure? for all things bad about windows, one cannot say that they didn't take backward compatibility quite seriously
    - tinalumfoil 4y ago
      They do change over time, and programs will break between updates on different windows versions of you make syscalls directly. https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/ (click show all)
    - matheusmoreira 4y ago
      Yes. Windows has changed system call numbers before. User space is supposed to use the Microsoft libraries.
Jolter 4y ago
Well, isn’t that why this web page exists? To advertise the nice libc implementation so users don’t have to make syscalls directly to the OS? Maybe I’ve misunderstood something…
euclaise 4y ago
Same goes for Windows, I believe?

Areading314 4y ago

one of the most cringe things I'll see in "high performance" code is ad hoc syscalls in the middle of loops and such. it should be much more obvious imo in environments like python notebooks that these less scalable operations are happening

ericpruitt 4y ago

I think the title would benefit from mentioning that this is for cosmopolitan libc. I was expecting this to be about syscall portability in general.

dvratil 4y ago
Syscalls are provided by the operating system, it doesn't matter which C library you use. The C library usually just provides a nice API to call the syscall. And it it doesn't, you can still call it directly via syscall(1).
- baq 4y ago
  ...unless the system libc is the syscall interface...
- kentonv 4y ago
  Yes but that's not what this table is describing. The table appears to be describing cosmopolitan libc's wrappers. Certainly most of these syscalls do not exist on Windows, so the "Windows" column couldn't possibly be describing raw syscalls.
- deleted 4y ago
  [deleted]

ridiculous_fish 4y ago

The coverage and the attention to detail here is incredible. Hats off to Justine. I would love to know how this table is generated and what the test process is. What's the best way to run tests like this across such a wide variety of OSes? Maybe vagrant images?

jart 4y ago
Justine here. I have a fleet of VMs for various OSes running the runitd.com daemon. I then write unit tests in Emacs and when I press CTRL-C CTRL-_ it builds the test, gzips it, and uses runit.com to deploy and run the executable to every VM, and reports back the output within milliseconds. Here's a screenshot of me testing the sendfile() system call: https://justine.lol/cosmopolitan/sendfile-testing.png https://justine.lol/cosmopolitan/sendfile-testing.png When I run `make test` it deploys and runs all 465 test executables to all seven test VMs currently in the fleet, and it takes about 10 seconds. Here's a video: https://storage.googleapis.com/justine/sizetricks/runit.mp4 https://storage.googleapis.com/justine/sizetricks/runit.mp4 All runit.com and runitd.com do (their sources are in tool/build/ of the cosmo github repo) is basically `scp program host: && ssh host ./program` which is what I used to do, except SSH was unacceptably slow, so I wrote runit, which use PSK TLS and DEFLATE to transfer files over the network.
- davedx 4y ago
  Love your naming of the magic numbers headers. https://github.com/jart/cosmopolitan/blob/1.0/libc/sysv/consts.sh#L2714 https://github.com/jart/cosmopolitan/blob/1.0/libc/sysv/cons...
  - serbrech 4y ago
    that file is incredible :)
    - smnscu 4y ago
      100%, and also how TIL about the Swedish Rhapsody https://www.youtube.com/watch?v=GUQUD3IMbb4&t=85s https://www.youtube.com/watch?v=GUQUD3IMbb4&t=85s -- link at the bottom of the file https://en.wikipedia.org/wiki/Swedish_Rhapsody_(numbers_station) https://en.wikipedia.org/wiki/Swedish_Rhapsody_(numbers_stat...

userbinator 4y ago

Most of the cell labels are self-explanatory, but some of them are a bit puzzling (as is the intended ordering) and a bit amusing: "hairy"? "no cow"?

corndoge 4y ago
copy on write
llanowarelves 4y ago
If you roll your mouse over an instance of hairy (meaning technical/complicated), the tooltip will give more detail
- matheusmoreira 4y ago
  What about people using smartphones?
  - Akronymus 4y ago
    Long press?
    - kortex 4y ago
      Doesn't work for me on android, tries to just select the text.
      - Akronymus 4y ago
        Oh, its not implemented as alt text but rather some JS. My bad. Thats what I get for assuming.
sbierwagen 4y ago
http://www.catb.org/jargon/html/H/hairy.html http://www.catb.org/jargon/html/H/hairy.html "Annoyingly complicated".

zxcvbn4038 4y ago

Not long ago golang switched from doing direct system calls to libc on *BSD systems. The explanation I believe was that the BSD systems fix a lot of kernel problems and compatibility issues in libc and it was just easier to use libc then try to rewrite all of that in the golang standard libraries. On Linux it uses direct system calls for everything.

catiopatio 4y ago
I believe the impetus for Golang was macOS and Windows, rather than the BSDs. It’s not that libc is used to fix kernel problems/compatibility issues, but rather, the kernel ABI is not a supported, stable public API. On macOS, there are cases where using the syscall ABI directly will result in your code breaking when interacting with any other code that does correctly use libc, due to out-of-sync userspace state maintained by libc. For example, the fork(2) implementation in macOS’ libSystem will invalidate the cached copy of the current process’ pid used by getpid(2). If you fork(2) by directly trapping to a syscall, the cached pid won’t be invalidated; any future calls to getpid(2) through libSystem will return the stale pid.
- glandium 4y ago
  Things like this are true on Linux too. Don't call the clone syscall manually because it will mess up with glibc internals.
  - iainmerrick 4y ago
    But only if you’re also using glibc, right? If you’re only using syscalls and no libc at all, you’re fine (on Linux, and only Linux).
    - glandium 4y ago
      If you're only using syscalls on macOS, you're fine too, as far as libSystem is involved. You're not fine if newer kernels change the syscall ABI, though.
    - 4y ago

zx8080 4y ago

Could not find what Metal is (bare-metal?). Can anyone explain it or point out to a relevant docs page?

sai_c 4y ago
From the front page: "Your program will also boot on bare metal too. In other words, you've written a normal textbook C program, and thanks to Cosmopolitan's low-level linker magic, you've effectively created your own operating system which happens to run on all the existing ones as well."
- stevewatson301 4y ago
  Does it have basic networking support (via Ethernet)? That should make for a cool way to build unikernels.
  - Tepix 4y ago
    Looking at the table, it looks as if networking is not yet supported on Metal (socket, bind, connect, accept, etc.)

goddstream 4y ago

Justine's posts are always worth reading, even if their subject is outside your normal area of interest.

beebmam 4y ago
- ninjin 4y ago
  Given my (rough) understanding of Justine’s political leanings (active in Occupy [1]), I think (and hope) that you are mistaken. The following searches turn up empty [2,3] as well. [1]: https://www.thenation.com/article/archive/breaking-occupy/ https://www.thenation.com/article/archive/breaking-occupy/ [2]: https://duckduckgo.com/?q=site%3Ajustine.lol+dark https://duckduckgo.com/?q=site%3Ajustine.lol+dark [3]: https://duckduckgo.com/?q=site%3Ajustine.lol+moldbug https://duckduckgo.com/?q=site%3Ajustine.lol+moldbug It is perfectly possible that you are still right. But I would be darn careful to make statements like you made without backing them up with facts. The Internet is a chilling enough place without even more vague rumours going around. Edit: Well, apparently your memory is on to something after all [4,5]. [4]: https://crooksandliars.com/2022/07/donald-j-moldbug-lurks-around-corner https://crooksandliars.com/2022/07/donald-j-moldbug-lurks-ar... [5]: https://www.thedailybeast.com/occupying-the-throne-justine-tunney-neoreactionaries-and-the-new-1 https://www.thedailybeast.com/occupying-the-throne-justine-t...
  - erichocean 4y ago
    - xupybd 4y ago
      It's an interesting article about tech. I don't care about the author's crazy political views but enjoyed the article. Please don't ban something because of an unrelated aspect of the author.

chrisseaton 4y ago

I’m not an expert but I was under the impression these we’re library calls on macOS, and the system calls were unstable and not supposed to be used directly?

deleted 4y ago

[deleted]

Test0129 4y ago

In case the author is reading these comments MalwareBytes blocks justine.lol as riskware for some reason.

fragmede 4y ago
They also blocked Google this morning so I'd take that with a grain of salt. https://twitter.com/malwarebytes/status/1572607583093161984 https://twitter.com/malwarebytes/status/1572607583093161984

m4r1k 4y ago

I wonder how would compare a more traditional Unix system like Solaris with all the rest here

trebot 4y ago

What is the "metal" platform listed in the rightmost column? Also what does "zipos" mean, in some of its cells?

vkoskiv 4y ago
Cosmopolitan/APE binaries can run on bare metal x86 with no operating system, so that column shows syscall coverage for that use case.
- trebot 4y ago
  Interesting, so it packages its own minimal OS in the binary? Does it interface with the EFI API surface, or similar?
  - jart 4y ago
    EFI is supported. But in order to use it, you have to trade away the ability to run your binary on Windows.

simplify 4y ago

Incredible output like this is why Justine is my proudest developer sponsorship https://github.com/sponsors/jart https://github.com/sponsors/jart

soheil 4y ago
So an HTML table is where you draw the line for deciding which developer you're most proud of? People should study the difference between libc and glibc and history of it. We've been down this path before.

lbotos 4y ago

When they say metal here, what does that actually mean?

jjice 4y ago
Bare metal, used to refer to a machine without a traditional operating system on it.
- nkozyra 4y ago
  Is the state of this still "experimental" as it was ~ year ago or so?

jacobkg 4y ago

I used to work with someone who was an expert in writing cryptography libraries. He insisted on never using any system calls for maximum portability (he also wrote his own memory allocators and optimized for small binary size). Seemed quirky to me but he was proven right multiple times as our ultraportable library let us book some big deals that would have been otherwise infeasible without a big rewrite

l33t233372 4y ago
How can you write your own memory allocator in a way that works on Linux and Windows? Don’t you need to request heap space from operating system via system call?
- ShroudedNight 4y ago
  I suppose if you're willing to malloc big chunks and then potentially never have them released until the process terminates (as free often doesn't do this), one might call that writing one's own memory allocator despite not using mmap / VirtualAlloc.
- ablob 4y ago
  If you know how much memory you need, you can just have a section of zero-initialized-data for that, which the allocator then handles. The allocator serves also as a proxy, as in: only the allocator needs to know how to request heap space. Portable in that case probably means that you have little to no code change required for a port. Something else to consider is that there may not even be an operating system the program was run on.
  - l33t233372 4y ago
    > there may not even be an operating system the program was run on. Sure, in this case you can just implement malloc the same way a kernel implements me. My failure of understanding is when there _is_ a kernel that your program needs to interact with.

Dwedit 4y ago

Some of the Win32 things mention "millisecond precision only", but this isn't quite correct. Waitable Timers on Win32 let you request time values in units of 200 nanoseconds. See `CreateWaitableTimer`, and `SetWaitableTimer`. `Sleep` and `SleepEx` can be implemented using a waitable timer, just use `WaitForSingleObjectEx`. If you need to wait for an object (semaphore, etc) using a time unit other than milliseconds, you can use `WaitForMultipleObjectsEx` with one of them being a waitable timer. --- The next question is if Win32 can actually deliver those requests for precise times or not. From the testing I did a while ago where I was simulating Sleep, I got actual sleep times rounded to about 4ms. So much for requesting nanosecond level precision.