3 ms·

Let me crawl out of the woodwork to ask a question about the consequences: I understood Windows changed its syscall API a few times. How do you deal with that

Let me crawl out of the woodwork to ask a question about the consequences: I understood Windows changed its syscall API a few times. How do you deal with that? What happens if they renumber again?

Quekid5 4y ago
Just because I'm a bit curious and don't know the history here: Did they need to renumber, and if so... why? (I'm not arguing for nor against syscalls-as-kernel-API, I'm honestly just curious. Without knowing too much about it, it seems pretty sensible to support statically linked executables, but I might be missing something. I guess linking against an "as old as you need to support" libc dynamically isn't the end of the world, but it does constrain the build environment somewhat.)
- catiopatio 4y ago
  > I guess linking against an "as old as you need to support" libc dynamically isn't the end of the world, but it does constrain the build environment somewhat. I can't speak for Windows, but on macOS, you build for older targets by passing `-(mmacos|ios)-version-min=` to the compiler to specify your deployment target version; this is used to determine symbol visibility, API visibility, toggled #ifdefs, etc. The provided version is also stored in the Mach-O load commands of your executable, and will used to select compatible symbols (and enable compatibility shims) when loading your binary's image at runtime. No need for static linking — or jumping through hoops to build against an "as old as you need to support" set of installed libraries.
- wongarsu 4y ago
  I'm pretty sure Windows just automatically generates syscall numbers in their build process. Since the kernel and the dynamic libraries that are the only blessed way to interact with the kernel are shipped together, that's not a problem from Microsoft's point of view. For example, look at [1] and search for e.g. 0x01aa, to see how the meaning of that syscall number changes in a pretty systematic way over releases. 1: https://github.com/j00ru/windows-syscalls/blob/master/x86/csv/nt.csv https://github.com/j00ru/windows-syscalls/blob/master/x86/cs...
- jsmith45 4y ago
  They don't per se need to renumber system calls, but for the most part the NT kernel developers saw little reason to enforce syscall number stability. Use of DLLs is baked VERY deeply into windows. The kernel is not even a monolithic file. It is an exe like most others pulling in dlls that implement other functionality. It even features like API-set DLL redirections which mean the bootloader for the kernel needs to implement a fair bit of the PE loader functionality of the kernel just to load the kernel. (I've no real clue if there is shared code between the two loaders, or if they are two separate loaders that implement similar things. Many of the options/features of the full kernel executable loader are not really needed in the bootloader.) So it is not much of a surprise that making the numbers stable to support fully statically linked executables was not really something they cared about. From their perspective you could always just pull in the ntdll.dll for your syscalls like they intended (or more likely a higher level win32 dll that uses the syscall).
jart 4y ago
We don't use SYSCALL on Windows. We link KERNEL32 and friends. It adds bloat to our binaries (for instance, hello world could be 4kb rather than 16kb if we didn't need DLLs) but Microsoft leaves us with little other option, since it'd require just as much bloat to use Xed to wrangle the syscall ordinals out of NTDLL. Our policy is to stick with stable supported interfaces whenever possible. Since XNU's SYSCALL interface is allowed, and it's nearly identical to those of Linux and BSDs, it'd be tragic to not use something that fits so perfectly hand in glove with the rest of our system call support. Look here, all we need to support a XNU system call is 16-bits of a word. https://github.com/jart/cosmopolitan/blob/2.1/libc/sysv/syscalls.sh https://github.com/jart/cosmopolitan/blob/2.1/libc/sysv/sysc...
- lgg 4y ago
  The macOS syscall interface policy is the same as Windows. In practice it has had less churn then then NTDLL which may make it feel more practical, but it is only more "allowed" in the sense that it is technically feasible, not that it is guaranteed to be stable or supported.
- brynet 4y ago
  > Our policy is to stick with stable supported interfaces whenever possible Except you're not doing that consistently though. On OpenBSD, mincore(2) was removed 3 years ago, and it's UNIMPL syscall 78 was eventually recycled by a different system call: mquery(2), which your library calls expecting mincore and passes bogus arguments to. I would be very surprised if there aren't more serious mistakes lurking in your library, in fact I know there are. https://github.com/openbsd/src/commit/54e4f6b9a1dc183e1dc7c4a722c2646aa309246a https://github.com/openbsd/src/commit/54e4f6b9a1dc183e1dc7c4... https://github.com/openbsd/src/commit/1d60349d0b961891264d426ffe1c0ced24b2374c https://github.com/openbsd/src/commit/1d60349d0b961891264d42...
  - anamexis 4y ago
    From the OP chart, it looks like mincore is indeed not supported on OpenBSD, so what's the issue?

Let me crawl out of the woodwork to ask a question about the consequences: I understood Windows changed its syscall API a few times. How do you deal with that? What happens if they renumber again?

Quekid5 4y ago

Just because I'm a bit curious and don't know the history here: Did they need to renumber, and if so... why? (I'm not arguing for nor against syscalls-as-kernel-API, I'm honestly just curious. Without knowing too much about it, it seems pretty sensible to support statically linked executables, but I might be missing something. I guess linking against an "as old as you need to support" libc dynamically isn't the end of the world, but it does constrain the build environment somewhat.)

catiopatio 4y ago
> I guess linking against an "as old as you need to support" libc dynamically isn't the end of the world, but it does constrain the build environment somewhat. I can't speak for Windows, but on macOS, you build for older targets by passing `-(mmacos|ios)-version-min=` to the compiler to specify your deployment target version; this is used to determine symbol visibility, API visibility, toggled #ifdefs, etc. The provided version is also stored in the Mach-O load commands of your executable, and will used to select compatible symbols (and enable compatibility shims) when loading your binary's image at runtime. No need for static linking — or jumping through hoops to build against an "as old as you need to support" set of installed libraries.
wongarsu 4y ago
I'm pretty sure Windows just automatically generates syscall numbers in their build process. Since the kernel and the dynamic libraries that are the only blessed way to interact with the kernel are shipped together, that's not a problem from Microsoft's point of view. For example, look at [1] and search for e.g. 0x01aa, to see how the meaning of that syscall number changes in a pretty systematic way over releases. 1: https://github.com/j00ru/windows-syscalls/blob/master/x86/csv/nt.csv https://github.com/j00ru/windows-syscalls/blob/master/x86/cs...
jsmith45 4y ago
They don't per se need to renumber system calls, but for the most part the NT kernel developers saw little reason to enforce syscall number stability. Use of DLLs is baked VERY deeply into windows. The kernel is not even a monolithic file. It is an exe like most others pulling in dlls that implement other functionality. It even features like API-set DLL redirections which mean the bootloader for the kernel needs to implement a fair bit of the PE loader functionality of the kernel just to load the kernel. (I've no real clue if there is shared code between the two loaders, or if they are two separate loaders that implement similar things. Many of the options/features of the full kernel executable loader are not really needed in the bootloader.) So it is not much of a surprise that making the numbers stable to support fully statically linked executables was not really something they cared about. From their perspective you could always just pull in the ntdll.dll for your syscalls like they intended (or more likely a higher level win32 dll that uses the syscall).

jart 4y ago

We don't use SYSCALL on Windows. We link KERNEL32 and friends. It adds bloat to our binaries (for instance, hello world could be 4kb rather than 16kb if we didn't need DLLs) but Microsoft leaves us with little other option, since it'd require just as much bloat to use Xed to wrangle the syscall ordinals out of NTDLL. Our policy is to stick with stable supported interfaces whenever possible. Since XNU's SYSCALL interface is allowed, and it's nearly identical to those of Linux and BSDs, it'd be tragic to not use something that fits so perfectly hand in glove with the rest of our system call support. Look here, all we need to support a XNU system call is 16-bits of a word. https://github.com/jart/cosmopolitan/blob/2.1/libc/sysv/syscalls.sh https://github.com/jart/cosmopolitan/blob/2.1/libc/sysv/sysc...

lgg 4y ago
The macOS syscall interface policy is the same as Windows. In practice it has had less churn then then NTDLL which may make it feel more practical, but it is only more "allowed" in the sense that it is technically feasible, not that it is guaranteed to be stable or supported.
brynet 4y ago
> Our policy is to stick with stable supported interfaces whenever possible Except you're not doing that consistently though. On OpenBSD, mincore(2) was removed 3 years ago, and it's UNIMPL syscall 78 was eventually recycled by a different system call: mquery(2), which your library calls expecting mincore and passes bogus arguments to. I would be very surprised if there aren't more serious mistakes lurking in your library, in fact I know there are. https://github.com/openbsd/src/commit/54e4f6b9a1dc183e1dc7c4a722c2646aa309246a https://github.com/openbsd/src/commit/54e4f6b9a1dc183e1dc7c4... https://github.com/openbsd/src/commit/1d60349d0b961891264d426ffe1c0ced24b2374c https://github.com/openbsd/src/commit/1d60349d0b961891264d42...
- anamexis 4y ago
  From the OP chart, it looks like mincore is indeed not supported on OpenBSD, so what's the issue?