4 ms·

> I believe the best chance we have of doing that, is by gluing together the binary interfaces that've already achieved a decades-long consensus, and ignoring t

> I believe the best chance we have of doing that, is by gluing together the binary interfaces that've already achieved a decades-long consensus, and ignoring the APIs. Yeah, this isn't going to fly for OpenBSD, which doesn't have ABI stability, programs are expected to use libc. Program for the API, not the ABI. The author is clever, but misguided.

ChrisSD 6y ago
I believe the claim is that there exists a stable subset: > Bell System Five is the umbrella term we use to describe Linux, FreeBSD, OpenBSD, and Mac OS X which all have nearly-identical application binary interfaces that stood the test of time, having definitions nearly the same as those of AT&T back in the 1980's. However, only Linux guarantees stability at this level so I too am very sceptical that this will "stand the test of time". And of course this completely excludes Windows.
- FeepingCreature 6y ago
  Isn't Windows also known for being committed to both API and ABI stability?
  - mrpippy 6y ago
    Yes, but the stable interface is functions exported from kernel32, ntdll, etc. Syscall numbers can and do change freely between builds.
    - ChrisSD 6y ago
      Indeed. The Win32 API (and UWP et al) is the only one that's officially stable. Kernel syscalls themselves are entirely unstable. Not even just the numbers but the calling convention and everything else. Microsoft reserves the right to add, remove or modify anything in or exposed by the NT kernel. Of course they won't do so frivolously but syscall numbers are especially prone to change with a new build of the kernel. So as with MacOS, BSD, etc the stable interface is provided by userspace libraries.
      - marcan_42 6y ago
        Go tried to break the rules and call macOS syscalls directly (like they do on Linux, because Go is a massively NIH language so they don't use libc where they can get away with it) and it bit them hard when an OS update broke all Go apps.
        jart 6y ago
        If Apple broke every Go program, then that's on Apple. Why are people blaming the victim? Go didn't do anything wrong. Apple came back in the late nineties, in part, because they copied and pasted the magic numbers from the Bell System Five codebase, e.g. 1 for exit(), etc. Capitalizing on UNIX popularity helped Apple succeed. So they shouldn't treat Bell Labs technologies like it's their own implementation detail. For example, Microsoft has the right to break NTDLL because they invented it, and never intended for it to be used directly by app developers. Windows was designed from its beginning to have dynamic system interfaces too. In the UNIX community, DSOs have always been extremely controversial, since they raise legal questions compared to SYSCALL and INT trapping, and they violate the ability of a program to assume complete control of the full virtual address space. If Apple devs are reading, then please, we love your products, but we don't love DSOs. Please grant us the freedom to choose. Please keep SYSCALL stable. It'd also be great if you didn't add special-cased code to XNU that blocks open source Actually Portable Executables, like you did with UPX. Email me (the author) your feedback instead. I'll do anything to help, short of adopting a second WIN32 linkage model.
        saagarjha 6y ago
        Are you familiar with Apple's concept of "private API"? Apple considers the list of exported symbols from a dynamic library to not be a boundary (in fact, they have a linker that ignores that and links against a predefined list of symbols regardless of what a binary exports), much less the kernel syscall interface. In fact, Apple silicon requires that your program is loaded by dyld as far as I can tell. While I wan't around in the nineties, I'm curious how syscall numbering helped Apple's success. I'm sure System-V ABI when they switched to 64-bit Intel was probably a nice bonus, but system call numbers?
    - FeepingCreature 6y ago
      Might this change now that things like game copy protections are implementing syscalls manually?
      - ChrisSD 6y ago
        Doubtful. There have long been software that abuses syscalls and other kernel level features to implement Sony rootkits and other DRM. Ironically those that implement game copy protection would be annoyed if syscalls were stabilized because the whole reason they use unstable syscalls is to obfuscate what they're doing and make the program extremely sensitive to tampering.
- trasz 6y ago
  FreeBSD guarantees syscall ABI stability too.
  - ChrisSD 6y ago
    Do you have a source for that? The closest I could find suggests otherwise, but admittedly I haven't yet found a more relevant document on policy: > No compatibility for API and ABI is guaranteed from one to the next major release, though an effort is made to make the upgrade process and source code changes as untroubled as possible. https://wiki.freebsd.org/VendorInformation https://wiki.freebsd.org/VendorInformation
    - trasz 6y ago
      It’s pretty obvious from the fact that people are routinely running jails with old userspace (eg 11.0) under newer kernels (eg 13.0). It doesn’t always work for management interfaces (so eg an old route(8) binary might be broken), but is guaranteed to work for normal user programs.
Zanneth 6y ago
I think the intent was to demonstrate an idea through a compelling proof-of-concept. I don’t think the author or anyone expects this to be usable in production settings (yet).

ChrisSD 6y ago

I believe the claim is that there exists a stable subset: > Bell System Five is the umbrella term we use to describe Linux, FreeBSD, OpenBSD, and Mac OS X which all have nearly-identical application binary interfaces that stood the test of time, having definitions nearly the same as those of AT&T back in the 1980's. However, only Linux guarantees stability at this level so I too am very sceptical that this will "stand the test of time". And of course this completely excludes Windows.

FeepingCreature 6y ago
Isn't Windows also known for being committed to both API and ABI stability?
- mrpippy 6y ago
  Yes, but the stable interface is functions exported from kernel32, ntdll, etc. Syscall numbers can and do change freely between builds.
  - ChrisSD 6y ago
    Indeed. The Win32 API (and UWP et al) is the only one that's officially stable. Kernel syscalls themselves are entirely unstable. Not even just the numbers but the calling convention and everything else. Microsoft reserves the right to add, remove or modify anything in or exposed by the NT kernel. Of course they won't do so frivolously but syscall numbers are especially prone to change with a new build of the kernel. So as with MacOS, BSD, etc the stable interface is provided by userspace libraries.
    - marcan_42 6y ago
      Go tried to break the rules and call macOS syscalls directly (like they do on Linux, because Go is a massively NIH language so they don't use libc where they can get away with it) and it bit them hard when an OS update broke all Go apps.
      - jart 6y ago
        If Apple broke every Go program, then that's on Apple. Why are people blaming the victim? Go didn't do anything wrong. Apple came back in the late nineties, in part, because they copied and pasted the magic numbers from the Bell System Five codebase, e.g. 1 for exit(), etc. Capitalizing on UNIX popularity helped Apple succeed. So they shouldn't treat Bell Labs technologies like it's their own implementation detail. For example, Microsoft has the right to break NTDLL because they invented it, and never intended for it to be used directly by app developers. Windows was designed from its beginning to have dynamic system interfaces too. In the UNIX community, DSOs have always been extremely controversial, since they raise legal questions compared to SYSCALL and INT trapping, and they violate the ability of a program to assume complete control of the full virtual address space. If Apple devs are reading, then please, we love your products, but we don't love DSOs. Please grant us the freedom to choose. Please keep SYSCALL stable. It'd also be great if you didn't add special-cased code to XNU that blocks open source Actually Portable Executables, like you did with UPX. Email me (the author) your feedback instead. I'll do anything to help, short of adopting a second WIN32 linkage model.
        saagarjha 6y ago
        Are you familiar with Apple's concept of "private API"? Apple considers the list of exported symbols from a dynamic library to not be a boundary (in fact, they have a linker that ignores that and links against a predefined list of symbols regardless of what a binary exports), much less the kernel syscall interface. In fact, Apple silicon requires that your program is loaded by dyld as far as I can tell. While I wan't around in the nineties, I'm curious how syscall numbering helped Apple's success. I'm sure System-V ABI when they switched to 64-bit Intel was probably a nice bonus, but system call numbers?
  - FeepingCreature 6y ago
    Might this change now that things like game copy protections are implementing syscalls manually?
    - ChrisSD 6y ago
      Doubtful. There have long been software that abuses syscalls and other kernel level features to implement Sony rootkits and other DRM. Ironically those that implement game copy protection would be annoyed if syscalls were stabilized because the whole reason they use unstable syscalls is to obfuscate what they're doing and make the program extremely sensitive to tampering.
trasz 6y ago
FreeBSD guarantees syscall ABI stability too.
- ChrisSD 6y ago
  Do you have a source for that? The closest I could find suggests otherwise, but admittedly I haven't yet found a more relevant document on policy: > No compatibility for API and ABI is guaranteed from one to the next major release, though an effort is made to make the upgrade process and source code changes as untroubled as possible. https://wiki.freebsd.org/VendorInformation https://wiki.freebsd.org/VendorInformation
  - trasz 6y ago
    It’s pretty obvious from the fact that people are routinely running jails with old userspace (eg 11.0) under newer kernels (eg 13.0). It doesn’t always work for management interfaces (so eg an old route(8) binary might be broken), but is guaranteed to work for normal user programs.

Zanneth 6y ago

I think the intent was to demonstrate an idea through a compelling proof-of-concept. I don’t think the author or anyone expects this to be usable in production settings (yet).