8 ms·
There's no good reason for a Unix system to have an unstable system call ABI. I'm sorry but companies like Apple (who broke every Go binary a few years ago) don
by jart 6y ago
There's no good reason for a Unix system to have an unstable system call ABI. I'm sorry but companies like Apple (who broke every Go binary a few years ago) don't get to copy syscall definitions from Bell System V and then declare it their own internal API.
Usually the only time the SYSCALL ABI breaks, it's because kernel authors intentionally chose to do it for no apparent reason. For example, OpenBSD at some point changed how mmap() was defined so that it takes seven arguments:
void *sys_mmap(void *addr, size_t len, int prot,
int flags, int fd, long pad, off_t pos);
The sixth argument doesn't do anything. It just breaks binary compatibility. It's also noncompliant with the System V ABI specification, which says system calls have six arguments max.
I work on a project called Cosmopolitan Libc which lets you create static binaries that just work on Linux + Mac + Windows + FreeBSD + OpenBSD. It was only possible to do this because Unix systems generally agree on definitions. I worked really hard to support OpenBSD since I believe in the project. I just hope they keep the ABI stable going forward.
For example, I'm really happy that this restriction only applies to dynamic binaries. It seems perfectly reasonable that they'd want to make the assumption that if a program chooses to link OpenBSD's Libc that it intends to use it. That's fine just so long as we continue having the ability to build static binaries with an alternative cross-platform Libc like Cosmopolitan.
Speaking of which, I think I might actually implement some of OpenBSD's ideas in Cosmopolitan. I could probably track down all the functions that need raw SYSCALL and use __section__ so they're all linked to the same part of the binary and then call the msyscall() function to limit it just to that page. That way as a guest libc author I'm upholding the spirit of the intent. When in Rome do as the Romans.
- tedunangst 6y agoYou should perhaps look into the history of the mmap syscall arguments.
- jart 6y ago4.2BSD System Manual specifies it as mmap(addr, len, prot, share, fd, pos). There are six arguments. My best guess is something went horribly wrong with off_t on big endian systems and that it somehow leaked into x86_64 system call abi. Do you know?
- tedunangst 6y agooff_t needs 64bit alignment so registers get copied to the stack argument structure correctly on kernel entry. This is ancient voodoo.
- josefx 6y ago> The sixth argument doesn't do anything. It just breaks binary compatibility. As far as I can tell it might be used to align the memory layout of the following 64 bit arguments to 64 bits. Or at least ensure that there is no auto generated padding that might contain random values. Edit: The link provided by ainar-g 3 mentions that the way gcc handled padding of the offset field changed between gcc 1 and 2.
- ainar-g 6y ago> For example, OpenBSD at some point changed how mmap() was defined so that it takes seven arguments: It seems like that was the case from day one, when a copy of the NetBSD code was imported to later become OpenBSD[1]. And if my reasoning is correct, saying that OpenBSD “changed it at some point” is not quite correct. https://github.com/openbsd/src/blob/df930be708d50e9715f173caa26ffe1b7599b157/lib/libc/sys/mmap.c https://github.com/openbsd/src/blob/df930be708d50e9715f173ca...
- comfydragon 6y agoBut, that link shows mmap taking 6 arguments?
- ainar-g 6y agoThe parent comment talked about the syscall, not the library function. return((caddr_t)(long)__syscall((quad_t)SYS_mmap, addr, len, prot, flags, fd, 0, offset)); The first argument is the syscall number, the rest are the seven arguments in question.
- ChrisSD 6y ago> Apple (who broke every Go binary a few years ago) That's a really weird way to phrase it. Apple says that interface is unstable. Go uses it anyway. The unstable interface turns out to be unstable. It wasn't Apple that broke Go binaries. It was Go.
- mhh__ 6y agoJust because they don't maintain it stably doesn't mean they shouldn't do so.
- ChrisSD 6y agoWishing Apple did something differently does not make it so. Either you accept the stable interface for what it is or you accept your binaries may break. And you've only yourself to "blame" in the latter case.
- mhh__ 6y agoI don't have to blame anyone because I don't own any Apple products, what I'm getting at is that Apple's "Minimal Documentation, force through our blessed channels" way of going about things is kind of remarkable given that microsoft have been absolutely slammed for similar actions in the past.
- my123 6y agoBSDs traditionally don't have a stable syscall ABI. Windows never had a stable syscall ABI either. Almost only Linux does it... because the kernel and libc are maintained as separate projects in the Linux world.
- mhh__ 6y agoI don't really do OS development so I was mostly using this as a microcosm of the wider practice. Practically as long as it's trivially callable from C I'm not bothered.
- quotemstr 6y agoThere's no reason for an operating system to have a stable system call ABI. A stable ABI means that the kernel support boundary grows forever and that userspace shims are impossible in the general case. And what's the point? Calling through libc or ntdll or whatever is appropriate on a given system is no great burden. Go's libc avoidance is just hubris.
- jart 6y agoEven if you use the system libc, if you pass the -static flag to gcc then you end up with a binary that depends on the syscall abi. If the kernel interface breaks, then all your programs need to be recompiled from scratch in order for them to work again. Are you opposed to static linking?
- quotemstr 6y agoYes, I am opposed to fully static linking. What's the point of static linking? Windows has no static linking (all system calls go through ntdll) and it has a compatibility story better than any Unix. Static linking to libc is unnecessary for long term ABI support. Dynamically link against libc and statically link the rest for all I care, but there's no reason not to talk to libc. Also: the vDSO is also a form of dynamic linking. Are you opposed to the vDSO?
- jart 6y agoI distribute binaries. My binaries work on six different operating systems. In order to do that I had to roll my own C library. I'm happy I did that since it's so much better than being required to use six different ones. I'm not opposed to vDSO but I disagree with how Linux maps it into memory by default. Linux should not be putting anything into the address space that the executable does not specify. MMUs are designed to give each process its own address space. Operating systems that violate that assumption are leaky abstractions imposing requirements where they shouldn't. The main thing dynamic shared objects accomplish is giving upstream dependencies leverage over your software. They have a long history of being mandated by legal requirements such as LGPL and Microsoft EULAs. It's nice to have the freedom to not link the things.
- saagarjha 6y agoYou've brought this up before, but I still don't have insight into why you say that copying system call definitions from Bell (if there was ever such a thing, POSIX standardized above that level for a reason…). macOS, Windows, the BSDs, all declare their stable ABI to be above the syscall layer. Why must it be defined lower?
- jart 6y agoNo they don't. Only Windows forbids developers from linking static binaries. That's because they change the SYSCALL ordinals every few months. Which means that in order to load a program on Windows you have to give up control of the virtual memory address space to the operating system so that it can load DLLs at arbitrary addresses. Unix systems have never had that restriction. Because if you use the official libc and pass -static to gcc then that means the "stable api" creates a binary that depends on the kernel abi. If the kernel authors break the abi then it means you need to recompile all your software in order to upgrade.
- masklinn 6y ago> No they don't. Yes they do. > Only Windows forbids developers from linking static binaries. That's because they change the SYSCALL ordinals every few months. That Windows somewhat actively precludes raw syscalls doesn’t mean they are supported elsewhere. It’s always been bsd (and especially macOS) policy that syscalls are an implementation detail. > Because if you use the official libc and pass -static to gcc then that means the "stable api" creates a binary that depends on the kernel abi. Try doing that on macOS, you’ll find out that there is no static version of libSystem, or crt0.
- pjmlp 6y agoPlenty of OS allow for static linking, including Windows.
- leeter 6y agoWindows does it because the vast majority of functionality is not implemented in the kernel per se, but rather in user space by preference for stability. Even things like display drivers are often primarily implemented in userspace. The main advantage of not having a stable kernel ABI is that you have the freedom to change it for security/performance reasons. This means the OS can change how they implement things over time without breaking things as easily. MS is notorious for this and for using this fact to allow them to 'emulate' older ways of doing things even when the real implementation has long since moved on. The main advantage of having a stable kernel ABI is that the userspace can be whatever the developer wants it to be realistically. If they want it to be just a single GO program and literally nothing else they can do that (routers are a good example of devices that do this).