7 ms·

> How do they deal with the syscall numbers being different from release to release, while still maintaining binary compatibility By not calling syscalls based

> How do they deal with the syscall numbers being different from release to release, while still maintaining binary compatibility By not calling syscalls based on their numbers on Windows. That might sound crazy, but actually to me coming from Windows, calling them by ordinals is what sounds crazy. :-) You're supposed to call exported functions in shared libraries (ntdll.dll etc.) that call the syscalls for you.

rocqua 7y ago
To be fair, that is how most of linux tends to work. With glib wrappers around syscalls. It's just that the numbers are considered part of the stable interface. Hence people can write their own wrappers. It's part of linux saying 'we don't break userspace' and the syscall numbers being how userspace talks to kernelspace.
- mehrdadn 7y ago
  Given glibc isn't part of the OS, and that hence you can statically link to it, it can't be part of the stable interface. glibc isn't a thing as far as the OS is concerned. It seems to me this also means Linux is more limited in what it can do in breaking syscalls too. On Windows, if you load any shared library (including ntdll) and an exported function isn't there, the loader can produce an error for you telling you that a function you need isn't there. On Linux... you're calling the syscall directly, so there's nothing that can stand between you and the syscalls to perform a check or anything. Overall it seems to me like coding against direct syscall numbers is a disadvantage on almost every front.
  - caf 7y ago
    So where this affects things is when you want to take advantage of a newly-added syscall, or a newly-added feature of an existing syscall. In the second case, both systems behave the same - if your program runs on an older kernel, the function entry point exists (either the syscall or ntdll symbol) but when you call the function to request the new feature, it returns a runtime error. In the first case, Linux behaves identically to the first case - you get a runtime error (ENOSYS) - whereas Windows will produce an error at load time. Typically what is done is that whatever is wrapping the syscall falls back to an alternate implementation for older kernels, if that's possible. If it's not, the program can either continue without that feature if it wants, or fail with a message that a newer kernel is required. I don't feel that there's really a significant difference in the two approaches - wherever you draw the backwards-compatibility line you're going to have to do the same sort of work to maintain it.
  - account42 7y ago
    > Given glibc isn't part of the OS, and that hence you can statically link to it Unless you want to use any other library on the OS, for example 3D graphics drivers which tend to have hardware-specific parts in userspace. > Overall it seems to me like coding against direct syscall numbers is a disadvantage on almost every front. Which is why noone does it unless they have a very good reason.
    - mehrdadn 7y ago
      > Unless you want to use any other library on the OS, for example 3D graphics drivers which tend to have hardware-specific parts in userspace. Not sure what you mean regarding this and glibc. > Which is why noone does it unless they have a very good reason. I think you changed the meaning of my sentence to be able to make this reply. I was referring to what Linux does re: syscalls vs. library exports, not hard-coding syscall numbers in the source code.
- asveikau 7y ago
  Nitpick: glibc is the GNU C library and wraps syscalls, among other things. glib is a bunch of cross platform interfaces and data structures that grew out of Gtk+.
ninkendo 7y ago
So that means ntdll.dll and family can never be statically linked? I guess that makes sense if the only responsibility of the dll is to expose syscalls, but does ntdll expose any other functionality? Functionality that may want to be overridden by a different implementation, etc? In the Linux world, since syscalls are exposed by libc, other non-C runtimes (for example, Go) end up needing to make system calls on their own without the libc wrappers. Is such a system just not a thing in Windows?
- slovenlyrobot 7y ago
  It's exactly the same deal on OS X. There is no stable system call interface that is not the C library, and it's one of the main reasons they don't allow static linking there either
- mehrdadn 7y ago
  > In the Linux world, since syscalls are exposed by libc, other non-C runtimes (for example, Go) end up needing to make system calls on their own without the libc wrappers. Is such a system just not a thing in Windows? Yeah, it's not a thing. You just import functions from ntdll the same way you import from anything else. The OS program loader reads your program's import table and loads them for you. > does ntdll expose any other functionality? Functionality that may want to be overridden by a different implementation, etc? It exposes lots of other stuff (DLL loading, memory heap, etc.) that you may want to do differently, but you can hook those at runtime if you really want to. It's not as convenient as just linking another library, but on the other hand, it makes it easier to replace them at runtime instead of at compile-time -- say, if you want a shared library to override a syscall.
- jcranmer 7y ago
  > In the Linux world, since syscalls are exposed by libc, other non-C runtimes (for example, Go) end up needing to make system calls on their own without the libc wrappers. Is such a system just not a thing in Windows? Linux is pretty much the only operating system that considers the actual system call ABI to be stable. The BSDs and OS X all want you to use the platform libc instead to access system calls, while Windows uses kernel32.dll et al. as its stable interface.
asveikau 7y ago
Ntdll being the de facto syscall abi is the right answer. But ntdll is also technically subject to change, much of it not publicly documented by Microsoft. So typically a third party application calls into things like kernel32 or advapi (though Win7 and later split those into different DLLs internally) which wraps ntdll which does the actual ordinal calls. But generally speaking, yeah, the stable interfaces are the C ones, exposed through system DLLs. Linux is unique in developing libc and kernel separately and the division between the two as a public abi boundary.
- mehrdadn 7y ago
  I mean, you could say anything is subject to change. The "documented" <-> "stable" mapping isn't a bijection. CreateFileTransacted was documented for kernel32 but that didn't stop it from being claimed as deprecated. And CallNtPowerInformation was documented and yet some of it simply doesn't work anymore. Also, Microsoft didn't necessarily document everything, but they did provide .lib files (and often headers) even for "undocumented" APIs in these libraries. Breaking these APIs would break programs they previously provided SDKs for. The reality is, so many third-party applications depend directly on ntdll APIs (even Chrome) and some of them literally cannot work with kernel32 stuff (boot-time partition managers, for one), so as far as I'm concerned, it's about as much set in stone as any library could be.
  - asveikau 7y ago
    Yeah. Direct dependence on ntdll occurs in the wild and it would break stuff to change it. It can also break MS's own code and potentially create massive engineering hurdles for them that are completely unnecessary. But they have always warned people in documentation that they don't consider it a stable interface to the same extent as Win32. In my experience a lot of the NT APIs are nicer, better thought out, more direct.
    - KyleJ61782 7y ago
      Using the NTDLL calls breaks stuff because NTDLL doesn't understand anything about the overlaid subsystems. NT was originally designed to support multiple subsystems in addition to Win32, like the Interix (POSIX compliant) subsystem and the OS/2 subsystem I believe. I think (I could be wrong on this) that this was possible by switching the system call table depending upon which process was currently active (doable by switching the system call vector).
masklinn 7y ago
> By not calling syscalls based on their numbers on Windows. That might sound crazy Not really. Only on Linux are direct syscalls an official ABI. On most unices (e.g. solaris, BSDs, OSX) you must dynamically link and go through the platform's libc.
- earenndil 7y ago
  I don't know about solaris, and I know macos had kernel changes which broke go, but freebsd has a stable kernel interface and unstable userspace. (So does linux, for that matter--ever try running a program linked to a newer version of glibc?)
  - account42 7y ago
    > So does linux, for that matter--ever try running a program linked to a newer version of glibc? That doesn't mean the interface isn't stable or at least not less stable than the kernel interface or literally any iterface for something that is still being developed. It would be nice if glibc would provide an easy way to target the ABI of an older version (which is compatible with all newer versions) but you can do that the manual way by compiling and linking against that old version.

rocqua 7y ago

To be fair, that is how most of linux tends to work. With glib wrappers around syscalls. It's just that the numbers are considered part of the stable interface. Hence people can write their own wrappers. It's part of linux saying 'we don't break userspace' and the syscall numbers being how userspace talks to kernelspace.

mehrdadn 7y ago
Given glibc isn't part of the OS, and that hence you can statically link to it, it can't be part of the stable interface. glibc isn't a thing as far as the OS is concerned. It seems to me this also means Linux is more limited in what it can do in breaking syscalls too. On Windows, if you load any shared library (including ntdll) and an exported function isn't there, the loader can produce an error for you telling you that a function you need isn't there. On Linux... you're calling the syscall directly, so there's nothing that can stand between you and the syscalls to perform a check or anything. Overall it seems to me like coding against direct syscall numbers is a disadvantage on almost every front.
- caf 7y ago
  So where this affects things is when you want to take advantage of a newly-added syscall, or a newly-added feature of an existing syscall. In the second case, both systems behave the same - if your program runs on an older kernel, the function entry point exists (either the syscall or ntdll symbol) but when you call the function to request the new feature, it returns a runtime error. In the first case, Linux behaves identically to the first case - you get a runtime error (ENOSYS) - whereas Windows will produce an error at load time. Typically what is done is that whatever is wrapping the syscall falls back to an alternate implementation for older kernels, if that's possible. If it's not, the program can either continue without that feature if it wants, or fail with a message that a newer kernel is required. I don't feel that there's really a significant difference in the two approaches - wherever you draw the backwards-compatibility line you're going to have to do the same sort of work to maintain it.
- account42 7y ago
  > Given glibc isn't part of the OS, and that hence you can statically link to it Unless you want to use any other library on the OS, for example 3D graphics drivers which tend to have hardware-specific parts in userspace. > Overall it seems to me like coding against direct syscall numbers is a disadvantage on almost every front. Which is why noone does it unless they have a very good reason.
  - mehrdadn 7y ago
    > Unless you want to use any other library on the OS, for example 3D graphics drivers which tend to have hardware-specific parts in userspace. Not sure what you mean regarding this and glibc. > Which is why noone does it unless they have a very good reason. I think you changed the meaning of my sentence to be able to make this reply. I was referring to what Linux does re: syscalls vs. library exports, not hard-coding syscall numbers in the source code.
asveikau 7y ago
Nitpick: glibc is the GNU C library and wraps syscalls, among other things. glib is a bunch of cross platform interfaces and data structures that grew out of Gtk+.

ninkendo 7y ago

So that means ntdll.dll and family can never be statically linked? I guess that makes sense if the only responsibility of the dll is to expose syscalls, but does ntdll expose any other functionality? Functionality that may want to be overridden by a different implementation, etc? In the Linux world, since syscalls are exposed by libc, other non-C runtimes (for example, Go) end up needing to make system calls on their own without the libc wrappers. Is such a system just not a thing in Windows?

slovenlyrobot 7y ago
It's exactly the same deal on OS X. There is no stable system call interface that is not the C library, and it's one of the main reasons they don't allow static linking there either
mehrdadn 7y ago
> In the Linux world, since syscalls are exposed by libc, other non-C runtimes (for example, Go) end up needing to make system calls on their own without the libc wrappers. Is such a system just not a thing in Windows? Yeah, it's not a thing. You just import functions from ntdll the same way you import from anything else. The OS program loader reads your program's import table and loads them for you. > does ntdll expose any other functionality? Functionality that may want to be overridden by a different implementation, etc? It exposes lots of other stuff (DLL loading, memory heap, etc.) that you may want to do differently, but you can hook those at runtime if you really want to. It's not as convenient as just linking another library, but on the other hand, it makes it easier to replace them at runtime instead of at compile-time -- say, if you want a shared library to override a syscall.
jcranmer 7y ago
> In the Linux world, since syscalls are exposed by libc, other non-C runtimes (for example, Go) end up needing to make system calls on their own without the libc wrappers. Is such a system just not a thing in Windows? Linux is pretty much the only operating system that considers the actual system call ABI to be stable. The BSDs and OS X all want you to use the platform libc instead to access system calls, while Windows uses kernel32.dll et al. as its stable interface.

asveikau 7y ago

Ntdll being the de facto syscall abi is the right answer. But ntdll is also technically subject to change, much of it not publicly documented by Microsoft. So typically a third party application calls into things like kernel32 or advapi (though Win7 and later split those into different DLLs internally) which wraps ntdll which does the actual ordinal calls. But generally speaking, yeah, the stable interfaces are the C ones, exposed through system DLLs. Linux is unique in developing libc and kernel separately and the division between the two as a public abi boundary.

mehrdadn 7y ago
I mean, you could say anything is subject to change. The "documented" <-> "stable" mapping isn't a bijection. CreateFileTransacted was documented for kernel32 but that didn't stop it from being claimed as deprecated. And CallNtPowerInformation was documented and yet some of it simply doesn't work anymore. Also, Microsoft didn't necessarily document everything, but they did provide .lib files (and often headers) even for "undocumented" APIs in these libraries. Breaking these APIs would break programs they previously provided SDKs for. The reality is, so many third-party applications depend directly on ntdll APIs (even Chrome) and some of them literally cannot work with kernel32 stuff (boot-time partition managers, for one), so as far as I'm concerned, it's about as much set in stone as any library could be.
- asveikau 7y ago
  Yeah. Direct dependence on ntdll occurs in the wild and it would break stuff to change it. It can also break MS's own code and potentially create massive engineering hurdles for them that are completely unnecessary. But they have always warned people in documentation that they don't consider it a stable interface to the same extent as Win32. In my experience a lot of the NT APIs are nicer, better thought out, more direct.
  - KyleJ61782 7y ago
    Using the NTDLL calls breaks stuff because NTDLL doesn't understand anything about the overlaid subsystems. NT was originally designed to support multiple subsystems in addition to Win32, like the Interix (POSIX compliant) subsystem and the OS/2 subsystem I believe. I think (I could be wrong on this) that this was possible by switching the system call table depending upon which process was currently active (doable by switching the system call vector).

masklinn 7y ago

> By not calling syscalls based on their numbers on Windows. That might sound crazy Not really. Only on Linux are direct syscalls an official ABI. On most unices (e.g. solaris, BSDs, OSX) you must dynamically link and go through the platform's libc.

earenndil 7y ago
I don't know about solaris, and I know macos had kernel changes which broke go, but freebsd has a stable kernel interface and unstable userspace. (So does linux, for that matter--ever try running a program linked to a newer version of glibc?)
- account42 7y ago
  > So does linux, for that matter--ever try running a program linked to a newer version of glibc? That doesn't mean the interface isn't stable or at least not less stable than the kernel interface or literally any iterface for something that is still being developed. It would be nice if glibc would provide an easy way to target the ABI of an older version (which is compatible with all newer versions) but you can do that the manual way by compiling and linking against that old version.