3 ms·

I would like to note that Linux is the only kernel which will allow you to do this! The Linux system call interface is stable and defined at the instruction set

I would like to note that Linux is the only kernel which will allow you to do this! The Linux system call interface is stable and defined at the instruction set level. Linking against some system library is absolutely required on every other system. I've written an article about this idea: https://www.matheusmoreira.com/articles/linux-system-calls https://www.matheusmoreira.com/articles/linux-system-calls You can get incredibly far with just this. I wrote a freestanding lisp interpreter with nothing but Linux system calls. It turned into a little framework for freestanding Linux programs. It's been incredibly fun. Freestanding C is a much better language. A lot of legacy nonsense is in the standard library. The Linux system call interface is really nice to work with. Calling write is not that hard. It's the printf style string building and formatting that I sometimes miss.

LegionMammal978 2y ago
"Absolutely required" is some strong language. It's perfectly possible to, e.g., perform direct syscalls on Windows, and you'll occasionally see malware that does it to avoid certain forms of detection. You just have to switch on the OS version, and update your binary if you want it to be compatible with a newer version.
- matheusmoreira 2y ago
  I agree that it was too strong a claim. It's not supported by the developers and if you bypass their system libraries your program will break when they change things up. Linux kernel is known to be able to run binaries compiled in the 90s. Breaking user space makes Linus yell at people until the breakage gets reverted. A platform that stable is worth building on top of. Updating executables is a lot of work, sometimes it's straight up impossible.
oguz-ismail 2y ago
> Linking against some system library is absolutely required on every other system. Not on FreeBSD, NetBSD, OpenBSD or Solaris. The article you linked says this but it's not true: > Sometimes it's not even possible to use system calls at all. OpenBSD has implemented system call origin verification, a security mechanism that only allows system calls originating from the system's libc. So not only is the kernel ABI unstable, normal programs are not even allowed to interface with the kernel at all. You can still make system calls from normal programs, you just need to list the addresses of system call instructions in an ELF section named openbsd.syscalls.
- matheusmoreira 2y ago
  > Not on FreeBSD, NetBSD, OpenBSD or Solaris. Can you cite any sources? I wasn't able to find any documentation that corroborates what you said when I wrote the article. The few texts I found actually suggested otherwise. Maybe things have changed since then? > You can still make system calls from normal programs, you just need to list the addresses of system call instructions in an ELF section named openbsd.syscalls. I see. So they have added a mechanism to list the sections allowed to perform system calls. That's news to me. Do they guarantee the system call numbers will remain stable though? That older system calls will remain available?
  - oguz-ismail 2y ago
    > Can you cite any sources? Personal experience. > Do they guarantee the system call numbers will remain stable though? No. Doesn't mean you can't make system calls from outside the libc though.
    - matheusmoreira 2y ago
      Every process must be able to make system calls. This is after all the mechanism by which the system libraries will interface with the kernel. The problem is the system's developers don't want us bypassing those libraries. We can do it but things can and probably will break in the future when they change things. It's not supported.
  - LegionMammal978 2y ago
    > Can you cite any sources? For one, the FreeBSD kernel specifically has a compatibility layer for Linux binaries to use their familiar syscalls [0]. For its ordinary syscalls, it also has a policy not to break binary compatibility without good reason [1]. Most other OSes just don't maintain quite the level of 'indefinite stability' that the Linux kernel does across different versions. And even Linux doesn't implement older versions of syscalls when the kernel is ported to new architectures, so eventually you have to rotate your implementation regardless, if you want people to run your code on new systems. > The few texts I found actually suggested otherwise. People often say "X is impossible" when the truth is "X is tricky and full of caveats, and I don't want to think about it, so stop asking". (Or if the devs themselves are saying it, it might be "I want to look like I'm 'tough on crime' toward users of undocumented behavior", as if that could stop Hyrum's law from running its course.) In this case, it's generally "If you do it on an OS other than Linux, you can run into big compatibility issues," not "It's impossible on OSes other than Linux." As for compatibility issues, you're running into that the moment you do undocumented fun stuff like omitting ELF sections or overlapping headers, which future Linux versions could start rejecting on the basis of "no one needs to do that legitimately". So I wouldn't start drawing the line on syscall number compatibility. [0] https://docs.freebsd.org/en/books/handbook/linuxemu/ https://docs.freebsd.org/en/books/handbook/linuxemu/ [1] https://wiki.freebsd.org/AddingSyscalls#Backward_compatibily https://wiki.freebsd.org/AddingSyscalls#Backward_compatibily
chipsrafferty 2y ago
As a web developer, 90% of what you just wrote is nonsense to me. How did you learn this stuff? Do you use it for useful projects or just for fun?
- matheusmoreira 2y ago
  Curiosity and free time. You learn stuff like this by reading tens of thousands of lines of text and code for every line of code that you write. I've always been all about the hidden fun stuff. The magical little programs that somehow configure audio cards. The ALSA mixer tool for example does it via special ioctls. I was reading its source code not too long ago. The manuals said those definitions were for the curious and that those ioctls were private, as though it was the library's author exclusive privilege to use those things. I seriously hate it when they say that. When they imply I'm some mere mortal who's better off using the libraries that were gifted to us by the gods of programming. Good or bad, quite a bit of hubris is involved. Takes a certain audacity to think I can make a better wheel than people who are probably much smarter than I am. Sometimes I start projects just to prove to myself that I'm not clinically insane for thinking a better way is possible. Sometimes it works, sometimes it doesn't. Someone once called an idea I had schizophrenic. I'll never forget that day. This Linux system call stuff started after I read an LWN article about glibc and Linux specific system call support, getrandom to be specific. Took glibc years to add support. I started a liblinux project because of that article. The idea was to get rid of libc and talk to Linux directly. In order to accomplish that, I was forced to learn a lot of compiler, linker and executable stuff. The musl libc source code taught me a lot. It seems like the C library is doing a huge amount of stuff but it turns out you don't actually need most of it. Linux just puts your binary in memory and jumps into some address specified in the ELF header. Normally this when the C library or dynamic linker takes over in order to prepare to call main(). Turns out I can just replace all that with some simple code that calls a function and then exits the process when it returns. It just works. I won't have init/fini section processing but I can live with that, that's harmful stuff that shouldn't even have been invented to begin with.
racingmars 2y ago
> I would like to note that Linux is the only kernel which will allow you to do this! I'm pretty sure that MVS syscalls (that is, the numbers you use with the SVC opcode) have remained backward-compatible at least as far back as MVS 3.8 in the 1970s and those binaries making those "raw" syscalls will still work on the latest z/OS releases. There are a _lot_ more operating systems than Linux, Windows, and the BSDs... making a statement that the Linux kernel is the only kernel to do something a certain way is a risky proposition :-)
- matheusmoreira 2y ago
  That's awesome. I didn't know about that system and never thought to look for it. Can you point me towards documentation where the vendor promises the interface will remain stable and backwards compatible? I'll remember it. The Linux promise: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/ABI/stable/syscalls https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

LegionMammal978 2y ago

"Absolutely required" is some strong language. It's perfectly possible to, e.g., perform direct syscalls on Windows, and you'll occasionally see malware that does it to avoid certain forms of detection. You just have to switch on the OS version, and update your binary if you want it to be compatible with a newer version.

matheusmoreira 2y ago
I agree that it was too strong a claim. It's not supported by the developers and if you bypass their system libraries your program will break when they change things up. Linux kernel is known to be able to run binaries compiled in the 90s. Breaking user space makes Linus yell at people until the breakage gets reverted. A platform that stable is worth building on top of. Updating executables is a lot of work, sometimes it's straight up impossible.

oguz-ismail 2y ago

> Linking against some system library is absolutely required on every other system. Not on FreeBSD, NetBSD, OpenBSD or Solaris. The article you linked says this but it's not true: > Sometimes it's not even possible to use system calls at all. OpenBSD has implemented system call origin verification, a security mechanism that only allows system calls originating from the system's libc. So not only is the kernel ABI unstable, normal programs are not even allowed to interface with the kernel at all. You can still make system calls from normal programs, you just need to list the addresses of system call instructions in an ELF section named openbsd.syscalls.

matheusmoreira 2y ago
> Not on FreeBSD, NetBSD, OpenBSD or Solaris. Can you cite any sources? I wasn't able to find any documentation that corroborates what you said when I wrote the article. The few texts I found actually suggested otherwise. Maybe things have changed since then? > You can still make system calls from normal programs, you just need to list the addresses of system call instructions in an ELF section named openbsd.syscalls. I see. So they have added a mechanism to list the sections allowed to perform system calls. That's news to me. Do they guarantee the system call numbers will remain stable though? That older system calls will remain available?
- oguz-ismail 2y ago
  > Can you cite any sources? Personal experience. > Do they guarantee the system call numbers will remain stable though? No. Doesn't mean you can't make system calls from outside the libc though.
  - matheusmoreira 2y ago
    Every process must be able to make system calls. This is after all the mechanism by which the system libraries will interface with the kernel. The problem is the system's developers don't want us bypassing those libraries. We can do it but things can and probably will break in the future when they change things. It's not supported.
- LegionMammal978 2y ago
  > Can you cite any sources? For one, the FreeBSD kernel specifically has a compatibility layer for Linux binaries to use their familiar syscalls [0]. For its ordinary syscalls, it also has a policy not to break binary compatibility without good reason [1]. Most other OSes just don't maintain quite the level of 'indefinite stability' that the Linux kernel does across different versions. And even Linux doesn't implement older versions of syscalls when the kernel is ported to new architectures, so eventually you have to rotate your implementation regardless, if you want people to run your code on new systems. > The few texts I found actually suggested otherwise. People often say "X is impossible" when the truth is "X is tricky and full of caveats, and I don't want to think about it, so stop asking". (Or if the devs themselves are saying it, it might be "I want to look like I'm 'tough on crime' toward users of undocumented behavior", as if that could stop Hyrum's law from running its course.) In this case, it's generally "If you do it on an OS other than Linux, you can run into big compatibility issues," not "It's impossible on OSes other than Linux." As for compatibility issues, you're running into that the moment you do undocumented fun stuff like omitting ELF sections or overlapping headers, which future Linux versions could start rejecting on the basis of "no one needs to do that legitimately". So I wouldn't start drawing the line on syscall number compatibility. [0] https://docs.freebsd.org/en/books/handbook/linuxemu/ https://docs.freebsd.org/en/books/handbook/linuxemu/ [1] https://wiki.freebsd.org/AddingSyscalls#Backward_compatibily https://wiki.freebsd.org/AddingSyscalls#Backward_compatibily

chipsrafferty 2y ago

As a web developer, 90% of what you just wrote is nonsense to me. How did you learn this stuff? Do you use it for useful projects or just for fun?

matheusmoreira 2y ago
Curiosity and free time. You learn stuff like this by reading tens of thousands of lines of text and code for every line of code that you write. I've always been all about the hidden fun stuff. The magical little programs that somehow configure audio cards. The ALSA mixer tool for example does it via special ioctls. I was reading its source code not too long ago. The manuals said those definitions were for the curious and that those ioctls were private, as though it was the library's author exclusive privilege to use those things. I seriously hate it when they say that. When they imply I'm some mere mortal who's better off using the libraries that were gifted to us by the gods of programming. Good or bad, quite a bit of hubris is involved. Takes a certain audacity to think I can make a better wheel than people who are probably much smarter than I am. Sometimes I start projects just to prove to myself that I'm not clinically insane for thinking a better way is possible. Sometimes it works, sometimes it doesn't. Someone once called an idea I had schizophrenic. I'll never forget that day. This Linux system call stuff started after I read an LWN article about glibc and Linux specific system call support, getrandom to be specific. Took glibc years to add support. I started a liblinux project because of that article. The idea was to get rid of libc and talk to Linux directly. In order to accomplish that, I was forced to learn a lot of compiler, linker and executable stuff. The musl libc source code taught me a lot. It seems like the C library is doing a huge amount of stuff but it turns out you don't actually need most of it. Linux just puts your binary in memory and jumps into some address specified in the ELF header. Normally this when the C library or dynamic linker takes over in order to prepare to call main(). Turns out I can just replace all that with some simple code that calls a function and then exits the process when it returns. It just works. I won't have init/fini section processing but I can live with that, that's harmful stuff that shouldn't even have been invented to begin with.

racingmars 2y ago

> I would like to note that Linux is the only kernel which will allow you to do this! I'm pretty sure that MVS syscalls (that is, the numbers you use with the SVC opcode) have remained backward-compatible at least as far back as MVS 3.8 in the 1970s and those binaries making those "raw" syscalls will still work on the latest z/OS releases. There are a _lot_ more operating systems than Linux, Windows, and the BSDs... making a statement that the Linux kernel is the only kernel to do something a certain way is a risky proposition :-)

matheusmoreira 2y ago
That's awesome. I didn't know about that system and never thought to look for it. Can you point me towards documentation where the vendor promises the interface will remain stable and backwards compatible? I'll remember it. The Linux promise: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/ABI/stable/syscalls https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...