10 ms·

Linux Syscall Support

207 points by btdmaster 2y ago

sph 2y ago
See also Linux's nolibc headers, which allows one to write C software that completely bypass libc, but instead directly operate through syscalls. https://github.com/torvalds/linux/tree/master/tools/include/nolibc https://github.com/torvalds/linux/tree/master/tools/include/... A sample use-case? I was developing an Erlang-like actor platform that should operate under Linux as well as a bare-metal microkernel, and all I needed is a light layer over syscalls instead of pulling the entire glibc. Also it provides a simple implementation for standard C functions (memcpy, printf) so I don't have to write them myself.
- guerrilla 2y ago
  > What would be a use-case? Maybe bootstapping a new language with no dependencies.
  - sph 2y ago
    Yes. Go for example doesn't use glibc and instead interfaces with syscalls directly. https://pkg.go.dev/syscall https://pkg.go.dev/syscall
    - cyberpunk 2y ago
      I’m aware of this but I really don’t the benefits of this approach; It causes issues in eg openbsd where you can only call syscalls from libc, and it seems like they’re trying to outsmart the os developers and I just don’t see an advantage. Is it faster? More stable?
      - oguz-ismail 2y ago
        > It causes issues in eg openbsd where you can only call syscalls from libc OpenBSD allows making syscalls from static binaries as well. If Go binaries are static, it shouldn't cause any problems.
        kbolino 2y ago
        > OpenBSD allows making syscalls from static binaries as well. Do you have a source for this? My Google searches and personal recollections say that OpenBSD does not have a stable syscall ABI in the way that Linux does and the proper/supported way to make syscalls on OpenBSD is through dynamically linked libc; statically linking libc, or invoking the syscall mechanism it uses directly, results in binaries that can be broken on future OpenBSD versions.
        deleted 2y ago
        [deleted]
        cesarb 2y ago
        > > OpenBSD allows making syscalls from static binaries as well. > Do you have a source for this? One article from 2019 about this can be found at https://lwn.net/Articles/806776/ https://lwn.net/Articles/806776/ (later updates https://lwn.net/Articles/949078/ https://lwn.net/Articles/949078/ and https://lwn.net/Articles/959562/ https://lwn.net/Articles/959562/). Yes, it does not have a stable system call ABI, but as long as your program was statically compiled with the libc from the same OpenBSD release, AFAIK it should work.
        oguz-ismail 2y ago
        Yeah. Do you have any information as to how/when the OpenBSD system call ABI has changed recently? I wouldn't expect that to happen very often.
        WhyNotHugo 2y ago
        From 2019: https://lwn.net/Articles/806776/ https://lwn.net/Articles/806776/
        kbolino 2y ago
        In particular, from Theo de Raadt himself: > we here at OpenBSD are the kings of ABI-instability > Program to the API rather than the ABI. When we see benefits, we change the ABI more often than the API. > I have altered the ABI. Pray I do not alter it further. The term ABI here though is a little imprecise. I believe it just refers to the syscall ABI. So, it should be possible to make an "almost static" binary by statically linking everything except libc, and that binary should continue to work in future versions of OpenBSD.
      - titzer 2y ago
        There are several advantages to using kernel syscalls directly: 1. No overhead from libc; minimizes syscall cost 2. No dependency on libc and C language ABI/toolchains 3. Reduced attack surface. libc can and does have bugs and potentially ROP or Spectre gadgets. 4. Bootstrapping other languages, e.g. Virgil
        kllrnohj 2y ago
        > 1. No overhead from libc; minimizes syscall cost The few nanoseconds of a straight function call are absolutely irrelevant vs the 10s of microseconds of a syscall cost and you lose out on any of the optimizations a libc has that you might not or didn't think about (like memoization of getpid() ) and you need to take on keeping up with syscall evolution / best practices which a libc generally has a good handle on. > No dependency on libc and C language ABI/toolchains This obviously doesn't apply to a C syscall header, though, such as the case in OP :)
        gpderetta 2y ago
        A syscall can be way less than 10us. Especially if it is not doing I/O.
        kaba0 2y ago
        But a kernel mode switch is definitely more expensive than a trivial (likely cached) jump instruction.
        matheusmoreira 2y ago
        > you lose out on any of the optimizations a libc has that you might not or didn't think about (like memoization of getpid() ) Not much of a big deal. These "optimizations" caused enough bugs that they actually got reverted. https://www.man7.org/linux/man-pages/man2/getpid.2.html https://www.man7.org/linux/man-pages/man2/getpid.2.html Because of the aforementioned problems, since glibc 2.25, the PID cache is removed: calls to getpid() always invoke the actual system call, rather than returning a cached value. Get rid of libc and you gain the ability to have zero global state in exchange. Freestanding C actually makes sense and is a very fun language to program in. No legacy nonsense to worry about. Even errno is gone.
      - masklinn 2y ago
        > I just don’t see an advantage. You don’t have to deal with C ABI requirements with respect to stack, or registers management. You also don’t need to do dynamic linking. On the other hand all of that comes back to bone you if you’re trying to benefit from vDSO without going through a libc.
        titzer 2y ago
        > You also don’t need to do dynamic linking. This is a big one. Linking against libc on many platforms also means making your binaries relocatable. It's a lot of unnecessary, incidental complexity.
        rcxdude 2y ago
        It also means giving up ASLR, though.
        titzer 2y ago
        You can still randomize heap allocations (but not with as much entropy), as usually the heap segment is quite large. But you don't get randomization of, e.g. the code. ASLR is a weak defense. It's akin to randomizing which of the kitchen drawers you'll put your jewelry in. Not the same level of security as say, a locked safe. Attacks are increasingly sophisticated, composed of multiple exploits in a chain, one of which is some form of ASLR bypass. It's usually one of the easiest links in the chain.
        LegionMammal978 2y ago
        > On the other hand all of that comes back to bone you if you’re trying to benefit from vDSO without going through a libc. At least the vDSO functions really don't need much in the way of stack space: generally there's nothing much there but clock_gettime() and gettimeofday(), which just read some values from the vvar area. The bigger pain, of course, is actually looking up the symbols in the vDSO, which takes at least a minimal ELF parser.
        2y ago
      - SAI_Peregrinus 2y ago
        GNU aren't the OS developers of the Linux kernel. Think of the Go standard library on Linux as another libc-level library. On the BSDs there is a single libc that's part of the OS, on Linux there are several options for libc.
    - josefx 2y ago
      Didn't they go back to Glibc in 2017 after a syscall silently corrupted several of their tightly packed tiny Go stacks? The page you link to seems to refer to a proposal from 2014 as "new".
      - melodyogonna 2y ago
        That is the documentation for the Go syscall package. If you scroll down to the bottom of the page you'll see links to the source files.
      - jsheard 2y ago
        IIRC that was specifically on macOS and other BSDs which don't have a stable syscall interface. They still use raw syscalls on Linux, which guarantees syscall stability on pain of Linus Torvalds yelling at you if you break it.
        titzer 2y ago
        I'm 100% with Linus on this one.
        favorited 2y ago
        Linus ships a kernel –– where would his stable interface live if not the syscall ABI? The *BSD and macOS folks ship operating systems, where they have the option of defining their ABI at a higher level of abstraction.
        matheusmoreira 2y ago
        Linux could have made their own libc and mandated use of it. But they didn't. They chose a language agnostic binary interface that's documented at the instruction set level. As a result of that brilliant design choice, every single language can make Linux system calls natively. It should be simple for JIT compilers to generate Linux system call code. No need to pull in some huge C library just for this. AOT compilers could have a linux_system_call builtin that just generates the required instructions. I actually posted this proposal to the GCC mailing list.
    - asveikau 2y ago
      Except there are some platforms where you need to go through libc and the direct syscall interface is considered private, or subject to change. OpenBSD is like this, and I believe Mac is too.
      - matheusmoreira 2y ago
        Linux is the only one that is not like this. I wrote an article about the subject: https://www.matheusmoreira.com/articles/linux-system-calls https://www.matheusmoreira.com/articles/linux-system-calls
        vetinari 2y ago
        While Linux does have stable syscall interface, unlike other OS-es, you still want to go through glibc. At least for NSS, otherwise your app could be broken. Golang has CGO_ENABLED=1 as the default for this reason.
    - boomskats 2y ago
      Worth mentioning that the golang.org/x/sys/unix package has better support for syscalls than the og syscall package nowadays, especially for some of the newer ones like cachestat[0] which was added to the kernel in 6.5. AFAIK the original syscall package was 'frozen' a while back to preserve backward compatibility, and at one point there was even a bit of drama[1] around it being marked as deprecated instead of frozen. [0]: https://github.com/golang/go/issues/61917 https://github.com/golang/go/issues/61917 [1]: https://github.com/golang/go/issues/60797 https://github.com/golang/go/issues/60797
      - lpapez 2y ago
        Undeprecating something is truly a rare sight. So far I only knew about PHP undeprecating "is_a" function, so I guess this puts Go in good company ^^
  - jcalabro 2y ago
    Indeed, Zig does this for instance (at least for x86_64 linux [0]) as a way to avoid having to link libc at all [0] https://github.com/ziglang/zig/blob/ee9f00d673f2bccddc2751c328758a2820d2bb70/lib/std/os/linux/x86_64.zig https://github.com/ziglang/zig/blob/ee9f00d673f2bccddc2751c3...
  - matheusmoreira 2y ago
    I've been working on that language for a while now! https://github.com/lone-lang/lone https://github.com/lone-lang/lone It's a lisp interpreter with a built in system-call primitive. The plan is to implement everything else from inside the language. Completely freestanding, no libc needed. In the future I expect to be able to boot Linux directly into this thing. Only major feature still needed for kernel support is a binary structure parser for the C structures. I already implemented and tested the primitives for it. I even added support for unaligned memory accesses. Iteration is the only major language feature that's still missing. I'm working on implementing continuations in the interpreter so that I can have elegant Ruby style iteration. This is taking longer than expected. This interpreter can make the Linux kernel load lisp modules before its code even runs. I invented a self-contained ELF loading system that allows embedding arbitrary data into a loadable ELF segment that the kernel automatically maps into memory. Then it's just a matter of reaching it via the auxiliary vector, The interpreter uses this to automatically run code, allowing it to become a freestanding lisp executable. I wrote an article about how it works here: https://www.matheusmoreira.com/articles/self-contained-lone-lisp-applications https://www.matheusmoreira.com/articles/self-contained-lone-...
    - samatman 2y ago
      > I'm working on implementing continuations in the interpreter You might find this paper (and everything else on the site) interesting and relevant. https://okmij.org/ftp/continuations/ZFS/context-OS.pdf https://okmij.org/ftp/continuations/ZFS/context-OS.pdf
- blueflow 2y ago
  > See also Linux's nolibc headers Kind of an understatement. The existence of an official interface obsoletes 3rd party projects like the one posted.
  - almostgotcaught 2y ago
    Ya totally - those wacky people at chrome must've just never heard of those headers /s What you don't understand, because you don't work on Chrome, or Chrome sized projects, is that generic, lowest common denominator implementations cannot be optimal for all use-cases and at scale (Chrome-sized project) those inefficiencies matter. That's why this exists, that's why folly exists, that's why abseil exists, that's why no not everyone can just use boost, etc etc etc
    - deleted 2y ago
      [deleted]
  - bla3 2y ago
    Might be a license thing? The Linux headers are probably GPL like the rest of Linux.
    - unmole 2y ago
      The Linux kernel licence explicitly says programs using the syscall interface are not considered derivative works and that GPL does not apply to them: https://github.com/torvalds/linux/blob/master/LICENSES/exceptions/Linux-syscall-note https://github.com/torvalds/linux/blob/master/LICENSES/excep...
    - sph 2y ago
      nolibc is NOT under GPL. See first line of each file. /* SPDX-License-Identifier: LGPL-2.1 OR MIT */ It's technically not part Linux's headers either. It's published under the tools subdirectory, so it's something that ships along with the kernel, but not used by the kernel itself. Basically it's there as some people might find it useful, but could've as well been a separate repo.
    - deleted 2y ago
- sylware 2y ago
  Well... last time I had a look at the assembly code of syscall entry on x86_64, I was scared away... this piece of "assembly" does require some very niche C compiler options to be compatible (stack alignment and more if I recall properly). Linux "C" code hard dependency on gcc/clang/("ultra complex compilers") is getting worse by the day. It should (very easy to say, I know) have stayed very simple and plain C99+ with smart macro definitions to be replaced with pure assembly or the missing bits for modern hardware programming (atomics/memory barriers, explicit unaligned access, etc), but those abominations like _generic (or static assert,__thread,etc) are toxic additions to the C standard (there are too many toxic additions and not enough removal/simplification/hardening in ISO C, yes, we will have to define a "C profile" which breaks backward compatibility with hardening and simplifications). I don't say all extensions are bad, but I think they need more "smart and pertinent pick and choose" (and I know this is a tough call), just because they "cost". For instance, for a kernel, we know it must have fine grained control of ELF object sections... or we would get much more source files (one per pertinent section) or "many more source configuration macros" (....but there I start to wonder if it was not actually the "right" way instead of requiring a C compiler to support such extension, it moves everything to the linker script... which is "required" anyway for a kernel). Linus T. is not omnipotent and can do only that much and a lot of "official" linux devs are putting really nasty SDK dependency requirements in everyday/everywhere kernels. That said, on my side, many of my user apps are now directly using linux syscalls... but are written in RISC-V assembly interpreted on x86_64 (I have a super lean interpreter/syscall translater written in x86_64 assembly and a super lean executable format wrapped in ELF executable format), or very plain and simple C99+ (legacy or because I want some apps to be a bit more 'platform crossy'... for now).
  - gpderetta 2y ago
    It is hard to take seriously someone that claims that thread locals are a toxic addition to the standard. (incidentally __thread is a GCC extension that predates the standard by almost a decade).
    - sylware 2y ago
      This little keyword makes the compiler generates calls to the system threading libs (libpthread), or even worse, declare a TLS slot in the ELF format, which is a limited resource (you cannot reasonably resize the TLS segment on the fly). Yes, that amount of complexity is obviously toxic... and saying otherwise is what will make you hard to be taken seriously, come on dude...
- jlokier 2y ago
  Another use-case is when you are writing threaded code that uses the clone() syscall instead of pthreads, usually for something with high performance, unusual clone flags, or a very small stack. Most libc functions, including the syscall wrappers and all pthreads functions, aren't safe to call in threads created by raw clone(). Anything that reads or writes errno, for example, is not safe. I've had to do this a couple of times. One a long time ago was an audio mixing real-time thread for a video game, which had to keep the audio device fed with low-latency frames for sound effects. In those days, pthreads wasn't good enough. For talking to the audio device, we had to use the Linux syscall wrapper macros, which have been replaced by nolibc now. More recently, a thread pool for high-performance storage I/O, which ran slightly faster than io_uring, and ran well on older kernels and ones with io_uring disabled for security.
- n_plus_1_acc 2y ago
  Do I understand correctly that nolibc is just another implementation of the C standard library in terms of Linux syscalls? Comparably to, say, musl libc?
  - sph 2y ago
    glibc is a space shuttle, musl is a hatchback, nolibc is a skateboard They all do the same thing (take you from A to B), but offer different levels of comfort, efficiency and utility :)
    - jonhohle 2y ago
      Who can take their space shuttle to work these days, what with the price of rocket fuel‽
      - sph 2y ago
        It's a bit unwieldy, but the good thing is that it comes for free with your copy of GNU/Linux!
      - nilamo 2y ago
        And parking is always a nightmare for my shuttles
      - malkia 2y ago
        Apparently almost every linux app
      - deleted 2y ago
        [deleted]
    - wbl 2y ago
      And passenger safety?
      - fallingsquirrel 2y ago
        In a head-on collision, the space shuttle passengers will fare better than the hatchback. Even so, it wouldn't be my first choice for most destinations.
- yencabulator 2y ago
  nolibc seems kinda neglected, or like a minimal subset of what's actually useful. There's no pread/pwrite etc, only read/write, forcing you to use lseek and ruining concurrent use.
  - pantalaimon 2y ago
    Wasn’t that originally just for integration tests where you wanted to boot a minimal image that just runs your kernel CI test?
  - t-8ch 2y ago
    It doesn't strive to be a complete implementation. If users send patches support for them, new syscalls can be added. Or downstream users can keep those definitions in their own program. More complex features like threads are out of scope. (Disclaimer: I'm one of the maintainers)
    - yencabulator 2y ago
      Sure, I'm more saying, because nolibc is so minimal, that still leaves room for things like this Chromium library.
- deleted 2y ago
  [deleted]
- arjvik 2y ago
  Where was this when I needed it! As one of my class projects, I built a Linux compatibility layer for the toy OS we had built, by adding a proper ELF loader and emulating syscalls. I really struggled to get glibc or even musl working, and so I ended up hand-coding some `-nostdlib` programs instead of being able to use coreutils. If nolibc really works as a minimal libc, would have been incredibly cool to be able to run coreutils on my OS!
jagrsw 2y ago
Just a friendly reminder that syscall() is a vararg function. Meaning, you can't just go throwing arguments at it (so maybe it's better to use this wrapper to avoid problems instead). For example, on a 64-bit arch, this code would be sus. syscall(__NR_syscall_taking_6_args, 1, 2, 3, 4, 5, 6); Quiz: why PS: it's a common mistake, so I thought I'd save you a trip down the debugging rabbit hole.
- remram 2y ago
  A quiz is the opposite of saving someone effort.
  - eddd-ddde 2y ago
    Exactly, I am now morally bound to figure out the answer instead of going to work.
    - oguz-ismail 2y ago
      The last argument would be on the stack instead of in a register which is where the kernel expects to find the arguments. But a proper syscall implementation would handle this just fine (e.g. <https://github.com/bminor/glibc/blob/ba60be873554ecd141b55ea82647714933c0dbd4/sysdeps/unix/sysv/linux/x86_64/syscall.S https://github.com/bminor/glibc/blob/ba60be873554ecd141b55ea...>), so I don't think there's anything sus about it.
      - im3w1l 2y ago
        > movq 8(%rsp),%r9 This is a huge edgecase but is 8(%rsp) guaranteed to be readable memory
        achierius 2y ago
        Yes, see https://en.wikipedia.org/wiki/Red_zone_(computing) https://en.wikipedia.org/wiki/Red_zone_(computing)
IAmLiterallyAB 2y ago
I've been using my own version of this. Maybe I'll switch over, this looks more complete.
kentonv 2y ago
Disappointing that errors are still signaled by assigning to `errno` (you can apparently override this to some other global, but it has to be a global or global-like lvalue). The kernel actually signals errors by returning a negative error code (on most arches), which seems like a better calling convention. Storing errors in something like `errno` opens a whole can of worms around thread safety and signal safety, while seemingly providing very little benefit beyond following tradition.
- sidkshatriya 2y ago
  code that uses errno is also a bit harder to understand. I like the way Rust does it -- if a function can fail, it returns a Result.
  - dailykoder 2y ago
    While that might be true and the industry has evolved and learned about "better" ways, the old systems still exist. I don't see any reason to complain about it. Yes, we can do better. Yes, we probably should do better. But in some cases you really have to think through every edge case and in the end someone has to do it. So just be grateful for what we have.
    - IshKebab 2y ago
      I don't think this is an "old system" though.
    - alexey-salmin 2y ago
      For old systems -- yes, of course. But designing a new, incompatible API around errno is just backwards.
- AndyKelley 2y ago
  Disappointing is an understatement. Can't believe these people are making a browser. I'm sure they have some Google-flavored excuse for why to repeat this ridiculous threadlocal errno API.
wg0 2y ago
So web apps can make Linux sys calls? Or its about Chrome OS?
- thinkharderdev 2y ago
  The chrome browser itself I would think
Aransentin 2y ago
> We try to hide some of the differences between arches when reasonably feasible. e.g. Newer architectures no longer provide an open syscall, but do provide openat. We will still expose a sys_open helper by default that calls into openat instead. Sometimes you actually want to make sure that the exact syscall is called; e.g. you're writing a little program protected by strict seccomp rules. If the layer can magically call some other syscall under the hood this won't work anymore.
- dundarious 2y ago
  musl does this too. glibc may also, I haven't checked in a long time. I bet rust, etc., does too. You always need to check.
  - king_geedorah 2y ago
    Glibc definitely does this transparent mapping as well. Calling int fd = open(<path>, O_RDONLY) yields openat(AT_FDCWD, <path>, O_RDONLY) when running through strace.
    - jchw 2y ago
      This really surprised me when I was digging into Linux tracing technology and noticed no `open` syscalls on my running system. It was all `openat`. I don't know when this transition happened, but I totally missed it.
mcnichol 2y ago
0-Day incoming
sedatk 2y ago
Can't wait for Zig team to adopt this over libc, citing concerns about "libc not existing on certain configurations"[1] [1] https://github.com/ziglang/zig/issues/1840 https://github.com/ziglang/zig/issues/1840
- TUSF 2y ago
  Zig on Linux already directly interfaces with syscalls,[0] unless your library or application directly links libc. [0]: https://ziglang.org/documentation/master/std/#std.os.linux https://ziglang.org/documentation/master/std/#std.os.linux
  - sedatk 2y ago
    Oh, there you go :)
- AndyKelley 2y ago
  Welcome to 2016. https://github.com/ziglang/zig/blob/5f0bfcac24036e1fff0b2beda643a60dad465213/std/syscall.zig https://github.com/ziglang/zig/blob/5f0bfcac24036e1fff0b2bed...
  - sedatk 2y ago
    :'(
iTokio 2y ago
Using go is a nice way to do that by default as it also directly uses syscalls (see the sys package)
zamalek 2y ago
You have to be pretty careful when using syscalls directly, at least in the presence of some libc. For example, from what I have gathered from random tidbits here and there, it's not a good idea to use any of the fork-like calls (fork, clone, etc.) if you have any threads that were created using glibc.

207 points by btdmaster 2y ago

sph 2y ago

See also Linux's nolibc headers, which allows one to write C software that completely bypass libc, but instead directly operate through syscalls. https://github.com/torvalds/linux/tree/master/tools/include/nolibc https://github.com/torvalds/linux/tree/master/tools/include/... A sample use-case? I was developing an Erlang-like actor platform that should operate under Linux as well as a bare-metal microkernel, and all I needed is a light layer over syscalls instead of pulling the entire glibc. Also it provides a simple implementation for standard C functions (memcpy, printf) so I don't have to write them myself.

guerrilla 2y ago
> What would be a use-case? Maybe bootstapping a new language with no dependencies.
- sph 2y ago
  Yes. Go for example doesn't use glibc and instead interfaces with syscalls directly. https://pkg.go.dev/syscall https://pkg.go.dev/syscall
  - cyberpunk 2y ago
    I’m aware of this but I really don’t the benefits of this approach; It causes issues in eg openbsd where you can only call syscalls from libc, and it seems like they’re trying to outsmart the os developers and I just don’t see an advantage. Is it faster? More stable?
    - oguz-ismail 2y ago
      > It causes issues in eg openbsd where you can only call syscalls from libc OpenBSD allows making syscalls from static binaries as well. If Go binaries are static, it shouldn't cause any problems.
      - kbolino 2y ago
        > OpenBSD allows making syscalls from static binaries as well. Do you have a source for this? My Google searches and personal recollections say that OpenBSD does not have a stable syscall ABI in the way that Linux does and the proper/supported way to make syscalls on OpenBSD is through dynamically linked libc; statically linking libc, or invoking the syscall mechanism it uses directly, results in binaries that can be broken on future OpenBSD versions.
        deleted 2y ago
        [deleted]
        cesarb 2y ago
        > > OpenBSD allows making syscalls from static binaries as well. > Do you have a source for this? One article from 2019 about this can be found at https://lwn.net/Articles/806776/ https://lwn.net/Articles/806776/ (later updates https://lwn.net/Articles/949078/ https://lwn.net/Articles/949078/ and https://lwn.net/Articles/959562/ https://lwn.net/Articles/959562/). Yes, it does not have a stable system call ABI, but as long as your program was statically compiled with the libc from the same OpenBSD release, AFAIK it should work.
        oguz-ismail 2y ago
        Yeah. Do you have any information as to how/when the OpenBSD system call ABI has changed recently? I wouldn't expect that to happen very often.
        WhyNotHugo 2y ago
        From 2019: https://lwn.net/Articles/806776/ https://lwn.net/Articles/806776/
        kbolino 2y ago
        In particular, from Theo de Raadt himself: > we here at OpenBSD are the kings of ABI-instability > Program to the API rather than the ABI. When we see benefits, we change the ABI more often than the API. > I have altered the ABI. Pray I do not alter it further. The term ABI here though is a little imprecise. I believe it just refers to the syscall ABI. So, it should be possible to make an "almost static" binary by statically linking everything except libc, and that binary should continue to work in future versions of OpenBSD.
    - titzer 2y ago
      There are several advantages to using kernel syscalls directly: 1. No overhead from libc; minimizes syscall cost 2. No dependency on libc and C language ABI/toolchains 3. Reduced attack surface. libc can and does have bugs and potentially ROP or Spectre gadgets. 4. Bootstrapping other languages, e.g. Virgil
      - kllrnohj 2y ago
        > 1. No overhead from libc; minimizes syscall cost The few nanoseconds of a straight function call are absolutely irrelevant vs the 10s of microseconds of a syscall cost and you lose out on any of the optimizations a libc has that you might not or didn't think about (like memoization of getpid() ) and you need to take on keeping up with syscall evolution / best practices which a libc generally has a good handle on. > No dependency on libc and C language ABI/toolchains This obviously doesn't apply to a C syscall header, though, such as the case in OP :)
        gpderetta 2y ago
        A syscall can be way less than 10us. Especially if it is not doing I/O.
        kaba0 2y ago
        But a kernel mode switch is definitely more expensive than a trivial (likely cached) jump instruction.
        matheusmoreira 2y ago
        > you lose out on any of the optimizations a libc has that you might not or didn't think about (like memoization of getpid() ) Not much of a big deal. These "optimizations" caused enough bugs that they actually got reverted. https://www.man7.org/linux/man-pages/man2/getpid.2.html https://www.man7.org/linux/man-pages/man2/getpid.2.html Because of the aforementioned problems, since glibc 2.25, the PID cache is removed: calls to getpid() always invoke the actual system call, rather than returning a cached value. Get rid of libc and you gain the ability to have zero global state in exchange. Freestanding C actually makes sense and is a very fun language to program in. No legacy nonsense to worry about. Even errno is gone.
    - masklinn 2y ago
      > I just don’t see an advantage. You don’t have to deal with C ABI requirements with respect to stack, or registers management. You also don’t need to do dynamic linking. On the other hand all of that comes back to bone you if you’re trying to benefit from vDSO without going through a libc.
      - titzer 2y ago
        > You also don’t need to do dynamic linking. This is a big one. Linking against libc on many platforms also means making your binaries relocatable. It's a lot of unnecessary, incidental complexity.
        rcxdude 2y ago
        It also means giving up ASLR, though.
        titzer 2y ago
        You can still randomize heap allocations (but not with as much entropy), as usually the heap segment is quite large. But you don't get randomization of, e.g. the code. ASLR is a weak defense. It's akin to randomizing which of the kitchen drawers you'll put your jewelry in. Not the same level of security as say, a locked safe. Attacks are increasingly sophisticated, composed of multiple exploits in a chain, one of which is some form of ASLR bypass. It's usually one of the easiest links in the chain.
      - LegionMammal978 2y ago
        > On the other hand all of that comes back to bone you if you’re trying to benefit from vDSO without going through a libc. At least the vDSO functions really don't need much in the way of stack space: generally there's nothing much there but clock_gettime() and gettimeofday(), which just read some values from the vvar area. The bigger pain, of course, is actually looking up the symbols in the vDSO, which takes at least a minimal ELF parser.
        2y ago
    - SAI_Peregrinus 2y ago
      GNU aren't the OS developers of the Linux kernel. Think of the Go standard library on Linux as another libc-level library. On the BSDs there is a single libc that's part of the OS, on Linux there are several options for libc.
  - josefx 2y ago
    Didn't they go back to Glibc in 2017 after a syscall silently corrupted several of their tightly packed tiny Go stacks? The page you link to seems to refer to a proposal from 2014 as "new".
    - melodyogonna 2y ago
      That is the documentation for the Go syscall package. If you scroll down to the bottom of the page you'll see links to the source files.
    - jsheard 2y ago
      IIRC that was specifically on macOS and other BSDs which don't have a stable syscall interface. They still use raw syscalls on Linux, which guarantees syscall stability on pain of Linus Torvalds yelling at you if you break it.
      - titzer 2y ago
        I'm 100% with Linus on this one.
        favorited 2y ago
        Linus ships a kernel –– where would his stable interface live if not the syscall ABI? The *BSD and macOS folks ship operating systems, where they have the option of defining their ABI at a higher level of abstraction.
        matheusmoreira 2y ago
        Linux could have made their own libc and mandated use of it. But they didn't. They chose a language agnostic binary interface that's documented at the instruction set level. As a result of that brilliant design choice, every single language can make Linux system calls natively. It should be simple for JIT compilers to generate Linux system call code. No need to pull in some huge C library just for this. AOT compilers could have a linux_system_call builtin that just generates the required instructions. I actually posted this proposal to the GCC mailing list.
  - asveikau 2y ago
    Except there are some platforms where you need to go through libc and the direct syscall interface is considered private, or subject to change. OpenBSD is like this, and I believe Mac is too.
    - matheusmoreira 2y ago
      Linux is the only one that is not like this. I wrote an article about the subject: https://www.matheusmoreira.com/articles/linux-system-calls https://www.matheusmoreira.com/articles/linux-system-calls
      - vetinari 2y ago
        While Linux does have stable syscall interface, unlike other OS-es, you still want to go through glibc. At least for NSS, otherwise your app could be broken. Golang has CGO_ENABLED=1 as the default for this reason.
  - boomskats 2y ago
    Worth mentioning that the golang.org/x/sys/unix package has better support for syscalls than the og syscall package nowadays, especially for some of the newer ones like cachestat[0] which was added to the kernel in 6.5. AFAIK the original syscall package was 'frozen' a while back to preserve backward compatibility, and at one point there was even a bit of drama[1] around it being marked as deprecated instead of frozen. [0]: https://github.com/golang/go/issues/61917 https://github.com/golang/go/issues/61917 [1]: https://github.com/golang/go/issues/60797 https://github.com/golang/go/issues/60797
    - lpapez 2y ago
      Undeprecating something is truly a rare sight. So far I only knew about PHP undeprecating "is_a" function, so I guess this puts Go in good company ^^
- jcalabro 2y ago
  Indeed, Zig does this for instance (at least for x86_64 linux [0]) as a way to avoid having to link libc at all [0] https://github.com/ziglang/zig/blob/ee9f00d673f2bccddc2751c328758a2820d2bb70/lib/std/os/linux/x86_64.zig https://github.com/ziglang/zig/blob/ee9f00d673f2bccddc2751c3...
- matheusmoreira 2y ago
  I've been working on that language for a while now! https://github.com/lone-lang/lone https://github.com/lone-lang/lone It's a lisp interpreter with a built in system-call primitive. The plan is to implement everything else from inside the language. Completely freestanding, no libc needed. In the future I expect to be able to boot Linux directly into this thing. Only major feature still needed for kernel support is a binary structure parser for the C structures. I already implemented and tested the primitives for it. I even added support for unaligned memory accesses. Iteration is the only major language feature that's still missing. I'm working on implementing continuations in the interpreter so that I can have elegant Ruby style iteration. This is taking longer than expected. This interpreter can make the Linux kernel load lisp modules before its code even runs. I invented a self-contained ELF loading system that allows embedding arbitrary data into a loadable ELF segment that the kernel automatically maps into memory. Then it's just a matter of reaching it via the auxiliary vector, The interpreter uses this to automatically run code, allowing it to become a freestanding lisp executable. I wrote an article about how it works here: https://www.matheusmoreira.com/articles/self-contained-lone-lisp-applications https://www.matheusmoreira.com/articles/self-contained-lone-...
  - samatman 2y ago
    > I'm working on implementing continuations in the interpreter You might find this paper (and everything else on the site) interesting and relevant. https://okmij.org/ftp/continuations/ZFS/context-OS.pdf https://okmij.org/ftp/continuations/ZFS/context-OS.pdf
blueflow 2y ago
> See also Linux's nolibc headers Kind of an understatement. The existence of an official interface obsoletes 3rd party projects like the one posted.
- almostgotcaught 2y ago
  Ya totally - those wacky people at chrome must've just never heard of those headers /s What you don't understand, because you don't work on Chrome, or Chrome sized projects, is that generic, lowest common denominator implementations cannot be optimal for all use-cases and at scale (Chrome-sized project) those inefficiencies matter. That's why this exists, that's why folly exists, that's why abseil exists, that's why no not everyone can just use boost, etc etc etc
  - deleted 2y ago
    [deleted]
- bla3 2y ago
  Might be a license thing? The Linux headers are probably GPL like the rest of Linux.
  - unmole 2y ago
    The Linux kernel licence explicitly says programs using the syscall interface are not considered derivative works and that GPL does not apply to them: https://github.com/torvalds/linux/blob/master/LICENSES/exceptions/Linux-syscall-note https://github.com/torvalds/linux/blob/master/LICENSES/excep...
  - sph 2y ago
    nolibc is NOT under GPL. See first line of each file. /* SPDX-License-Identifier: LGPL-2.1 OR MIT */ It's technically not part Linux's headers either. It's published under the tools subdirectory, so it's something that ships along with the kernel, but not used by the kernel itself. Basically it's there as some people might find it useful, but could've as well been a separate repo.
  - deleted 2y ago
sylware 2y ago
Well... last time I had a look at the assembly code of syscall entry on x86_64, I was scared away... this piece of "assembly" does require some very niche C compiler options to be compatible (stack alignment and more if I recall properly). Linux "C" code hard dependency on gcc/clang/("ultra complex compilers") is getting worse by the day. It should (very easy to say, I know) have stayed very simple and plain C99+ with smart macro definitions to be replaced with pure assembly or the missing bits for modern hardware programming (atomics/memory barriers, explicit unaligned access, etc), but those abominations like _generic (or static assert,__thread,etc) are toxic additions to the C standard (there are too many toxic additions and not enough removal/simplification/hardening in ISO C, yes, we will have to define a "C profile" which breaks backward compatibility with hardening and simplifications). I don't say all extensions are bad, but I think they need more "smart and pertinent pick and choose" (and I know this is a tough call), just because they "cost". For instance, for a kernel, we know it must have fine grained control of ELF object sections... or we would get much more source files (one per pertinent section) or "many more source configuration macros" (....but there I start to wonder if it was not actually the "right" way instead of requiring a C compiler to support such extension, it moves everything to the linker script... which is "required" anyway for a kernel). Linus T. is not omnipotent and can do only that much and a lot of "official" linux devs are putting really nasty SDK dependency requirements in everyday/everywhere kernels. That said, on my side, many of my user apps are now directly using linux syscalls... but are written in RISC-V assembly interpreted on x86_64 (I have a super lean interpreter/syscall translater written in x86_64 assembly and a super lean executable format wrapped in ELF executable format), or very plain and simple C99+ (legacy or because I want some apps to be a bit more 'platform crossy'... for now).
- gpderetta 2y ago
  It is hard to take seriously someone that claims that thread locals are a toxic addition to the standard. (incidentally __thread is a GCC extension that predates the standard by almost a decade).
  - sylware 2y ago
    This little keyword makes the compiler generates calls to the system threading libs (libpthread), or even worse, declare a TLS slot in the ELF format, which is a limited resource (you cannot reasonably resize the TLS segment on the fly). Yes, that amount of complexity is obviously toxic... and saying otherwise is what will make you hard to be taken seriously, come on dude...
jlokier 2y ago
Another use-case is when you are writing threaded code that uses the clone() syscall instead of pthreads, usually for something with high performance, unusual clone flags, or a very small stack. Most libc functions, including the syscall wrappers and all pthreads functions, aren't safe to call in threads created by raw clone(). Anything that reads or writes errno, for example, is not safe. I've had to do this a couple of times. One a long time ago was an audio mixing real-time thread for a video game, which had to keep the audio device fed with low-latency frames for sound effects. In those days, pthreads wasn't good enough. For talking to the audio device, we had to use the Linux syscall wrapper macros, which have been replaced by nolibc now. More recently, a thread pool for high-performance storage I/O, which ran slightly faster than io_uring, and ran well on older kernels and ones with io_uring disabled for security.
n_plus_1_acc 2y ago
Do I understand correctly that nolibc is just another implementation of the C standard library in terms of Linux syscalls? Comparably to, say, musl libc?
- sph 2y ago
  glibc is a space shuttle, musl is a hatchback, nolibc is a skateboard They all do the same thing (take you from A to B), but offer different levels of comfort, efficiency and utility :)
  - jonhohle 2y ago
    Who can take their space shuttle to work these days, what with the price of rocket fuel‽
    - sph 2y ago
      It's a bit unwieldy, but the good thing is that it comes for free with your copy of GNU/Linux!
    - nilamo 2y ago
      And parking is always a nightmare for my shuttles
    - malkia 2y ago
      Apparently almost every linux app
    - deleted 2y ago
      [deleted]
  - wbl 2y ago
    And passenger safety?
    - fallingsquirrel 2y ago
      In a head-on collision, the space shuttle passengers will fare better than the hatchback. Even so, it wouldn't be my first choice for most destinations.
yencabulator 2y ago
nolibc seems kinda neglected, or like a minimal subset of what's actually useful. There's no pread/pwrite etc, only read/write, forcing you to use lseek and ruining concurrent use.
- pantalaimon 2y ago
  Wasn’t that originally just for integration tests where you wanted to boot a minimal image that just runs your kernel CI test?
- t-8ch 2y ago
  It doesn't strive to be a complete implementation. If users send patches support for them, new syscalls can be added. Or downstream users can keep those definitions in their own program. More complex features like threads are out of scope. (Disclaimer: I'm one of the maintainers)
  - yencabulator 2y ago
    Sure, I'm more saying, because nolibc is so minimal, that still leaves room for things like this Chromium library.
deleted 2y ago
[deleted]
arjvik 2y ago
Where was this when I needed it! As one of my class projects, I built a Linux compatibility layer for the toy OS we had built, by adding a proper ELF loader and emulating syscalls. I really struggled to get glibc or even musl working, and so I ended up hand-coding some `-nostdlib` programs instead of being able to use coreutils. If nolibc really works as a minimal libc, would have been incredibly cool to be able to run coreutils on my OS!

jagrsw 2y ago

Just a friendly reminder that syscall() is a vararg function. Meaning, you can't just go throwing arguments at it (so maybe it's better to use this wrapper to avoid problems instead). For example, on a 64-bit arch, this code would be sus. syscall(__NR_syscall_taking_6_args, 1, 2, 3, 4, 5, 6); Quiz: why PS: it's a common mistake, so I thought I'd save you a trip down the debugging rabbit hole.

remram 2y ago
A quiz is the opposite of saving someone effort.
- eddd-ddde 2y ago
  Exactly, I am now morally bound to figure out the answer instead of going to work.
  - oguz-ismail 2y ago
    The last argument would be on the stack instead of in a register which is where the kernel expects to find the arguments. But a proper syscall implementation would handle this just fine (e.g. <https://github.com/bminor/glibc/blob/ba60be873554ecd141b55ea82647714933c0dbd4/sysdeps/unix/sysv/linux/x86_64/syscall.S https://github.com/bminor/glibc/blob/ba60be873554ecd141b55ea...>), so I don't think there's anything sus about it.
    - im3w1l 2y ago
      > movq 8(%rsp),%r9 This is a huge edgecase but is 8(%rsp) guaranteed to be readable memory
      - achierius 2y ago
        Yes, see https://en.wikipedia.org/wiki/Red_zone_(computing) https://en.wikipedia.org/wiki/Red_zone_(computing)

IAmLiterallyAB 2y ago

I've been using my own version of this. Maybe I'll switch over, this looks more complete.

kentonv 2y ago

Disappointing that errors are still signaled by assigning to `errno` (you can apparently override this to some other global, but it has to be a global or global-like lvalue). The kernel actually signals errors by returning a negative error code (on most arches), which seems like a better calling convention. Storing errors in something like `errno` opens a whole can of worms around thread safety and signal safety, while seemingly providing very little benefit beyond following tradition.

sidkshatriya 2y ago
code that uses errno is also a bit harder to understand. I like the way Rust does it -- if a function can fail, it returns a Result.
- dailykoder 2y ago
  While that might be true and the industry has evolved and learned about "better" ways, the old systems still exist. I don't see any reason to complain about it. Yes, we can do better. Yes, we probably should do better. But in some cases you really have to think through every edge case and in the end someone has to do it. So just be grateful for what we have.
  - IshKebab 2y ago
    I don't think this is an "old system" though.
  - alexey-salmin 2y ago
    For old systems -- yes, of course. But designing a new, incompatible API around errno is just backwards.
AndyKelley 2y ago
Disappointing is an understatement. Can't believe these people are making a browser. I'm sure they have some Google-flavored excuse for why to repeat this ridiculous threadlocal errno API.

wg0 2y ago

So web apps can make Linux sys calls? Or its about Chrome OS?

thinkharderdev 2y ago
The chrome browser itself I would think

Aransentin 2y ago

> We try to hide some of the differences between arches when reasonably feasible. e.g. Newer architectures no longer provide an open syscall, but do provide openat. We will still expose a sys_open helper by default that calls into openat instead. Sometimes you actually want to make sure that the exact syscall is called; e.g. you're writing a little program protected by strict seccomp rules. If the layer can magically call some other syscall under the hood this won't work anymore.

dundarious 2y ago
musl does this too. glibc may also, I haven't checked in a long time. I bet rust, etc., does too. You always need to check.
- king_geedorah 2y ago
  Glibc definitely does this transparent mapping as well. Calling int fd = open(<path>, O_RDONLY) yields openat(AT_FDCWD, <path>, O_RDONLY) when running through strace.
  - jchw 2y ago
    This really surprised me when I was digging into Linux tracing technology and noticed no `open` syscalls on my running system. It was all `openat`. I don't know when this transition happened, but I totally missed it.

mcnichol 2y ago

0-Day incoming

sedatk 2y ago

Can't wait for Zig team to adopt this over libc, citing concerns about "libc not existing on certain configurations"[1] [1] https://github.com/ziglang/zig/issues/1840 https://github.com/ziglang/zig/issues/1840

TUSF 2y ago
Zig on Linux already directly interfaces with syscalls,[0] unless your library or application directly links libc. [0]: https://ziglang.org/documentation/master/std/#std.os.linux https://ziglang.org/documentation/master/std/#std.os.linux
- sedatk 2y ago
  Oh, there you go :)
AndyKelley 2y ago
Welcome to 2016. https://github.com/ziglang/zig/blob/5f0bfcac24036e1fff0b2beda643a60dad465213/std/syscall.zig https://github.com/ziglang/zig/blob/5f0bfcac24036e1fff0b2bed...
- sedatk 2y ago
  :'(

iTokio 2y ago

Using go is a nice way to do that by default as it also directly uses syscalls (see the sys package)

zamalek 2y ago

You have to be pretty careful when using syscalls directly, at least in the presence of some libc. For example, from what I have gathered from random tidbits here and there, it's not a good idea to use any of the fork-like calls (fork, clone, etc.) if you have any threads that were created using glibc.