7 ms·

Not long ago golang switched from doing direct system calls to libc on *BSD systems. The explanation I believe was that the BSD systems fix a lot of kernel prob

Not long ago golang switched from doing direct system calls to libc on *BSD systems. The explanation I believe was that the BSD systems fix a lot of kernel problems and compatibility issues in libc and it was just easier to use libc then try to rewrite all of that in the golang standard libraries. On Linux it uses direct system calls for everything.

catiopatio 4y ago
I believe the impetus for Golang was macOS and Windows, rather than the BSDs. It’s not that libc is used to fix kernel problems/compatibility issues, but rather, the kernel ABI is not a supported, stable public API. On macOS, there are cases where using the syscall ABI directly will result in your code breaking when interacting with any other code that does correctly use libc, due to out-of-sync userspace state maintained by libc. For example, the fork(2) implementation in macOS’ libSystem will invalidate the cached copy of the current process’ pid used by getpid(2). If you fork(2) by directly trapping to a syscall, the cached pid won’t be invalidated; any future calls to getpid(2) through libSystem will return the stale pid.
- glandium 4y ago
  Things like this are true on Linux too. Don't call the clone syscall manually because it will mess up with glibc internals.
  - iainmerrick 4y ago
    But only if you’re also using glibc, right? If you’re only using syscalls and no libc at all, you’re fine (on Linux, and only Linux).
    - glandium 4y ago
      If you're only using syscalls on macOS, you're fine too, as far as libSystem is involved. You're not fine if newer kernels change the syscall ABI, though.
    - matheusmoreira 4y ago
      > But only if you’re also using glibc, right? Yes. As a glibc maintainer once said, if you use clone you're on your own. And that is fine.
- codeflo 4y ago
  Yes, I believe what language implementation would want to do is to use the lowest-level API that’s supported and stable. On Linux, that happens to be syscalls, on BSD and macOS, that’s libc, and on Windows, that’s the so-called “Win32” API (kernel32.dll and such).
- wongarsu 4y ago
  Meanwhile Windows makes no effort at all to keep syscall numbers stable, because you aren't supposed to use them directly. For example NtCompleteConnectPort has changed id five times in Windows 10 alone. Meanwhile OpenBSD was discussing ideas to extend their system-call-origin verification to only allow syscalls from libc.
  - cturner 4y ago
    Has any OS offered introspection so that you could determine such things at runtime?
    - stuaxo 4y ago
      In Windows does COM cover the NT APIs ?
- dom96 4y ago
  What's the motivation for Golang targeting syscalls directly? Is it just to avoid the libc dependency?
  - chc 4y ago
    I'm not sure it was libc in particular. In general, the Go compiler's initial design was very enthusiastic about the idea of fully statically linked binaries and easy cross-compilation. Avoiding system libs does kind of flow naturally from those goals.
chc4 4y ago
Some BSDs (and Zircon/Fuchsia) block any syscall instructions that don't originate from inside the vDSO. Which means the only way to do them is to go through libc (well, the parts of libc that are provided by the vDSO, but semantics). https://fuchsia.dev/fuchsia-src/concepts/kernel/vdso https://fuchsia.dev/fuchsia-src/concepts/kernel/vdso mentions this, for example.
- erichocean 4y ago
  I'm almost 100% certain that anyone can directly call Fuchsia's vDSO syscalls without linking in any form of libc. It's not at all like the macOS situation.
  - matheusmoreira 4y ago
    vDSOs are essentially pre-linked C libraries though. On Linux they are normal shared objects with the C ABI. They can even be found in the file system with names like linux-vdso.so.1.
    - account42 4y ago
      vDSOs are however an API provided by the kernel (hence the v, they aren't separate .so files that you need to load, even on Linux)
      - matheusmoreira 4y ago
        The vDSO is a normal .so file that gets built alongside the Linux kernel. The .so file is installed on the file system like any other library. When mapped into the process address space, it's just a normal ELF image. The functions all conform to the normal C ABI for the platform. It's true that we don't need to load the vDSO into the address space of the process since the kernel does it for us. We do need to manually link the vDSO functions though: the kernel uses the auxiliary vector to pass the address of the vDSO ELF image to the process, allowing it to parse the ELF header and find the addresses of the functions. In most cases, the libc will do this while initializing itself prior to calling main.
jart 4y ago
Why is it that that every time the topic of system calls comes up, all these people come out of the woodwork to say we're not allowed to not link a platform's libc? That's an impossible to compromise with position, because Cosmopolitan Libc is a libc, so it can't very well depend on six other c libraries. It would destroy the project. Also where is this even written? The maintainers of systems like FreeBSD and NetBSD have done SO MUCH to help our project. I don't think they're anything like the anti-competitive userspace control freaks people on forums suggest they are. I think the folks making kernels are great people and outstanding engineers who want to cultivate an environment where programs can do what they want.
- nibbleshifter 4y ago
  Many developers seem to have a cultish belief that the systems libc is the Only True Libc, missing the point (or being incapable of understanding) that alternatives are possible.
  - iainmerrick 4y ago
    Is that really any more cultish than the idea that the kernel interface is the only true interface, and everything else should be replaceable? It’s just a different approach. What’s so crazy about an OS vendor wanting to exercise control over that userspace shim layer, rather than assuming that user programs will be poking directly at the kernel? You can argue that having multiple libc implementations is good, sure, but it’s not the only way of doing things and it’s not without tradeoffs.
    - mananaysiempre 4y ago
      The argument in the first paragraph doesn’t really work, I think: the kernel-user boundary is a security one, while the libc-app one is not (OpenBSD is something of an outlier in this respect, and even its case is a bit murky). So it is a tad more natural to draw the ABI boundary at the former point than at the latter, though of course this is not at all the only design consideration. Looking at this through a KERNEL32 lens, I can actually somewhat see the alternative point of view: in NT < 4 it was mostly an in-process RPC proxy for the subsystem server in CSRSS, and one can argue that libc on Unix-likes is in the same position (cf the vDSO on Linux). You can build RPC to a trusted server either way: define a stable wire protocol; or require the client to load proxy code into its address space. COM on Windows and hardware-accelerated graphics on Linux both take the second approach. But then why the hell does the proxy also have opinions about memory allocation, assignment of TLS slots, or floating-point formatting? (KERNEL32 has them on the first two points as well, mind you.) History[1] aside, does this really look like good engineering? (KERNEL32 makes a bit of a point there, but given how much HeapAlloc sucks as an allocator compared to the interoperability benefits it brings, I’m not sure it should be taken too seriously.) [1] https://utcc.utoronto.ca/~cks/space/blog/unix/UnixAPIAndCRuntime https://utcc.utoronto.ca/~cks/space/blog/unix/UnixAPIAndCRun...
__turbobrew__ 4y ago
Yea, if I remember correctly OpenBSD only allows syscalls to originate from libc so the OpenBSD devs had to create an exception for golang back when it didn’t use the system libc.

catiopatio 4y ago

I believe the impetus for Golang was macOS and Windows, rather than the BSDs. It’s not that libc is used to fix kernel problems/compatibility issues, but rather, the kernel ABI is not a supported, stable public API. On macOS, there are cases where using the syscall ABI directly will result in your code breaking when interacting with any other code that does correctly use libc, due to out-of-sync userspace state maintained by libc. For example, the fork(2) implementation in macOS’ libSystem will invalidate the cached copy of the current process’ pid used by getpid(2). If you fork(2) by directly trapping to a syscall, the cached pid won’t be invalidated; any future calls to getpid(2) through libSystem will return the stale pid.

glandium 4y ago
Things like this are true on Linux too. Don't call the clone syscall manually because it will mess up with glibc internals.
- iainmerrick 4y ago
  But only if you’re also using glibc, right? If you’re only using syscalls and no libc at all, you’re fine (on Linux, and only Linux).
  - glandium 4y ago
    If you're only using syscalls on macOS, you're fine too, as far as libSystem is involved. You're not fine if newer kernels change the syscall ABI, though.
  - matheusmoreira 4y ago
    > But only if you’re also using glibc, right? Yes. As a glibc maintainer once said, if you use clone you're on your own. And that is fine.
codeflo 4y ago
Yes, I believe what language implementation would want to do is to use the lowest-level API that’s supported and stable. On Linux, that happens to be syscalls, on BSD and macOS, that’s libc, and on Windows, that’s the so-called “Win32” API (kernel32.dll and such).
wongarsu 4y ago
Meanwhile Windows makes no effort at all to keep syscall numbers stable, because you aren't supposed to use them directly. For example NtCompleteConnectPort has changed id five times in Windows 10 alone. Meanwhile OpenBSD was discussing ideas to extend their system-call-origin verification to only allow syscalls from libc.
- cturner 4y ago
  Has any OS offered introspection so that you could determine such things at runtime?
  - stuaxo 4y ago
    In Windows does COM cover the NT APIs ?
dom96 4y ago
What's the motivation for Golang targeting syscalls directly? Is it just to avoid the libc dependency?
- chc 4y ago
  I'm not sure it was libc in particular. In general, the Go compiler's initial design was very enthusiastic about the idea of fully statically linked binaries and easy cross-compilation. Avoiding system libs does kind of flow naturally from those goals.

chc4 4y ago

Some BSDs (and Zircon/Fuchsia) block any syscall instructions that don't originate from inside the vDSO. Which means the only way to do them is to go through libc (well, the parts of libc that are provided by the vDSO, but semantics). https://fuchsia.dev/fuchsia-src/concepts/kernel/vdso https://fuchsia.dev/fuchsia-src/concepts/kernel/vdso mentions this, for example.

erichocean 4y ago
I'm almost 100% certain that anyone can directly call Fuchsia's vDSO syscalls without linking in any form of libc. It's not at all like the macOS situation.
- matheusmoreira 4y ago
  vDSOs are essentially pre-linked C libraries though. On Linux they are normal shared objects with the C ABI. They can even be found in the file system with names like linux-vdso.so.1.
  - account42 4y ago
    vDSOs are however an API provided by the kernel (hence the v, they aren't separate .so files that you need to load, even on Linux)
    - matheusmoreira 4y ago
      The vDSO is a normal .so file that gets built alongside the Linux kernel. The .so file is installed on the file system like any other library. When mapped into the process address space, it's just a normal ELF image. The functions all conform to the normal C ABI for the platform. It's true that we don't need to load the vDSO into the address space of the process since the kernel does it for us. We do need to manually link the vDSO functions though: the kernel uses the auxiliary vector to pass the address of the vDSO ELF image to the process, allowing it to parse the ELF header and find the addresses of the functions. In most cases, the libc will do this while initializing itself prior to calling main.

jart 4y ago

Why is it that that every time the topic of system calls comes up, all these people come out of the woodwork to say we're not allowed to not link a platform's libc? That's an impossible to compromise with position, because Cosmopolitan Libc is a libc, so it can't very well depend on six other c libraries. It would destroy the project. Also where is this even written? The maintainers of systems like FreeBSD and NetBSD have done SO MUCH to help our project. I don't think they're anything like the anti-competitive userspace control freaks people on forums suggest they are. I think the folks making kernels are great people and outstanding engineers who want to cultivate an environment where programs can do what they want.

nibbleshifter 4y ago
Many developers seem to have a cultish belief that the systems libc is the Only True Libc, missing the point (or being incapable of understanding) that alternatives are possible.
- iainmerrick 4y ago
  Is that really any more cultish than the idea that the kernel interface is the only true interface, and everything else should be replaceable? It’s just a different approach. What’s so crazy about an OS vendor wanting to exercise control over that userspace shim layer, rather than assuming that user programs will be poking directly at the kernel? You can argue that having multiple libc implementations is good, sure, but it’s not the only way of doing things and it’s not without tradeoffs.
  - mananaysiempre 4y ago
    The argument in the first paragraph doesn’t really work, I think: the kernel-user boundary is a security one, while the libc-app one is not (OpenBSD is something of an outlier in this respect, and even its case is a bit murky). So it is a tad more natural to draw the ABI boundary at the former point than at the latter, though of course this is not at all the only design consideration. Looking at this through a KERNEL32 lens, I can actually somewhat see the alternative point of view: in NT < 4 it was mostly an in-process RPC proxy for the subsystem server in CSRSS, and one can argue that libc on Unix-likes is in the same position (cf the vDSO on Linux). You can build RPC to a trusted server either way: define a stable wire protocol; or require the client to load proxy code into its address space. COM on Windows and hardware-accelerated graphics on Linux both take the second approach. But then why the hell does the proxy also have opinions about memory allocation, assignment of TLS slots, or floating-point formatting? (KERNEL32 has them on the first two points as well, mind you.) History[1] aside, does this really look like good engineering? (KERNEL32 makes a bit of a point there, but given how much HeapAlloc sucks as an allocator compared to the interoperability benefits it brings, I’m not sure it should be taken too seriously.) [1] https://utcc.utoronto.ca/~cks/space/blog/unix/UnixAPIAndCRuntime https://utcc.utoronto.ca/~cks/space/blog/unix/UnixAPIAndCRun...

__turbobrew__ 4y ago

Yea, if I remember correctly OpenBSD only allows syscalls to originate from libc so the OpenBSD devs had to create an exception for golang back when it didn’t use the system libc.