4 ms·

There is a difference though: libSystem on Darwin is a very thin wrapper over the kernel syscalls; on the contrary, libc is a library that was designed for C, t

There is a difference though: libSystem on Darwin is a very thin wrapper over the kernel syscalls; on the contrary, libc is a library that was designed for C, then standardized in POSIX, and has several layer of abstraction over kernel syscalls including many bad defaults that are universally recognized as wrong today (eg: libc’s created file descriptors will all inherit by default).

cesarb 7y ago
> (eg: libc’s created file descriptors will all inherit by default) Isn't that the kernel default? Even if you use system calls directly, file descriptors still inherit by default.
- Gaelan 7y ago
  I mean, theoretically (i.e. I have no idea if anything does this), you could have underlying system file descriptors, which did not inherit, then a mapping from "libc" file descriptors onto OS ones and some code in fork() to copy over any OS file descriptors that are exposed to libc-using code.
  - loeg 7y ago
    The mapping is the identity function and whether a descriptor is inherit or not is just a function of O_CLOEXEC / fcntl(FD_CLOEXEC) / some special fd types are always cloexec, such as kqueues on FreeBSD. Libc fds aren't special to the operating system in any way.
- loeg 7y ago
  Yeah, libc's syscall wrappers just do what you tell them. If you don't pass O_CLOEXEC to the kernel syscalls, you get the inherit behavior. Libc's syscall wrappers don't change this in any way. To the extent that Go's default for file descriptors today is !inherit (I'm unfamiliar, but if so, it's a good choice), the Go runtime must already add O_CLOEXEC to bare syscalls. There's no reason to believe it incapable of adding the flag to libc syscalls instead.
  - giovannibajo1 7y ago
    You can't do that atomically with libc. There's a short window in which the file descriptor will potentially be inherited, if another thread forks.
    - jlokier 7y ago
      That's incorrect. You are thinking of the older way, where fcntl(fd, F_SETFD, FD_CLOEXEC) must be used after open(), leaving a short window in which the file descriptor may be inherited. The newer way passes the O_CLOEXEC to open() and there is no fcntl() call. This is atomic with respect to inheritability: The kernel returns a non-inheritable file descriptor to libc, and libc returns it to the application. Other syscalls that return a file descriptor have similar flags, so they are atomic too. These flags and behaviours are exactly the same, whether done by calling through libc as most programs do, or direct kernel syscalls bypassing libc, as Go and a few other programs do.
    - loeg 7y ago
      Unfortunately, you misunderstand how CLOEXEC works and how the Go runtime implements the feature you think libc lacks. This syscall level behavior is POSIX-specified[1] since at least the 2008 edition[2]: > O_CLOEXEC > If set, the FD_CLOEXEC flag for the new file descriptor shall be set. What that means is, any C program or Go program that passes the O_CLOEXEC flag to open(2) on a POSIX 2008 conforming system (including Linux and the BSDs, for example), will atomically create the fd without inherit behavior. There is no "short window" and hasn't been for more than a decade. The Go runtime must use that flag to provide that property; there is no other way on these systems. Libc users are of course able to use the same flag. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/o... [2]: https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/functions/open.html https://pubs.opengroup.org/onlinepubs/9699919799.2008edition...
loeg 7y ago
Go isn't obligated to use any libc APIs or abstractions other than those providing syscalls. You're incorrect or maybe just misleading about libc created file descriptors inheriting, as stated. Either way, it is unrelated to using libc for syscalls vs bare machine traps.
- giovannibajo1 7y ago
  I think it's mandated by POSIX standard; but even if I'm wrong, there's still the problem that libc doesn't allow to do that atomically, for instance. In general, it's an old interface that doesn't fully expose the full power of all modern syscalls.
  - tedunangst 7y ago
    Most libc you'd care to use are going to include functions like dup3. If they don't, it's probably because the system call isn't available either.
  - loeg 7y ago
    You're mistaken. https://news.ycombinator.com/item?id=21740035 https://news.ycombinator.com/item?id=21740035 https://news.ycombinator.com/item?id=21740498 https://news.ycombinator.com/item?id=21740498

cesarb 7y ago

> (eg: libc’s created file descriptors will all inherit by default) Isn't that the kernel default? Even if you use system calls directly, file descriptors still inherit by default.

Gaelan 7y ago
I mean, theoretically (i.e. I have no idea if anything does this), you could have underlying system file descriptors, which did not inherit, then a mapping from "libc" file descriptors onto OS ones and some code in fork() to copy over any OS file descriptors that are exposed to libc-using code.
- loeg 7y ago
  The mapping is the identity function and whether a descriptor is inherit or not is just a function of O_CLOEXEC / fcntl(FD_CLOEXEC) / some special fd types are always cloexec, such as kqueues on FreeBSD. Libc fds aren't special to the operating system in any way.
loeg 7y ago
Yeah, libc's syscall wrappers just do what you tell them. If you don't pass O_CLOEXEC to the kernel syscalls, you get the inherit behavior. Libc's syscall wrappers don't change this in any way. To the extent that Go's default for file descriptors today is !inherit (I'm unfamiliar, but if so, it's a good choice), the Go runtime must already add O_CLOEXEC to bare syscalls. There's no reason to believe it incapable of adding the flag to libc syscalls instead.
- giovannibajo1 7y ago
  You can't do that atomically with libc. There's a short window in which the file descriptor will potentially be inherited, if another thread forks.
  - jlokier 7y ago
    That's incorrect. You are thinking of the older way, where fcntl(fd, F_SETFD, FD_CLOEXEC) must be used after open(), leaving a short window in which the file descriptor may be inherited. The newer way passes the O_CLOEXEC to open() and there is no fcntl() call. This is atomic with respect to inheritability: The kernel returns a non-inheritable file descriptor to libc, and libc returns it to the application. Other syscalls that return a file descriptor have similar flags, so they are atomic too. These flags and behaviours are exactly the same, whether done by calling through libc as most programs do, or direct kernel syscalls bypassing libc, as Go and a few other programs do.
  - loeg 7y ago
    Unfortunately, you misunderstand how CLOEXEC works and how the Go runtime implements the feature you think libc lacks. This syscall level behavior is POSIX-specified[1] since at least the 2008 edition[2]: > O_CLOEXEC > If set, the FD_CLOEXEC flag for the new file descriptor shall be set. What that means is, any C program or Go program that passes the O_CLOEXEC flag to open(2) on a POSIX 2008 conforming system (including Linux and the BSDs, for example), will atomically create the fd without inherit behavior. There is no "short window" and hasn't been for more than a decade. The Go runtime must use that flag to provide that property; there is no other way on these systems. Libc users are of course able to use the same flag. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/o... [2]: https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/functions/open.html https://pubs.opengroup.org/onlinepubs/9699919799.2008edition...

loeg 7y ago

Go isn't obligated to use any libc APIs or abstractions other than those providing syscalls. You're incorrect or maybe just misleading about libc created file descriptors inheriting, as stated. Either way, it is unrelated to using libc for syscalls vs bare machine traps.

giovannibajo1 7y ago
I think it's mandated by POSIX standard; but even if I'm wrong, there's still the problem that libc doesn't allow to do that atomically, for instance. In general, it's an old interface that doesn't fully expose the full power of all modern syscalls.
- tedunangst 7y ago
  Most libc you'd care to use are going to include functions like dup3. If they don't, it's probably because the system call isn't available either.
- loeg 7y ago
  You're mistaken. https://news.ycombinator.com/item?id=21740035 https://news.ycombinator.com/item?id=21740035 https://news.ycombinator.com/item?id=21740498 https://news.ycombinator.com/item?id=21740498