6 ms·
Is the C runtime and library a legitimate part of the Unix API? (2017)
- sally1620 6y agoGo does things very differently. They will probably roll their own drivers at some point and run directly on Xen or KVM. The designers of Go have a lot of problems with evolution of Unix in general. So it is understandable that they don't want to get involved with libc clusterfluck.
- camgunz 6y agoYeah but that means if you want to fix things at the libc level, you necessarily skip Go. OpenBSD had/has this problem. But more broadly, I think the policy is less "Unix took a wrong turn" and more that goroutines make C/Go interop painful.
- jdub 6y agoBut then, Go is the lurching zombie of Plan 9. :-)
- pjmlp 6y agoInferno.
- jart 6y agoEven if it was "unix took a wrong turn" there wouldn't be any project on earth more qualified to make that claim. I mean, it's Rob Pike & Co. and if you look at their Git repo the first several commits are by Brian Kernighan. They're former Bell Labs employees who worked on things like UNIX.
- rwmj 6y agoAnd even with all that experience they still came up with a terrible programming language.
- Ericson2314 6y agoIf I were Google, go should be shut down, and the Plan 9 crowd reassigned to Fuschia and forced to only write in languages outside their puny comfort zone like Rust.
- pjmlp 6y agoThe fact that everyone forgets about Limbo and Inferno, and keeps talking about Plan 9, proves how successful Go would have been with another employer.
- Ericson2314 6y agoHuh? I think Rob Pike et al. suck at language design, period, but had some decent ideas for systems. I think Go would suck with any employer, but they could do better work on non-programming language things.
- pjmlp 6y agoNope, as proven with Limbo. Go 1.0 is basically Limbo with a bit of Oberon-2 syntax. Outside the chocolate factory, it would have been just as successful, to the point that everyone forgets about Inferno and only talks about Plan 9.
- mseepgood 6y ago> The designers of Go have a lot of problems with evolution of Unix in general. Well, they invented Unix, and later its improved successor (Plan 9).
- asdfasgasdgasdg 6y ago> The standard C library's runtime environment is designed for C, and it generally contains a tangled skein of assumptions about how things work. Forcing all other languages to fit themselves into these undocumented constraints is clearly confining, and the standard C library generally isn't designed to be a transparent API; in fact, at least GNU libc deliberately manipulates what it does under the hood to be more useful to C programs. It's hard to reconcile this with the fact that every other language besides Go manages to interface with the standard c library just fine. It seems like if it were such a big problem, you'd see more languages opting out.
- atombender 6y agoThe reason Go does it is because the overhead of calling C code from Go is relatively high. Other languages typically don't suffer from this friction. The closest I can think of is Java, but I have no idea how much is reimplemented in pure Java to avoid the overhead of JNI.
- asdfasgasdgasdg 6y agoThat may be so. But that just seems like another way of saying, "Go has an issue with this and other languages don't." Why is calling into C so slow on Go and so not slow in other languages like Java or C#?
- atombender 6y agoBecause Go decided on a runtime design that doesn't play well with the OS. As far as I can gather: * Go uses its own calling convention that's different from C. * Go doesn't use system stacks (goroutines have their own growable stacks). * Goroutines are implemented as green threads using collaborative scheduling. Any call to C has to play nice with the scheduler to avoid blocking other goroutines running on the same thread. * Garbage collection often requires that data be copied in order to be passed to the C library, since the original data may be freed by GC during execution.
- 6y ago
- camgunz 6y agoI'm not 100% sure this is right, but if there's anything that can be called a Unix API it's POSIX, and libc is part of the specification. So I think the answer is yes. OP references Illumos, but I also think the only way you get access to syscalls on macOS is through Apple's libc.
- asdfasgasdgasdg 6y ago> OP references Illumos, but I also think the only way you get access to syscalls on macOS is through Apple's libc. That's right. In fact this was a source of a time-related bug in Go on Mac OS. https://github.com/golang/go/issues/17490 https://github.com/golang/go/issues/17490 https://github.com/golang/go/issues/16570 https://github.com/golang/go/issues/16570
- carterschonwald 6y agoThere’s even some fun gotchas whereby to correctly bind some of the symbols, you need to make sure the correct header expanding c stub wrapper is used to make sure the sym linker works correctly on OS X. Was helping a friend with some ffi binding for Mac earlier this year and that wound up being key
- TwoBit 6y agoIt scares me that passing too little stack space to libc results in such subtle failures.
- ridiculous_fish 6y agoThis was libc attempting to detect insufficient stack space, under the assumption of a guard page. Go chose to not create a guard page. Nobody can force you to fasten your seatbelt.
- saagarjha 6y agoBut nobody requires that you fasten your seatbelt, either. I believe a stack check is still opt-in using the default flags on most compilers, unless you alloca. Plus, stack probes are merely precautionary, they can still be broken by an inconvenient sequence of stack allocations and signal handlers being run.
- slrz 6y agoNot libc. The stack probe was inserted by the compiler when compiling the vDSO (part of the kernel, but mapped into user processes) with certain ricer^Whardening flags enabled.
- saagarjha 6y agoThis is one of the “gotchas” in C, because the standard doesn’t define anything about the stack at all but it is really quite easy to run into issues if you’re not careful even with “standards compliant” programs. What’s scarier is that this can happen at any function call, and there is no standard way to detect this issue. (Most compilers do support stack probes, though.)
- andi999 6y agoIf there is not enough heap there is the OOM Killer (basically randomly kill processes), also scary.
- jart 6y agoGoogle has a completely different culture from the open source community when it comes to stack. Many FOSS programmers assume an 8mb stack due to a belief that malloc() is slow. Google built all its production services under the assumption of a 64kb stack, because they found a way to make malloc() go fast and wanted to have lots of threads. So it shouldn't be scary or unexpected that there's a little bit of friction bringing those two worldviews in harmony. Google was nice enough to open source tcmalloc.
- peter_d_sherman 6y ago>"Ordinary non-C languages on Unixes generally implement a great many low level operations by calling into the standard C library. This starts with things like making system calls, but also includes operations such as getaddrinfo(3). Go doesn't do this; it implements as much as possible itself, going straight down to direct system calls in assembly language. Occasionally there are problems that ensue." This is an interesting tidbit about Go to the perspective of an OS designer; basically an aspiring OS designer would only need to implement the subset of the Unix API syscalls that Go actually required, and only to the degree that Go needed specific functionality from them; that is, it would be an interesting idea to code a new OS and/or OS microkernel -- which would only support Go's subset of Unix's syscalls/API... Or, perhaps even more interesting -- code a 'bare-metal' version of Go which includes its own OS-like support code (mini internal OS functionality), implementing the Unix syscalls it makes, but on bare metal hardware..."
- zokier 6y agoIsn't Tamago (that hit the front page few days ago) exactly that sort of bare-metal Go https://github.com/f-secure-foundry/tamago https://github.com/f-secure-foundry/tamago
- etaioinshrdlu 6y agoWell, apparently this idea didn't always work. Go uses libc on macOS now. https://groups.google.com/g/golang-nuts/c/uX8eUeyuuAY https://groups.google.com/g/golang-nuts/c/uX8eUeyuuAY
- halayli 6y agowhile there is an overlap between libc and POSIX where by POSIX defines its standard in terms of the C language spec and its std lib, getaddrinfo(3) behavior is part of POSIX standard only. Obviously go devs are attempting to do the right thing here and trying to implement async version of getaddrinfo(3) because POSIX's version is not and this route is extremely messy as it ends up requiring to reimplement bind's libs, and or the whole dns standard. But this has nothing to do with C's std lib. its posix interface is defined in C, that's all about it. What's being attempted here is to implement a POSIX defined function in another language and with a different behavior (async). If there exist a getaddrinfo_a() in the posix standard, go devs wouldn't have had any issue and they would have went the typical route and created a binding between go's call and the posix call provided by the OS.
- jlokier 6y ago> If there exist a getaddrinfo_a() in the posix standard, In principle, if you can assume modern POSIX, then you can also spawn a POSIX thread to call getaddrinfo() and wait asynchronously for the result. In practice getaddrinfo() isn't always thread-safe :/ But it's supposed to be.
- userbinator 6y agoIn the microcomputer world, things are very different and libc is basically never the standard API; for example, in DOS, the API was not libc, but of course many applications would just go down to the BIOS or the hardware directly, for efficiency reasons, and C was only one of many other languages used --- Pascal and Asm were also quite common encountered. On Windows, the stable API is a set of exported functions from DLLs like GDI, KERNEL, and USER, and although libc is available, it is a layer on top of those. The libc as the API seems to be a specific characteristic of POSIX/Unix systems.
- yuhong 6y agoThe politics between Windows and DevDiv are just as interesting as well. For example it was @malwareminigun (used to work on MSVC compiler team) on Twitter that complained about the CPU time that CompatTelRunner consumes (mostly by Appraiser gathering application and device inventory).
- userbinator 6y agoI'm pretty sure not everyone at MS agrees with the spyware they're adding to Windows either.
- yuhong 6y ago@MalwareMinigun is Billy O'Neal BTW. I have a Wikipedia article about this topic: https://en.wikipedia.org/wiki/Draft:Upgrade_Readiness https://en.wikipedia.org/wiki/Draft:Upgrade_Readiness Appraiser not only consumes CPU (and HDD/SSD!) time, but is also technically quite interesting as well. For example it loads a simple kernel driver (nxquery.sys) into your Windows 7 machine to read a MSR to determine if NX has been disabled in the BIOS.
- saagarjha 6y agoYour comment seems a bit out-of-place, am I missing a connection between your link and this Hacker News post?
- rwmj 6y agoThis causes a problem in OpenBSD where they want to verify that system calls are only made from libc (https://lwn.net/Articles/806776/ https://lwn.net/Articles/806776/).
- 2sk21 6y agoBack in the 1980s when I was learning programming on a Unix machine, it took me quite a while to understand the difference between sections 2 and 3 of the man pages.
- tyingq 6y agoYou need it to know which syscall is which, so I suppose yes. There's /proc/kallsyms for Linux, but that's not portable.