6 ms·

I understand that but if they're going to go down that route for OpenBSD why not bring it to all platforms? While I can believe that going the raw syscall route

by simias 6y ago

I understand that but if they're going to go down that route for OpenBSD why not bring it to all platforms? While I can believe that going the raw syscall route can be easier than dealing with libc idiosyncrasies, it seems to me that maintaining both interfaces depending on the target OS would be trickier than just cutting your losses and going the libc route everywhere. After all as far I can tell it's not just an OpenBSD problem since famously they got breakage in MacOS as well. I must admit that I haven't taken the time to analyze in depth the pros and cons here, but Go's history of NIH coupled with the fact that basically every other mainstream language manages to work by binding the libc leaves me very perplex. In particular some of the points raised in the blogpost I linked seem fishy to me. For instance errno being a global: this is in no way a Go-specific problem (multithreaded C couldn't run concurrent syscalls if it was true). In practice errno is thread-local instead of being a true global. It's explicitly documented in the man page: > errno is defined by the ISO C standard to be a modifiable lvalue of type int, and must not be explicitly declared; errno may be a macro. errno is thread-local; setting it in one thread does not affect its value in any other thread. I can believe that there are other issues I fail to consider, but again it works for everybody else, what makes Go so special here?

wbl 6y ago
Go wants low overhead multithreading and static linkage. Glibc is inherently dynamically linked, and sets up threads in ways Go doesn't want. Error doesn't work for everyone else. It's a hack, you wouldn't do it if it wasn't for backwards compatability.
- IgorPartola 6y ago
  Not every libc is a glibc.
  - wbl 6y ago
    True but errorno is still a hack.
- Blikkentrekker 6y ago
  But why does it want to not dynamically link against a platform's libc? What is the advantage in that? including that into a binary seems about as senseless to me as including the entire kernel in it.
  - zlynx 6y ago
    For containers Go static binaries are great. Used to build Docker containers for Kubernetes using a 8 MB Go binary. That was all you needed. No libraries. No "minimum OS' Alpine or Ubuntu image.
    - Blikkentrekker 6y ago
      That seems to be a sensible use yes, but those aren't really any targeted and supported platform.
  - badsectoracula 6y ago
    Not all platforms have a "libc". Linux is an example: there is no standard c library and while glibc is very common, there are several distributions that use other C libraries like uClibc and musl. For example a gaming handheld i have which is running Linux uses uClibc Another example is Windows, the platform API does not provide a C library (even MSVC has its own). While there is an MSVCRT.DLL it is not recommended to link against it as it is there only because some other software relies on it and its semantics are for around Visual C++ 4 (IIRC).
    - pjmlp 6y ago
      Windows used to be an example like any other non-POSIX OS, however as of Windows 10, the C standard library is part of the OS. https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/ https://devblogs.microsoft.com/cppblog/introducing-the-unive...
      - badsectoracula 6y ago
        Ah yes, i've forgotten about it. I wonder how reliable that is as i noticed a bunch of applications i have ship their own copy of ucrtbase.dll.
masklinn 6y ago
> I understand that but if they're going to go down that route for OpenBSD why not bring it to all platforms? Because they really don’t want to, so they will avoid doing it until forced to, as previously happened with e.g. macOS. > I must admit that I haven't taken the time to analyze in depth the pros and cons here, but Go's history of NIH coupled with the fact that basically every other mainstream language manages to work by binding the libc leaves me very perplex. One of the issues Go has is it uses its own non-C stack. Libcs generally assume a C stack and don’t understand growable movable stacks so they can get very cross when called with an unexpectedly small stack (iirc Go defaults to 2k while the smallest C stack I know of is 64k on old OpenBSD, then macOS’s 512k for the non-main threads).
- aduitsis 6y ago
  Obviously I'm missing something here, because I was under the impression that, regardless of how big the stack is, the pages are going to be mapped (in the data structure pointed to by the CR3 register) when accessed and no earlier. Unless the allocation of the stack makes sure that it is mapped in its entirety upfront and stack access cannot incur minor page faults, which may not be so outlandish come to think of it. Can you please provide some clarification here? Thanks!
  - masklinn 6y ago
    I'm not sure what clarification I could provide given I don't know what you're missing. You're correct (AFAIK anyway) that stack pages are mapped lazily. That doesn't change the fact that C stacks are large allocations, while Go will only allocate a very small stack (2k last I checked) per goroutine.
    - aduitsis 6y ago
      Yes, sorry, let me be more clear, I don't understand why Go should allocate a small 2k stack (you are correct, the Go stack size has fluctuated through the years, nowadays it's 2k) and if it grows go to the trouble of copying it to a larger structure? Instead of just allocating a big stack that will grow lazily anyway? Obviously the folks that created Go aren't stupid, far from it, so there must be a real and valid reason. But can't easily imagine what it is.
Quekid5 6y ago
> > [errno] > I can believe that there are other issues I fail to consider, but again it works for everybody else, what makes Go so special here? errno being thread-local doesn't really help all that much with a M:N threading model -- the runtime is going to have to be extremely careful to not stomp all over it. (Whenever calling into anything which uses errno. Of course system calls do that as well, but it's a much smaller surface area than "most of libc".)

by simias 6y ago

wbl 6y ago

Go wants low overhead multithreading and static linkage. Glibc is inherently dynamically linked, and sets up threads in ways Go doesn't want. Error doesn't work for everyone else. It's a hack, you wouldn't do it if it wasn't for backwards compatability.

IgorPartola 6y ago
Not every libc is a glibc.
- wbl 6y ago
  True but errorno is still a hack.
Blikkentrekker 6y ago
But why does it want to not dynamically link against a platform's libc? What is the advantage in that? including that into a binary seems about as senseless to me as including the entire kernel in it.
- zlynx 6y ago
  For containers Go static binaries are great. Used to build Docker containers for Kubernetes using a 8 MB Go binary. That was all you needed. No libraries. No "minimum OS' Alpine or Ubuntu image.
  - Blikkentrekker 6y ago
    That seems to be a sensible use yes, but those aren't really any targeted and supported platform.
- badsectoracula 6y ago
  Not all platforms have a "libc". Linux is an example: there is no standard c library and while glibc is very common, there are several distributions that use other C libraries like uClibc and musl. For example a gaming handheld i have which is running Linux uses uClibc Another example is Windows, the platform API does not provide a C library (even MSVC has its own). While there is an MSVCRT.DLL it is not recommended to link against it as it is there only because some other software relies on it and its semantics are for around Visual C++ 4 (IIRC).
  - pjmlp 6y ago
    Windows used to be an example like any other non-POSIX OS, however as of Windows 10, the C standard library is part of the OS. https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/ https://devblogs.microsoft.com/cppblog/introducing-the-unive...
    - badsectoracula 6y ago
      Ah yes, i've forgotten about it. I wonder how reliable that is as i noticed a bunch of applications i have ship their own copy of ucrtbase.dll.

masklinn 6y ago

> I understand that but if they're going to go down that route for OpenBSD why not bring it to all platforms? Because they really don’t want to, so they will avoid doing it until forced to, as previously happened with e.g. macOS. > I must admit that I haven't taken the time to analyze in depth the pros and cons here, but Go's history of NIH coupled with the fact that basically every other mainstream language manages to work by binding the libc leaves me very perplex. One of the issues Go has is it uses its own non-C stack. Libcs generally assume a C stack and don’t understand growable movable stacks so they can get very cross when called with an unexpectedly small stack (iirc Go defaults to 2k while the smallest C stack I know of is 64k on old OpenBSD, then macOS’s 512k for the non-main threads).

aduitsis 6y ago
Obviously I'm missing something here, because I was under the impression that, regardless of how big the stack is, the pages are going to be mapped (in the data structure pointed to by the CR3 register) when accessed and no earlier. Unless the allocation of the stack makes sure that it is mapped in its entirety upfront and stack access cannot incur minor page faults, which may not be so outlandish come to think of it. Can you please provide some clarification here? Thanks!
- masklinn 6y ago
  I'm not sure what clarification I could provide given I don't know what you're missing. You're correct (AFAIK anyway) that stack pages are mapped lazily. That doesn't change the fact that C stacks are large allocations, while Go will only allocate a very small stack (2k last I checked) per goroutine.
  - aduitsis 6y ago
    Yes, sorry, let me be more clear, I don't understand why Go should allocate a small 2k stack (you are correct, the Go stack size has fluctuated through the years, nowadays it's 2k) and if it grows go to the trouble of copying it to a larger structure? Instead of just allocating a big stack that will grow lazily anyway? Obviously the folks that created Go aren't stupid, far from it, so there must be a real and valid reason. But can't easily imagine what it is.

Quekid5 6y ago

> > [errno] > I can believe that there are other issues I fail to consider, but again it works for everybody else, what makes Go so special here? errno being thread-local doesn't really help all that much with a M:N threading model -- the runtime is going to have to be extremely careful to not stomp all over it. (Whenever calling into anything which uses errno. Of course system calls do that as well, but it's a much smaller surface area than "most of libc".)