6 ms·

> Basically, the Go folks want to minimize external dependencies and the web of failure that can lead to. What is he ranting on about? The reason Go doesn't li

by smegel 11y ago

> Basically, the Go folks want to minimize external dependencies and the web of failure that can lead to. What is he ranting on about? The reason Go doesn't link against system libraries is that Go in binary-incompatible with C, as in it's calling convention is different. There is a way to call C through a shim but it's extremely slow which is why they implemented their own syscall interface for Go. Static linking is just a side-effect of this. Edit: the Go team want to support dynamic linking and it is on the roadmap but not fully implemented as of 1.5 https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGhFDelrAP0NqSS_00RgZQ/edit# https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGh...

readams 11y ago
Static linking is independent of this concern. Go could implement dynamic linking but doesn't for ideological reasons. I'd really hate to be a distribution maintainer in a world where go components are common. Security bug in the TLS library? Time to download 14GB of updates!
- nulltype 11y ago
  Just download the deltas instead?
- xyproto 11y ago
  Not ideological reasons. The latest version of Go supports dynamic linking for some platforms.
  - pjmlp 11y ago
    On Android there is no way around it, as native code can only be accessed via dynamic linking. This is what some Go designers think of dynamic linking: http://harmful.cat-v.org/software/dynamic-linking/ http://harmful.cat-v.org/software/dynamic-linking/
    - shadowmint 11y ago
      Yeeesss... but ultimately, Go is also a pragmatic language. Dynamic linking is coming to Go, like it or not; because it's a feature that is needed in certain circumstances, and that has been acknowledged, even by the Go team.
    - GauntletWizard 11y ago
      And those reasons are clear and clear-cut. Dynamic linking is a net negative in most use cases, and certainly in all the use-cases that the Go developers are developing for, where reproducible builds are mandatory, code size is a tiny fraction of main memory, and code quality is very strictly maintained. When you've built a good development and deploy strategy, rebuilding and redistributing your binary is an everyday activity. There's no additional cost.
    - bahamat 11y ago
      That same web page lists Linux as harmful. http://harmful.cat-v.org/software/operating-systems/linux/ http://harmful.cat-v.org/software/operating-systems/linux/
- marssaxman 11y ago
  I'm happy to be an end user in a world where go components are becoming more common, because statically-linked executables are far less likely to break when I upgrade unrelated programs.
tedunangst 11y ago
I doubt go to C calling conventions are going to be the slowest part of calling tcsetattr(). Using ioctl() is being difficult for the sake of being difficult.
- pcwalton 11y ago
  > I doubt go to C calling conventions are going to be the slowest part of calling tcsetattr(). Well, by "C calling conventions" it really means "performing a stack switch to get out of M:N, allocating a big stack if necessary or taking a lock to check one out". It's quite a lot of overhead and I wouldn't be surprised if it dwarfs the cost of the SYSENTER/SYSEXIT pair in many cases.
  - tedunangst 11y ago
    All of which is likely dwarfed by all the code that runs between SYSENTER and SYSEXIT. And how many freaking times do you need to call tcsetattr()? Even worst case, if calling overhead is a problem, the Go team can likely improve it at some point, but they are less likely to magically map special cases of syscall.SYS_IOCTL to tcsetattr for the sake of portability.
    - 4ad 11y ago
      Not sure I understand your point, tcsetattr calls a system call on its own... So the alternatives are between doing a trivial system call on the Go stack, or doing a complex stack switch operation that does the (too complicated) System V calling convention ABI translation, and then calls a function that does a system call...
  - 4ad 11y ago
    It's not necessary to allocate a large stack, there is always one available because threads have their own stack, and you can switch to it at all times because goroutine threading is cooperative. In other words, if you are scheduled to run on a thread, you and only you have access to that thread stack. Stack switching doesn't take more than a few machine instructions. The problem with tcsetattr is that it's a library function, so you (generally[1]) need the target library in order to create a program that calls it. This is bad because in Go we value cross-compiling a lot, so we can't depend on having other people's shared libraries available in the build environment. Syscalls, however, we know how to do them and we know what they are, so we don't need any special target library in order to generate programs for the target. [1] On platforms like Solaris and Windows, where we are forced to use target shared libraries, we cheat a little and encode the target symbol names that we need, and we rely on standardized ELF/PE symbol resolution mechanism creating a scheme that's essentially equivalent to having import libraries available. But this is very complicated and takes a lot of code in the linker, so it's not at all unexpected that we avoid it whenever possible.
4ad 11y ago
> The reason Go doesn't link against system libraries is that Go in binary-incompatible with C, as in it's calling convention is different. I'm sorry, but the system call "calling convention" is different too. This argument doesn't hold water. Calling convention translation is a trivial issue. Go doesn't usually use system libraries in order to minimise dependencies and to allow easy cross-compiling. > There is a way to call C through a shim but it's extremely slow Not really, it's actually pretty fast. Of course it will be always slower than just a regular function call, but it's sufficiently low-overhead that in most cases you can ignore the overhead. Of course there are still cases where you can't ignore the overhead. On systems like Windows and Solaris, the cgo mechanism is actually used to do back-end "system calls" all the time.
- smegel 11y ago
  > I'm sorry, but the system call "calling convention" is different too. Sufficiently different to not be a problem? As I understand it system calls don't usually rely on the stack whereas C calls do, and it is the stack mismatch between C and Go that is the issue. > but it's sufficiently low-overhead that in most cases you can ignore the overhead Maybe for user called function...but if every single system call had to swap out the stack surely that would have a pretty heavy impact...
  - geofft 11y ago
    The Go calling convention passes everything, in both directions, on the stack. Most platform-native ("C") calling conventions put as many things in registers as possible, and reuse one of the registers for the return value when possible. However, this is just a convention from one function to the next, and they can share the same stack; the caller of a C standard library function merely needs to know what to put on the stack and what to put in registers (and which registers to save). And in fact, the C calling convention is more efficient than Go's from the caller's point of view (fewer things have to go on the stack, some registers are promised not to be clobbered), so calling a C library function with the native calling convention is, on all reasonable platforms, a fine idea performance-wise. There is the problem of the stack itself. The C model envisions a large, contiguous stack, possibly with some kernel magic to grow the stack as you approach the end (by handling page faults). Go previously used "segmented stacks", essentially a linked list of heap-allocated buffers used as stack, where functions knew how to allocate and deallocate segments when necessary. C doesn't know how to do that, so a C call previously would have to allocate plenty of memory just in case the function wanted to use a bunch of stack space. Go no longer uses segmented stacks, so that's essentially not a concern. Go still does reallocate stacks, but does so by copying the entire stack, and the stack is basically never freed. So there might be a one-time performance penalty on the first C call (unless the compiler is smart enough to tell when a Goroutine might possibly call into C, and pre-allocate that space), or the first C call after most of the existing stack was used in Go code, but not after that. tl;dr: The C calling convention is strictly faster than the Go one (but more complicated), and in Go 1.3+ the existing stack is fine to use, so there shouldn't be performance concerns. (Unless, of course, you think the C library itself is slow. But that's a very different problem.)

by smegel 11y ago

readams 11y ago

Static linking is independent of this concern. Go could implement dynamic linking but doesn't for ideological reasons. I'd really hate to be a distribution maintainer in a world where go components are common. Security bug in the TLS library? Time to download 14GB of updates!

nulltype 11y ago
Just download the deltas instead?
xyproto 11y ago
Not ideological reasons. The latest version of Go supports dynamic linking for some platforms.
- pjmlp 11y ago
  On Android there is no way around it, as native code can only be accessed via dynamic linking. This is what some Go designers think of dynamic linking: http://harmful.cat-v.org/software/dynamic-linking/ http://harmful.cat-v.org/software/dynamic-linking/
  - shadowmint 11y ago
    Yeeesss... but ultimately, Go is also a pragmatic language. Dynamic linking is coming to Go, like it or not; because it's a feature that is needed in certain circumstances, and that has been acknowledged, even by the Go team.
  - GauntletWizard 11y ago
    And those reasons are clear and clear-cut. Dynamic linking is a net negative in most use cases, and certainly in all the use-cases that the Go developers are developing for, where reproducible builds are mandatory, code size is a tiny fraction of main memory, and code quality is very strictly maintained. When you've built a good development and deploy strategy, rebuilding and redistributing your binary is an everyday activity. There's no additional cost.
  - bahamat 11y ago
    That same web page lists Linux as harmful. http://harmful.cat-v.org/software/operating-systems/linux/ http://harmful.cat-v.org/software/operating-systems/linux/
marssaxman 11y ago
I'm happy to be an end user in a world where go components are becoming more common, because statically-linked executables are far less likely to break when I upgrade unrelated programs.

tedunangst 11y ago

I doubt go to C calling conventions are going to be the slowest part of calling tcsetattr(). Using ioctl() is being difficult for the sake of being difficult.

pcwalton 11y ago
> I doubt go to C calling conventions are going to be the slowest part of calling tcsetattr(). Well, by "C calling conventions" it really means "performing a stack switch to get out of M:N, allocating a big stack if necessary or taking a lock to check one out". It's quite a lot of overhead and I wouldn't be surprised if it dwarfs the cost of the SYSENTER/SYSEXIT pair in many cases.
- tedunangst 11y ago
  All of which is likely dwarfed by all the code that runs between SYSENTER and SYSEXIT. And how many freaking times do you need to call tcsetattr()? Even worst case, if calling overhead is a problem, the Go team can likely improve it at some point, but they are less likely to magically map special cases of syscall.SYS_IOCTL to tcsetattr for the sake of portability.
  - 4ad 11y ago
    Not sure I understand your point, tcsetattr calls a system call on its own... So the alternatives are between doing a trivial system call on the Go stack, or doing a complex stack switch operation that does the (too complicated) System V calling convention ABI translation, and then calls a function that does a system call...
- 4ad 11y ago
  It's not necessary to allocate a large stack, there is always one available because threads have their own stack, and you can switch to it at all times because goroutine threading is cooperative. In other words, if you are scheduled to run on a thread, you and only you have access to that thread stack. Stack switching doesn't take more than a few machine instructions. The problem with tcsetattr is that it's a library function, so you (generally[1]) need the target library in order to create a program that calls it. This is bad because in Go we value cross-compiling a lot, so we can't depend on having other people's shared libraries available in the build environment. Syscalls, however, we know how to do them and we know what they are, so we don't need any special target library in order to generate programs for the target. [1] On platforms like Solaris and Windows, where we are forced to use target shared libraries, we cheat a little and encode the target symbol names that we need, and we rely on standardized ELF/PE symbol resolution mechanism creating a scheme that's essentially equivalent to having import libraries available. But this is very complicated and takes a lot of code in the linker, so it's not at all unexpected that we avoid it whenever possible.

4ad 11y ago

> The reason Go doesn't link against system libraries is that Go in binary-incompatible with C, as in it's calling convention is different. I'm sorry, but the system call "calling convention" is different too. This argument doesn't hold water. Calling convention translation is a trivial issue. Go doesn't usually use system libraries in order to minimise dependencies and to allow easy cross-compiling. > There is a way to call C through a shim but it's extremely slow Not really, it's actually pretty fast. Of course it will be always slower than just a regular function call, but it's sufficiently low-overhead that in most cases you can ignore the overhead. Of course there are still cases where you can't ignore the overhead. On systems like Windows and Solaris, the cgo mechanism is actually used to do back-end "system calls" all the time.

smegel 11y ago
> I'm sorry, but the system call "calling convention" is different too. Sufficiently different to not be a problem? As I understand it system calls don't usually rely on the stack whereas C calls do, and it is the stack mismatch between C and Go that is the issue. > but it's sufficiently low-overhead that in most cases you can ignore the overhead Maybe for user called function...but if every single system call had to swap out the stack surely that would have a pretty heavy impact...
- geofft 11y ago
  The Go calling convention passes everything, in both directions, on the stack. Most platform-native ("C") calling conventions put as many things in registers as possible, and reuse one of the registers for the return value when possible. However, this is just a convention from one function to the next, and they can share the same stack; the caller of a C standard library function merely needs to know what to put on the stack and what to put in registers (and which registers to save). And in fact, the C calling convention is more efficient than Go's from the caller's point of view (fewer things have to go on the stack, some registers are promised not to be clobbered), so calling a C library function with the native calling convention is, on all reasonable platforms, a fine idea performance-wise. There is the problem of the stack itself. The C model envisions a large, contiguous stack, possibly with some kernel magic to grow the stack as you approach the end (by handling page faults). Go previously used "segmented stacks", essentially a linked list of heap-allocated buffers used as stack, where functions knew how to allocate and deallocate segments when necessary. C doesn't know how to do that, so a C call previously would have to allocate plenty of memory just in case the function wanted to use a bunch of stack space. Go no longer uses segmented stacks, so that's essentially not a concern. Go still does reallocate stacks, but does so by copying the entire stack, and the stack is basically never freed. So there might be a one-time performance penalty on the first C call (unless the compiler is smart enough to tell when a Goroutine might possibly call into C, and pre-allocate that space), or the first C call after most of the existing stack was used in Go code, but not after that. tl;dr: The C calling convention is strictly faster than the Go one (but more complicated), and in Go 1.3+ the existing stack is fine to use, so there shouldn't be performance concerns. (Unless, of course, you think the C library itself is slow. But that's a very different problem.)