7 ms·

On Go, Portability, and System Interfaces (2015)

99 points by Annatar 8y ago

coldtea 8y ago
>How did we wind up in this ugly situation? The problem I believe stems from some misconceptions, and some historical precedents in the Go community. First the Go community has long touted static linking as one of its significant advantages. However, I believe this has been taken too far. How about people just go out to solve their problems, in their own platforms, and don't care to support all platforms? It's not like most of those Go projects are big in the first place, and one or a couple of contributors don't have the means to test other platforms much, or the time to write it platform-agnostically.
- fndrplayer13 8y ago
  I think this is likely the reality. I don't think this is necessarily a community problem so much as it is the reality of software development today. Most developers given the time and resources would probably enjoy making their libraries and applications as portable and flexible as possible -- who doesn't love to see their work reused? That being said, on a typical software delivery cycle you optimize for what you think _most_ people are using, and likely what you yourself are using -- standard flavors of Linux operating systems running in one of the big cloud providers. This is not to say Go doesnt have applications outside of this space -- it clearly does.
  - scarface74 8y ago
    The only Go apps that I’ve knowingly used are from Hashicorp - Nomad, Consul and Vault and they were all running on Windows. I found it quite refreshing that the “install” process was copying one relatively small executable. That being said, Terraform is quite popular and I would think that it’s used by Windows admins/devops locally.
    - jniedrauer 8y ago
      You've also probably used docker, which is written in Go.
      - scarface74 8y ago
        Never used Docker. That was the advantage of using Nomad. It can orchestrate anything - Docker containers, executables, Shell scripts, etc.
        monocasa 8y ago
        I mean, afaict nomad uses docker to orchestrate docker containers...
- cwyers 8y ago
  It's not like the author is asking for people to write a bunch of whatever Go's equivalent of IFDEF is. The author's proposed alternative code is just as concise and straightforward, it just has the benefit of using something that's portable across multiple platforms.
- cube2222 8y ago
  However, this created a great ecosystem where everything is usually cross-platform by default. With a lot of projects being developed in Go in the cloud ecosystem, developers can more freely choose the platform they want to develop with. Same story with tools written in rust.
- gok 8y ago
  If the goal was "Go is a language for Linux", fine. But that wasn't the goal. From the first release it was targeted to macOS, Windows and other platforms, but implemented in a way that didn't make any sense on those platforms.
  - pjmlp 8y ago
    Actually Go was implemented as OS X and Linux language initially. They left up to the community to build up Windows support. Certain packages are still not available on Windows, like plugins for example. Easy to find that out searching early gonuts posts.
    - gok 8y ago
      Ok I stand corrected; they only botched it on 50% of their initial targets :)
- pcwalton 8y ago
  Because that's doing a disservice to potential users? If the Go culture doesn't favor writing portable code, that's absolutely a problem with Go's culture.
cdoxsey 8y ago
> Basically, the Go folks want to minimize external dependencies and the web of failure that can lead to. Fixing that is a goal I heartily agree with. However, we cannot eliminate our dependency on the platform. And using system calls directly is actually worse, because it moves our dependency from something that is stable and defined by standards bodies, to an interface that is undocumented, not portable, and may change at any time. It's also for performance reasons. Calling a C function from Go is actually quite complicated and can involve copying a lot of data. Using a syscall directly avoids that cost.
- gok 8y ago
  That has got to be a total drop in the bucket compared the cost of the syscall itself.
  - valarauca1 8y ago
    Kernel interrupt handling is incredibly quick. Tasking switching an the related MMU updates is slower. As you have to perform a kernel interrupt handling action just to start initializing the task switch.
    - majewsky 8y ago
      Interrupt handling is quick, paging isn't.
      - monocasa 8y ago
        The KPTI Meltdown fixes mean that interrupt handling _is_ paging on a lot of chips out there.
    - amluto 8y ago
      Careful. x86 interrupt handling is slow. System calls use an accelerated mechanism that is much faster.
  - deleted 8y ago
    [deleted]
masklinn 8y ago
> The problem I believe stems from some misconceptions, and some historical precedents in the Go community. It's not just "historical precedents in the Go community" it's "ongoing in Go itself": Go's developers will try as hard and as long as they can to perform raw syscalls even on platforms where it's not officially supported, and even on the one platform where it is genuinely actually supported they'll do raw vDSO calls (which are not) which predictably blows up (https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/ https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/). [0] which is pretty much all of them aside from Linux, Golang 1.11 finally stops performing raw syscalls on OSX
- nhooyr 8y ago
  It uses ago for system calls on macOS and linux?
  - masklinn 8y ago
    I don't know what you're asking.
    - telotortium 8y ago
      I presume cgo?
- amluto 8y ago
  Raw vDSO calls are absolutely supported. The thread you linked is about a Linux kernel that was miscompiled due to a demented Gentoo-hardened toolchain.
- Immortalin 8y ago
  Other than buffering, why are libc calls faster than system calls?
  - masklinn 8y ago
    libc calls are not faster than system calls.
lelf 8y ago
Not to mention systems without syscalls [as concept] at all. Simple example: GNU/Hurd’s glibc works quite differently.
twic 8y ago
Is there an official documented rationale for why the Go standard library doesn't go use libc, and instead does all this direct syscall stuff? I've heard various stories about it over the years, so i would appreciate an authoritative source.
- steveklabnik 8y ago
  My understanding, which is certainly not authoritative, is that Go wants to be 100% statically linked. libc is usually dynamically linked. MUSL is Linux only.
- oconnor663 8y ago
  It often does use libc. OP's article mentioned that they're using it more on Darwin now, but they've also been using it on Linux for a long time to do some of their DNS resolution (where implementing a stable parser for /etc/resolv.conf and friends is nobody's idea of a good time). I don't have an authoritative source, but I think one of the big downsides of CGO for them is that it hurts their cross-compilation story. Cross-compiling a pure Go program is trivial, which is great, but by default it disables CGO.
  - cube2222 8y ago
    Exactly, it disables CGO. So if you cross compile Windows -> Linux the pure Go DNS resolution code gets used.
- valarauca1 8y ago
  why the Go standard library doesn't go use libc They do you use a `libc` they use `musl-libc` [1] as apposed to `gnu-libc` (gblic) [2]. There are some differences [3], the largest (without getting into various API incompatibilities) is licensing. `musl-libc` is BSD licensed, while `gnu-libc` (glibc) is GPL licensed. The parent post does explain, about as well as any member of the go-team has to date for their usage of `musl-libc` over `gnu-libc` (glibc). Overall `musl-libc` does focus on correctness a lot more then `glibc`, and static linking is a large benefit. Static linking isn't an issue when you expect applications to be short lived transient items. Where system images only exist for days, or hours before being redeployed by either patches, or CI-Managered triggered updates. I think this is the largest disconnect. Go expects the development pipeline to be quick so nobody expects multi-year lifespan binaries, which is directly contradictory to how many expect executable binaries to behave. No that I'm authority on this, just stating a common disconnect I've experienced working with ex-Googlers. [1] https://www.musl-libc.org/ https://www.musl-libc.org/ [2] https://www.gnu.org/software/libc/ https://www.gnu.org/software/libc/ [3] https://wiki.musl-libc.org/functional-differences-from-glibc.html https://wiki.musl-libc.org/functional-differences-from-glibc... [4] https://github.com/golang/go/issues/9627 https://github.com/golang/go/issues/9627

99 points by Annatar 8y ago

>How did we wind up in this ugly situation? The problem I believe stems from some misconceptions, and some historical precedents in the Go community. First the Go community has long touted static linking as one of its significant advantages. However, I believe this has been taken too far. How about people just go out to solve their problems, in their own platforms, and don't care to support all platforms? It's not like most of those Go projects are big in the first place, and one or a couple of contributors don't have the means to test other platforms much, or the time to write it platform-agnostically.

fndrplayer13 8y ago
I think this is likely the reality. I don't think this is necessarily a community problem so much as it is the reality of software development today. Most developers given the time and resources would probably enjoy making their libraries and applications as portable and flexible as possible -- who doesn't love to see their work reused? That being said, on a typical software delivery cycle you optimize for what you think _most_ people are using, and likely what you yourself are using -- standard flavors of Linux operating systems running in one of the big cloud providers. This is not to say Go doesnt have applications outside of this space -- it clearly does.
- scarface74 8y ago
  The only Go apps that I’ve knowingly used are from Hashicorp - Nomad, Consul and Vault and they were all running on Windows. I found it quite refreshing that the “install” process was copying one relatively small executable. That being said, Terraform is quite popular and I would think that it’s used by Windows admins/devops locally.
  - jniedrauer 8y ago
    You've also probably used docker, which is written in Go.
    - scarface74 8y ago
      Never used Docker. That was the advantage of using Nomad. It can orchestrate anything - Docker containers, executables, Shell scripts, etc.
      - monocasa 8y ago
        I mean, afaict nomad uses docker to orchestrate docker containers...
cwyers 8y ago
It's not like the author is asking for people to write a bunch of whatever Go's equivalent of IFDEF is. The author's proposed alternative code is just as concise and straightforward, it just has the benefit of using something that's portable across multiple platforms.
cube2222 8y ago
However, this created a great ecosystem where everything is usually cross-platform by default. With a lot of projects being developed in Go in the cloud ecosystem, developers can more freely choose the platform they want to develop with. Same story with tools written in rust.
gok 8y ago
If the goal was "Go is a language for Linux", fine. But that wasn't the goal. From the first release it was targeted to macOS, Windows and other platforms, but implemented in a way that didn't make any sense on those platforms.
- pjmlp 8y ago
  Actually Go was implemented as OS X and Linux language initially. They left up to the community to build up Windows support. Certain packages are still not available on Windows, like plugins for example. Easy to find that out searching early gonuts posts.
  - gok 8y ago
    Ok I stand corrected; they only botched it on 50% of their initial targets :)
pcwalton 8y ago
Because that's doing a disservice to potential users? If the Go culture doesn't favor writing portable code, that's absolutely a problem with Go's culture.

cdoxsey 8y ago

> Basically, the Go folks want to minimize external dependencies and the web of failure that can lead to. Fixing that is a goal I heartily agree with. However, we cannot eliminate our dependency on the platform. And using system calls directly is actually worse, because it moves our dependency from something that is stable and defined by standards bodies, to an interface that is undocumented, not portable, and may change at any time. It's also for performance reasons. Calling a C function from Go is actually quite complicated and can involve copying a lot of data. Using a syscall directly avoids that cost.

gok 8y ago
That has got to be a total drop in the bucket compared the cost of the syscall itself.
- valarauca1 8y ago
  Kernel interrupt handling is incredibly quick. Tasking switching an the related MMU updates is slower. As you have to perform a kernel interrupt handling action just to start initializing the task switch.
  - majewsky 8y ago
    Interrupt handling is quick, paging isn't.
    - monocasa 8y ago
      The KPTI Meltdown fixes mean that interrupt handling _is_ paging on a lot of chips out there.
  - amluto 8y ago
    Careful. x86 interrupt handling is slow. System calls use an accelerated mechanism that is much faster.
- deleted 8y ago
  [deleted]

masklinn 8y ago

> The problem I believe stems from some misconceptions, and some historical precedents in the Go community. It's not just "historical precedents in the Go community" it's "ongoing in Go itself": Go's developers will try as hard and as long as they can to perform raw syscalls even on platforms where it's not officially supported, and even on the one platform where it is genuinely actually supported they'll do raw vDSO calls (which are not) which predictably blows up (https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/ https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/). [0] which is pretty much all of them aside from Linux, Golang 1.11 finally stops performing raw syscalls on OSX

nhooyr 8y ago
It uses ago for system calls on macOS and linux?
- masklinn 8y ago
  I don't know what you're asking.
  - telotortium 8y ago
    I presume cgo?
amluto 8y ago
Raw vDSO calls are absolutely supported. The thread you linked is about a Linux kernel that was miscompiled due to a demented Gentoo-hardened toolchain.
Immortalin 8y ago
Other than buffering, why are libc calls faster than system calls?
- masklinn 8y ago
  libc calls are not faster than system calls.

lelf 8y ago

Not to mention systems without syscalls [as concept] at all. Simple example: GNU/Hurd’s glibc works quite differently.

twic 8y ago

Is there an official documented rationale for why the Go standard library doesn't go use libc, and instead does all this direct syscall stuff? I've heard various stories about it over the years, so i would appreciate an authoritative source.

steveklabnik 8y ago
My understanding, which is certainly not authoritative, is that Go wants to be 100% statically linked. libc is usually dynamically linked. MUSL is Linux only.
oconnor663 8y ago
It often does use libc. OP's article mentioned that they're using it more on Darwin now, but they've also been using it on Linux for a long time to do some of their DNS resolution (where implementing a stable parser for /etc/resolv.conf and friends is nobody's idea of a good time). I don't have an authoritative source, but I think one of the big downsides of CGO for them is that it hurts their cross-compilation story. Cross-compiling a pure Go program is trivial, which is great, but by default it disables CGO.
- cube2222 8y ago
  Exactly, it disables CGO. So if you cross compile Windows -> Linux the pure Go DNS resolution code gets used.
valarauca1 8y ago
why the Go standard library doesn't go use libc They do you use a `libc` they use `musl-libc` [1] as apposed to `gnu-libc` (gblic) [2]. There are some differences [3], the largest (without getting into various API incompatibilities) is licensing. `musl-libc` is BSD licensed, while `gnu-libc` (glibc) is GPL licensed. The parent post does explain, about as well as any member of the go-team has to date for their usage of `musl-libc` over `gnu-libc` (glibc). Overall `musl-libc` does focus on correctness a lot more then `glibc`, and static linking is a large benefit. Static linking isn't an issue when you expect applications to be short lived transient items. Where system images only exist for days, or hours before being redeployed by either patches, or CI-Managered triggered updates. I think this is the largest disconnect. Go expects the development pipeline to be quick so nobody expects multi-year lifespan binaries, which is directly contradictory to how many expect executable binaries to behave. No that I'm authority on this, just stating a common disconnect I've experienced working with ex-Googlers. [1] https://www.musl-libc.org/ https://www.musl-libc.org/ [2] https://www.gnu.org/software/libc/ https://www.gnu.org/software/libc/ [3] https://wiki.musl-libc.org/functional-differences-from-glibc.html https://wiki.musl-libc.org/functional-differences-from-glibc... [4] https://github.com/golang/go/issues/9627 https://github.com/golang/go/issues/9627