9 ms·

Golang on ARM

105 points by selvan 11y ago

bigdubs 11y ago
Maybe I'm in the minority but I would rather have resources devoted to improving compile times in 1.5+ versions vs. having golang work on arm. Isn't the best course usually to build a solid foundation, then branch out?
- vessenes 11y ago
  I, too, would like compile and lint times faster, ideally a lot faster. That said, the counterpoint is that cross-compilation is not something you can easily bolt on later -- if you aren't doing, and testing it along the way, you will never be able to add it back in. It needs to be there from the beginning and stay a first-class support goal, otherwise mistakes, good intentions and laziness will all yield a product that just can't easily make the leap back to solid cross compilation with no hassle.
- api 11y ago
  ARM is really important. It's probably the fastest growing architecture other than x86 where people use things like this. I'd also like to see Go on MIPS64, though probably less important than ARM.
  - 4ad 11y ago
    Both MIPS64, MIPS, and SPARC64 are in the works (some MIPS bits already merged in).
- tptacek 11y ago
  Really? I know 1.5 is slower than previous versions, but it's still by far the fastest compiler I've ever worked with.
  - bigdubs 11y ago
    In the global scheme of things it's still pretty fast, but builds on 1.5 take 2x the time of 1.4, it's pretty bad for existing golang users.
    - nickysielicki 11y ago
      Why is that a problem for you? With golint / gocode I never have any need for extremely fast compilation. What's your setup like that you're rebuilding so often?
      - vox_mollis 11y ago
        Personally, my biggest beef would be that the go developers specifically stood against generics on account of compiler complexity => slow compiling. It seems they accepted this sacrifice for some other unimportant feature* *edit: Looks as though the feature they sacrificed performance for was writing Go in Go.
        tptacek 11y ago
        Compilation in Go remains lightning fast, again, faster than any other compiler I use; fast enough that it's sane to run the compiler every time I save a file so my editor can highlight the errors.
        Locke1689 11y ago
        I think there are two schools of thought: 1) Perf for a product should be marked against some shipping benchmark and intermediate regressions don't matter as long as we come below the benchmark at ship time. 2) Perf improvements should be "locked in" and not regressed, unless strenuously justified. I think the parent is of group (2), where it doesn't necessarily matter that it's better than the benchmark of "against other compilers" but only that Go regressed from their earlier perf metrics for what they believe to be inadequate justification. I think both groups have valid points. I think (2) is especially good at preventing "perf creep" wherein you continually justify small perf regressions for new features, explaining it away that competitors are doing the same. Eventually, you get to the point where everyone's just slow, but they have no impetus to improve because they're still better than the competition.
- binarycrusader 11y ago
  Many of the contributions for the ports aren't from the Go team themselves, but from external contributors. Also, work is in progress to improve build times, although I don't know if that's an active focus, rather than just something that's being fixed as time permits. A few improvements have already gone back to the current development gate for a variety of cases. I think it's important to remember that development can be done in parallel, just because you see work being done on an area that isn't important to you doesn't mean that there isn't work being done on the area that's important to you as well. Some of these changes take a long time.
- diggan 11y ago
  > Isn't the best course usually to build a solid foundation, then branch out? Yes, and I think that is what they are doing. First build something stable. Then target a lot of platforms. Then optimize the language itself and also the build times.
vessenes 11y ago
It's incredible to run buildall.sh on a recent go repo and see all the cross-compilers get built with no fuss or hassle. There are at least 30 different target platforms I can get to from my ubuntu box with a single command line. I'm not aware of anything that even comes close to tooling out cross compilation so easily.
- bch 11y ago
  It's called NetBSD. build.sh cross compiles one codebase for 57(+)[0] architectures. Even for a "native" build, the tooling is the same abstraction that builds for a completely foreign target, so it's continuously well-exercised. [0] http://www.netbsd.org/ports/ http://www.netbsd.org/ports/
- bbrazil 11y ago
  I do nightly binaries for Prometheus in all the possible archs, while things should compile perfectly there can be small things that prevent things from cross-compiling. darwin/arm, darwin/arm64, plan9/386, plan9/amd64 and solaris/amd64 don't compile for any of the 11 binaries (e.g. plan9 doesn't work due to logrus not having support for it). This leaves me with the list at http://www.robustperception.io/prometheus-nightly-binaries/ http://www.robustperception.io/prometheus-nightly-binaries/ That's still 20 architectures I can compile almost everything to automatically with an ansible+jenkins setup.
  - 4ad 11y ago
    What is the problem with solaris/amd64?
    - bbrazil 11y ago
      ../../Sirupsen/logrus/text_formatter.go:28: undefined: IsTerminal The logrus library we use for logging doesn't support Solaris.
      - threemux 11y ago
        Looks like it does now: https://github.com/Sirupsen/logrus/blob/master/terminal_solaris.go https://github.com/Sirupsen/logrus/blob/master/terminal_sola...
ihsw 11y ago
This is for both development on ARM systems (standard go commands, eg: go build, etc) and deploying to ARM systems (setting ARM as a compile target). That said, I'm genuinely curious why anyone would develop on an ARM-based system as opposed to your run of the mill desktop. I realize that standard towers may be supplanted by their SoC brethren, but so what? Will we all be using iMac-like computers as desktops in the next 36-72 months? Will there even be any differentiation between "regular" desktops with discrete CPUs, memory, etc and the system-on-a-chip variant?
- detaro 11y ago
  Not everybody compiles all binaries they deploy on developer workstations. If you run ARM servers, it makes sense that your build servers are on ARM as well, especially since you should run tests on the same architecture you're deploying to.
  - ersoft 11y ago
    Well, when your build takes 1min on ARM and 3 seconds on your laptop, it makes sense.
- laurent123456 11y ago
  I run a Go program on a Raspberry Pi device, and it's actually very convenient to be able to quickly edit a Go file, and test with `go run` directly on the machine.
  - GauntletWizard 11y ago
    I also run go on RPi, and compiled either there or on a qemu layer because the support for cgo cross-compilation for ARM was lacking. This was back in 1.3; It may be better now.
    - laurent123456 11y ago
      cgo support seems fine, at least for my use. I've compiled and use the go-sqlite3 package without any problem.
vferreira 11y ago
How about the support by the LLVM. Shouldn't this be enough?
- kyrra 11y ago
  Go already has 2 compiler implementations. There is gc (Go Compiler) and gccgo[0]. [0] https://golang.org/doc/install/gccgo https://golang.org/doc/install/gccgo
  - frame_perfect 11y ago
    But LLVM already has a lot of good backends (emscripten, arm, ppc, x86, amd64). Why reinvent the wheel when you could just make Go a LLVM frontend and be done with it?
    - bassislife 11y ago
      Compile times ? Flexibility over the backend ? That would be my hunch. I think it makes a lot of sense. Also, note that there is that project which exists : http://llvm.org/viewvc/llvm-project/llgo/ http://llvm.org/viewvc/llvm-project/llgo/
      - pcwalton 11y ago
        > Compile times ? There can be a compile time hit, but you can use FastISel, which is not bad. The compile time hit is mitigated by the fact that you get a far, far more mature series of optimization passes. Since the Go compiler is itself written in Go, more optimizations would make the Go compiler itself faster, further reducing the performance impact. > Flexibility over the backend ? Writing to LLVM in no way restricts you to a backend. You simply need a language-level IR that you can retarget to different backends as you would like. This is what Swift, for example, is doing with SIL.
        bassislife 11y ago
        > Writing to LLVM in no way restricts you to a backend. You simply need a language-level IR that you can retarget to different backends as you would like. It still ignores toolchain flexibility. Especially given the specific needs of Go's managed runtime.
dekhn 11y ago
One of the funniest experiences I had was building an ARM go binary on an x86-64 linux box using a cross compiler (nothing new there). Then, by accident I tried to run the ARM binary on my x86-64 box, instead of on the ARM box. It ran like a normal executable, which confused me until I realized I had installed QEMU-arm, which installed a linux binary format handler for ARM binaries, and it emulated ARM to run my binary. Of course, copied over to the Raspi, it ran fine there too.
- biomcgary 11y ago
  Did you benchmark performance on QEMU vs bare metal?
IshKebab 11y ago
This is one of Go's best features for me. You can trivially cross-compile statically linked binaries, copy them over to an ARM system and they run fine. Contrast that with C/C++ - sure you can cross-compile, and I have in the past, but it's a huge pain to set up and who ever remembers the command line options. Those "target triples" aren't even documented anywhere. Plus, good luck building a statically linked C++ program. Lua might be a decent option too.
- frame_perfect 11y ago
  What about LLVM? Go's support of cross compilation is laughable in comparison to the different LLVM backends. And you can just use the -arch armv7 and -arch arm64 shortcuts instead of those target triples. Also, FWIW, the reason why Go can even be cross compiled in the first place is because it has platform specific C and assembly code from cgo's runtime so it can support the different architectures/platforms. Of course, if Go was just an LLVM frontend instead of using gcc then it wouldn't even need the platform specific C code... And it would support a shitton more platforms like asm.js. But I guess Google just hates Apple that much?
  - pcwalton 11y ago
    > But I guess Google just hates Apple that much? This is silly. Google contributes a huge amount to LLVM. They're very invested in the project. (NB: I'm not defending, and don't agree with, the choice to avoid LLVM in Go.)
  - deleted 11y ago
    [deleted]
  - arthurbrown 11y ago
    Russ Cox wrote a great comment[1] on HN about why using the Plan9 toolchain rather LLVM or even GCC was a competitive advantage for them in terms of speed to delivery, control, and iteration. "Most importantly, it has been something we understand completely and is quite easy to adapt as needed. The standard ABIs and toolchains are for C, and the decisions made there may be completely inappropriate for languages with actual runtimes. For example, no standard ABIs and toolchains supported segmented stacks; we had to build that, so it was going to be incompatible from day one. If step one had been "learn the GCC or LLVM toolchains well enough to add segmented stacks", I'm not sure we'd have gotten to step two." "... the custom toolchain is one of the key reasons we've accomplished so much in so little time. I think the author doesn't fully appreciate all the reasons that C tools don't work well for languages that do so much more than C. Most other managed languages are doing JIT compilation and don't go through standard linkers either." [1] https://news.ycombinator.com/item?id=8817990 https://news.ycombinator.com/item?id=8817990
Locke1689 11y ago
Cool! What's the Go memory model look like? Is it stricter than ARM or could possibly working programs be broken when run there?
- 4ad 11y ago
  https://golang.org/ref/mem https://golang.org/ref/mem Btw, in case it's not clear, Go has worked on ARM for the last 6 years, this is not a new thing. It is the job of the compiler and runtime writer to ensure this model on whatever target platform. It doesn't matter if the target platform has a strong or weak memory model. Correct Go programs written for the Go memory model should work correctly on any target. Of course that it's possible to write Go programs that violate the memory model but work on amd64, and not on some other platforms. But those are incorrect Go programs (and the race detector has a very good chance of detecting these problems). I will note that it's significantly harder to write these programs accidentally in Go though, than it is to write them in C.

105 points by selvan 11y ago

Maybe I'm in the minority but I would rather have resources devoted to improving compile times in 1.5+ versions vs. having golang work on arm. Isn't the best course usually to build a solid foundation, then branch out?

vessenes 11y ago
I, too, would like compile and lint times faster, ideally a lot faster. That said, the counterpoint is that cross-compilation is not something you can easily bolt on later -- if you aren't doing, and testing it along the way, you will never be able to add it back in. It needs to be there from the beginning and stay a first-class support goal, otherwise mistakes, good intentions and laziness will all yield a product that just can't easily make the leap back to solid cross compilation with no hassle.
api 11y ago
ARM is really important. It's probably the fastest growing architecture other than x86 where people use things like this. I'd also like to see Go on MIPS64, though probably less important than ARM.
- 4ad 11y ago
  Both MIPS64, MIPS, and SPARC64 are in the works (some MIPS bits already merged in).
tptacek 11y ago
Really? I know 1.5 is slower than previous versions, but it's still by far the fastest compiler I've ever worked with.
- bigdubs 11y ago
  In the global scheme of things it's still pretty fast, but builds on 1.5 take 2x the time of 1.4, it's pretty bad for existing golang users.
  - nickysielicki 11y ago
    Why is that a problem for you? With golint / gocode I never have any need for extremely fast compilation. What's your setup like that you're rebuilding so often?
    - vox_mollis 11y ago
      Personally, my biggest beef would be that the go developers specifically stood against generics on account of compiler complexity => slow compiling. It seems they accepted this sacrifice for some other unimportant feature* *edit: Looks as though the feature they sacrificed performance for was writing Go in Go.
      - tptacek 11y ago
        Compilation in Go remains lightning fast, again, faster than any other compiler I use; fast enough that it's sane to run the compiler every time I save a file so my editor can highlight the errors.
        Locke1689 11y ago
        I think there are two schools of thought: 1) Perf for a product should be marked against some shipping benchmark and intermediate regressions don't matter as long as we come below the benchmark at ship time. 2) Perf improvements should be "locked in" and not regressed, unless strenuously justified. I think the parent is of group (2), where it doesn't necessarily matter that it's better than the benchmark of "against other compilers" but only that Go regressed from their earlier perf metrics for what they believe to be inadequate justification. I think both groups have valid points. I think (2) is especially good at preventing "perf creep" wherein you continually justify small perf regressions for new features, explaining it away that competitors are doing the same. Eventually, you get to the point where everyone's just slow, but they have no impetus to improve because they're still better than the competition.
binarycrusader 11y ago
Many of the contributions for the ports aren't from the Go team themselves, but from external contributors. Also, work is in progress to improve build times, although I don't know if that's an active focus, rather than just something that's being fixed as time permits. A few improvements have already gone back to the current development gate for a variety of cases. I think it's important to remember that development can be done in parallel, just because you see work being done on an area that isn't important to you doesn't mean that there isn't work being done on the area that's important to you as well. Some of these changes take a long time.
diggan 11y ago
> Isn't the best course usually to build a solid foundation, then branch out? Yes, and I think that is what they are doing. First build something stable. Then target a lot of platforms. Then optimize the language itself and also the build times.

vessenes 11y ago

It's incredible to run buildall.sh on a recent go repo and see all the cross-compilers get built with no fuss or hassle. There are at least 30 different target platforms I can get to from my ubuntu box with a single command line. I'm not aware of anything that even comes close to tooling out cross compilation so easily.

bch 11y ago
It's called NetBSD. build.sh cross compiles one codebase for 57(+)[0] architectures. Even for a "native" build, the tooling is the same abstraction that builds for a completely foreign target, so it's continuously well-exercised. [0] http://www.netbsd.org/ports/ http://www.netbsd.org/ports/
bbrazil 11y ago
I do nightly binaries for Prometheus in all the possible archs, while things should compile perfectly there can be small things that prevent things from cross-compiling. darwin/arm, darwin/arm64, plan9/386, plan9/amd64 and solaris/amd64 don't compile for any of the 11 binaries (e.g. plan9 doesn't work due to logrus not having support for it). This leaves me with the list at http://www.robustperception.io/prometheus-nightly-binaries/ http://www.robustperception.io/prometheus-nightly-binaries/ That's still 20 architectures I can compile almost everything to automatically with an ansible+jenkins setup.
- 4ad 11y ago
  What is the problem with solaris/amd64?
  - bbrazil 11y ago
    ../../Sirupsen/logrus/text_formatter.go:28: undefined: IsTerminal The logrus library we use for logging doesn't support Solaris.
    - threemux 11y ago
      Looks like it does now: https://github.com/Sirupsen/logrus/blob/master/terminal_solaris.go https://github.com/Sirupsen/logrus/blob/master/terminal_sola...

ihsw 11y ago

This is for both development on ARM systems (standard go commands, eg: go build, etc) and deploying to ARM systems (setting ARM as a compile target). That said, I'm genuinely curious why anyone would develop on an ARM-based system as opposed to your run of the mill desktop. I realize that standard towers may be supplanted by their SoC brethren, but so what? Will we all be using iMac-like computers as desktops in the next 36-72 months? Will there even be any differentiation between "regular" desktops with discrete CPUs, memory, etc and the system-on-a-chip variant?

detaro 11y ago
Not everybody compiles all binaries they deploy on developer workstations. If you run ARM servers, it makes sense that your build servers are on ARM as well, especially since you should run tests on the same architecture you're deploying to.
- ersoft 11y ago
  Well, when your build takes 1min on ARM and 3 seconds on your laptop, it makes sense.
laurent123456 11y ago
I run a Go program on a Raspberry Pi device, and it's actually very convenient to be able to quickly edit a Go file, and test with `go run` directly on the machine.
- GauntletWizard 11y ago
  I also run go on RPi, and compiled either there or on a qemu layer because the support for cgo cross-compilation for ARM was lacking. This was back in 1.3; It may be better now.
  - laurent123456 11y ago
    cgo support seems fine, at least for my use. I've compiled and use the go-sqlite3 package without any problem.

vferreira 11y ago

How about the support by the LLVM. Shouldn't this be enough?

kyrra 11y ago
Go already has 2 compiler implementations. There is gc (Go Compiler) and gccgo[0]. [0] https://golang.org/doc/install/gccgo https://golang.org/doc/install/gccgo
- frame_perfect 11y ago
  But LLVM already has a lot of good backends (emscripten, arm, ppc, x86, amd64). Why reinvent the wheel when you could just make Go a LLVM frontend and be done with it?
  - bassislife 11y ago
    Compile times ? Flexibility over the backend ? That would be my hunch. I think it makes a lot of sense. Also, note that there is that project which exists : http://llvm.org/viewvc/llvm-project/llgo/ http://llvm.org/viewvc/llvm-project/llgo/
    - pcwalton 11y ago
      > Compile times ? There can be a compile time hit, but you can use FastISel, which is not bad. The compile time hit is mitigated by the fact that you get a far, far more mature series of optimization passes. Since the Go compiler is itself written in Go, more optimizations would make the Go compiler itself faster, further reducing the performance impact. > Flexibility over the backend ? Writing to LLVM in no way restricts you to a backend. You simply need a language-level IR that you can retarget to different backends as you would like. This is what Swift, for example, is doing with SIL.
      - bassislife 11y ago
        > Writing to LLVM in no way restricts you to a backend. You simply need a language-level IR that you can retarget to different backends as you would like. It still ignores toolchain flexibility. Especially given the specific needs of Go's managed runtime.

dekhn 11y ago

One of the funniest experiences I had was building an ARM go binary on an x86-64 linux box using a cross compiler (nothing new there). Then, by accident I tried to run the ARM binary on my x86-64 box, instead of on the ARM box. It ran like a normal executable, which confused me until I realized I had installed QEMU-arm, which installed a linux binary format handler for ARM binaries, and it emulated ARM to run my binary. Of course, copied over to the Raspi, it ran fine there too.

biomcgary 11y ago
Did you benchmark performance on QEMU vs bare metal?

IshKebab 11y ago

This is one of Go's best features for me. You can trivially cross-compile statically linked binaries, copy them over to an ARM system and they run fine. Contrast that with C/C++ - sure you can cross-compile, and I have in the past, but it's a huge pain to set up and who ever remembers the command line options. Those "target triples" aren't even documented anywhere. Plus, good luck building a statically linked C++ program. Lua might be a decent option too.

frame_perfect 11y ago
What about LLVM? Go's support of cross compilation is laughable in comparison to the different LLVM backends. And you can just use the -arch armv7 and -arch arm64 shortcuts instead of those target triples. Also, FWIW, the reason why Go can even be cross compiled in the first place is because it has platform specific C and assembly code from cgo's runtime so it can support the different architectures/platforms. Of course, if Go was just an LLVM frontend instead of using gcc then it wouldn't even need the platform specific C code... And it would support a shitton more platforms like asm.js. But I guess Google just hates Apple that much?
- pcwalton 11y ago
  > But I guess Google just hates Apple that much? This is silly. Google contributes a huge amount to LLVM. They're very invested in the project. (NB: I'm not defending, and don't agree with, the choice to avoid LLVM in Go.)
- deleted 11y ago
  [deleted]
- arthurbrown 11y ago
  Russ Cox wrote a great comment[1] on HN about why using the Plan9 toolchain rather LLVM or even GCC was a competitive advantage for them in terms of speed to delivery, control, and iteration. "Most importantly, it has been something we understand completely and is quite easy to adapt as needed. The standard ABIs and toolchains are for C, and the decisions made there may be completely inappropriate for languages with actual runtimes. For example, no standard ABIs and toolchains supported segmented stacks; we had to build that, so it was going to be incompatible from day one. If step one had been "learn the GCC or LLVM toolchains well enough to add segmented stacks", I'm not sure we'd have gotten to step two." "... the custom toolchain is one of the key reasons we've accomplished so much in so little time. I think the author doesn't fully appreciate all the reasons that C tools don't work well for languages that do so much more than C. Most other managed languages are doing JIT compilation and don't go through standard linkers either." [1] https://news.ycombinator.com/item?id=8817990 https://news.ycombinator.com/item?id=8817990

Locke1689 11y ago

Cool! What's the Go memory model look like? Is it stricter than ARM or could possibly working programs be broken when run there?

4ad 11y ago
https://golang.org/ref/mem https://golang.org/ref/mem Btw, in case it's not clear, Go has worked on ARM for the last 6 years, this is not a new thing. It is the job of the compiler and runtime writer to ensure this model on whatever target platform. It doesn't matter if the target platform has a strong or weak memory model. Correct Go programs written for the Go memory model should work correctly on any target. Of course that it's possible to write Go programs that violate the memory model but work on amd64, and not on some other platforms. But those are incorrect Go programs (and the race detector has a very good chance of detecting these problems). I will note that it's significantly harder to write these programs accidentally in Go though, than it is to write them in C.