4 ms·

> I'm kind of curious how much faster compiling a large Go codebase is. How fast, e.g. does the entirety of Kubernetes compile? I was curious too, so I just tr

by dilap 7y ago

> I'm kind of curious how much faster compiling a large Go codebase is. How fast, e.g. does the entirety of Kubernetes compile? I was curious too, so I just tried it. On my 6-core previous-gen MacBook pro, I get 2m18s. That's for SLoc = 3296486 (assuming all deps are vendored and non-deps are not-vendored, which I think is true), so about 16K SLoc/s. Or to put it another way, if Rust compiled as quickly as Go, we'd expect to compile a release build of TiKV in 2m.

amelius 7y ago
Can you see how long incremental builds take (under a minor change)?
- dilap 7y ago
  it was very slow, almost a minute! but it has a custom build script, that looks like it’s not really made for incremental builds? it looks like there’s also bazel support, which i would guess is much faster, but i couldn’t figure out how to get it to work...
dnpp123 7y ago
It would be interesting to have a `reverse` blog post "why Go compilation time is fast" e.g. which optimizations are overlooked/done quickly by the go compiler ? Could the binary quality be improved by increasing the compilation time ? AFAIK you don't have optimization flags in go (or the default one are optimum). And to be honnest, I have no idea what the compiled Go code looks like. As much as I have no idea what actual instructions are executed by the python interpreter.
- gwd 7y ago
  Well one big reason is the biggest feature people are complaining about: A lack of generics. According to [1], there's a fundamental trade-off with respect to generics: "The generic dilemma is this: do you want slow programmers, slow compilers and bloated binaries, or slow execution times?" At the moment, Go has no generics, which translates to "slow programmers", but not "slow compilers". It's likely that the simplicity of the Go language as a whole -- which leads to many complaints -- is a major factor in the fast compile times. EDIT: Lest it seem like I'm bashing Go here, the quote above is from one of the core Golang developers (in 2009 no less), and is mirrored in this from the proposal [2] from another core Golang developer. It's meant to be a shorthand for, "Generics do actually save programmer time and effort, and Go programmers are working harder than necessary because the language doesn't have generics yet." The point of this post is to point out that generics have a cost (slow compilation time) and that fast compilation time has a cost (no generics -- at least, not without inventing a new way of doing generics). It's possible to love something while still seeing its deficiencies and wishing its improvement. In fact, I'd argue that's the only way to truly love anyone or anything. [1] https://research.swtch.com/generic https://research.swtch.com/generic [2] https://go.googlesource.com/proposal/+/master/design/15292-generics.md https://go.googlesource.com/proposal/+/master/design/15292-g...
  - pmezard 7y ago
    > At the moment, Go has no generics, which translates to "slow programmers", but not "slow compilers". Also note that one of these assertions is measurable, and the other an unsubstantiated opinion.
  - pcwalton 7y ago
    What I don't like about the "generics dilemma" framing is that the problem exists regardless of whether your language has generics or not. Here's what I mean. Say you have a Vector class that can operate on ints or floats. You could make that a generic, in which case the compiler can either (a) duplicate the code for each type (monomorphize) or (b) do dictionary passing and get slower runtime. But if your language doesn't have generics, you have exactly the same problem: you as the programmer must (a) duplicate the code for ints and floats or (b) use an interface and get slower runtime. Not having generics doesn't solve anything. It just means that you, the programmer, have to do things that the compiler would otherwise do for you.
    - Twisol 7y ago
      Although, one interesting aspect of the trade-off here is that a programmer who manually monomorphizes only has to do so once(^) -- their effort is reused across multiple compilation runs -- whereas the compiler normally has to monomorphize on each compilation run. (^) Of course, you then have the burden of keeping multiple monomorphized implementations consistent when you make a change that needs to apply to all of them.
      - eddyb 7y ago
        Monomorphization tends to happen "on the fly", and even if it wasn't, it would still be cheaper than duplicating all of the work of parsing and type-checking (which is needed if the user manually monomorphized).
  - thu2111 7y ago
    That can't be right though. Java and Kotlin have generics, fast programmers, fast compile times and as generics are erased there's no more binary bloat than without them. I think the assumption implicit in that statement is "generics over value types compiled AOT". But that's not the only way to do it.
bonzini 7y ago
Is this with gccgo or gc? Compiled code quality is worse for gc, though that is mostly apparent on CPU bound code that most likely won't be written in Go (but might be written in Rust).
- dilap 7y ago
  gc

by dilap 7y ago

amelius 7y ago

Can you see how long incremental builds take (under a minor change)?

dilap 7y ago
it was very slow, almost a minute! but it has a custom build script, that looks like it’s not really made for incremental builds? it looks like there’s also bazel support, which i would guess is much faster, but i couldn’t figure out how to get it to work...

dnpp123 7y ago

It would be interesting to have a `reverse` blog post "why Go compilation time is fast" e.g. which optimizations are overlooked/done quickly by the go compiler ? Could the binary quality be improved by increasing the compilation time ? AFAIK you don't have optimization flags in go (or the default one are optimum). And to be honnest, I have no idea what the compiled Go code looks like. As much as I have no idea what actual instructions are executed by the python interpreter.

gwd 7y ago
Well one big reason is the biggest feature people are complaining about: A lack of generics. According to [1], there's a fundamental trade-off with respect to generics: "The generic dilemma is this: do you want slow programmers, slow compilers and bloated binaries, or slow execution times?" At the moment, Go has no generics, which translates to "slow programmers", but not "slow compilers". It's likely that the simplicity of the Go language as a whole -- which leads to many complaints -- is a major factor in the fast compile times. EDIT: Lest it seem like I'm bashing Go here, the quote above is from one of the core Golang developers (in 2009 no less), and is mirrored in this from the proposal [2] from another core Golang developer. It's meant to be a shorthand for, "Generics do actually save programmer time and effort, and Go programmers are working harder than necessary because the language doesn't have generics yet." The point of this post is to point out that generics have a cost (slow compilation time) and that fast compilation time has a cost (no generics -- at least, not without inventing a new way of doing generics). It's possible to love something while still seeing its deficiencies and wishing its improvement. In fact, I'd argue that's the only way to truly love anyone or anything. [1] https://research.swtch.com/generic https://research.swtch.com/generic [2] https://go.googlesource.com/proposal/+/master/design/15292-generics.md https://go.googlesource.com/proposal/+/master/design/15292-g...
- pmezard 7y ago
  > At the moment, Go has no generics, which translates to "slow programmers", but not "slow compilers". Also note that one of these assertions is measurable, and the other an unsubstantiated opinion.
- pcwalton 7y ago
  What I don't like about the "generics dilemma" framing is that the problem exists regardless of whether your language has generics or not. Here's what I mean. Say you have a Vector class that can operate on ints or floats. You could make that a generic, in which case the compiler can either (a) duplicate the code for each type (monomorphize) or (b) do dictionary passing and get slower runtime. But if your language doesn't have generics, you have exactly the same problem: you as the programmer must (a) duplicate the code for ints and floats or (b) use an interface and get slower runtime. Not having generics doesn't solve anything. It just means that you, the programmer, have to do things that the compiler would otherwise do for you.
  - Twisol 7y ago
    Although, one interesting aspect of the trade-off here is that a programmer who manually monomorphizes only has to do so once(^) -- their effort is reused across multiple compilation runs -- whereas the compiler normally has to monomorphize on each compilation run. (^) Of course, you then have the burden of keeping multiple monomorphized implementations consistent when you make a change that needs to apply to all of them.
    - eddyb 7y ago
      Monomorphization tends to happen "on the fly", and even if it wasn't, it would still be cheaper than duplicating all of the work of parsing and type-checking (which is needed if the user manually monomorphized).
- thu2111 7y ago
  That can't be right though. Java and Kotlin have generics, fast programmers, fast compile times and as generics are erased there's no more binary bloat than without them. I think the assumption implicit in that statement is "generics over value types compiled AOT". But that's not the only way to do it.

bonzini 7y ago

Is this with gccgo or gc? Compiled code quality is worse for gc, though that is mostly apparent on CPU bound code that most likely won't be written in Go (but might be written in Rust).

dilap 7y ago
gc