4 ms·

> Go has the goal of having a fast compiler. And importantly, one which is guaranteed to be fast for any program. > ... > This is especially important for env

> Go has the goal of having a fast compiler. And importantly, one which is guaranteed to be fast for any program. > ... > This is especially important for environments like the Go playground, which regularly compiles untrusted code. To be clear here, the tradeoff being made is that it is more important that the compiler is fast than that it is able to tell programmers about clear bugs in their code. I see the justification of the playground as unconvincing too. You can run 'for {}' in the playground, and you'll get "timeout running program". They already have solved the problem for the playground because obviously at runtime a go program can take infinite time, and it doesn't seem like it's any harder to also constrain compilation in the same way. The actual cost that the go compiler being fast has is that other parts of go are slow. In other languages, a macro will execute as part of compilation, and incremental compilation will work with it. In go, if you want a macro, you write a separate go program that generates code, write a "//go:generate" comment, and then have no way of knowing when to run it, no way of having incremental compilation with it, and it invariably ends up being slower than the compiler expanding it itself would have been. Now, if you want to know if you have an impossible type set, you'll run "go vet" (which likely doesn't do this validation, I haven't checked) or one of the various linting tools, it'll have to re-parse and re-compile all the code, it won't cache, and it'll ultimately give you the same answer the compiler could have given you. I suppose this is superior in one way: when performing such generation and validation, you can do it against only your own code, rather than against third party code you import, while if it were part of the compiler directly it would typically end up running against all compilation units, even third party ones.... In practice though, even for "slow to compile" languages, I find that I spend more time finding and fixing bugs than I do waiting for compilation, especially with incremental compilation. On the other hand, with go, I spend less time total compiling code, more time total fixing bugs, and more time total running golangci-lint and various other bits of code-generation. I spend exactly 0 time worrying that some adversarial third-party dependency I import will make my compile-times exponential, both in go where it can't happen by design, but also in other languages where it could happen, but you know, doesn't.

skybrian 4y ago
> In go, if you want a macro, you write a separate go program that generates code, write a "//go:generate" comment, and then have no way of knowing when to run it You run it every time you do a commit (at least), and then you check in the changes. Also, you probably run it during development by using a Makefile that will automatically run it whenever the input changes. This means that downstream packages never have to run it. They don't even have to have the code generation tool installed. This means a package maintainer can use any code generation tool they like. It was probably done that way due to experience with Bazel, where doing a build means you need to build and run code generators written in multiple programming languages. It does indeed slow things down.
- jrockway 4y ago
  I think the Bazel style is to never check in the generated code, and to generate it at build time (potentially hitting a cache based on the declared inputs, of course). For cases where I use "go generate", I tend to check in the generated file so that you can install the resulting package with "go install". (It's also nice when someone who just wants to work on another part of the code doesn't have to install the generation tool just to run the application/module.) In this case, you know to run "go generate" because the feature you just added doesn't work. (Though I tend to add a CI rule to make sure that the generator in CI produces the file that is checked into the repo. Because, yeah, you can forget, and it's nice for the computer to double check that.)
  - tylerhou 4y ago
    > I think the Bazel style is to never check in the generated code Not necessarily; there is plenty of checked in generated Go code at Google (as well as other PLs, but probably not as much), and other projects (e.g. Entgo) seem to encourage checking in generated Go code as well.
  - Arbortheus 4y ago
    I don't really understand how this approach would work well with IDEs/autocompletion though. For example, suppose I'm writing some mocked unit tests in Go and want to use mockgen to generate some mocks on an interface... //go:generate mockgen (N.B. This generation is done by Bazel) type Object interface { Thing() string } Then I need to use the generated mocked type in my test: mockThing := NewMockObject(ctrl) mockThing.EXPECT().Thing().Return("Foo") But NewMockObject doesn't exist as far as my IDE knows, because the generated code is only generated at build-time and only exists in the Bazel sandbox. The code will compile and run successfully, but the developer experience is terrible because you lose the ability to use your IDE's code lens, syntax checks, autocompletion. Everything that's generated shows a red squiggly this-doesnt-exist line. Maybe I'm just fundamentally misunderstanding how Bazel is supposed to be used. My setup is pretty typical. I develop in VS Code, and build using Bazel, but I do not allow Bazel to automate code generation because the generated sources do not appear in my local environment - just in the bazel sandbox. Maybe someone can suggest a better approach?
tapirl 4y ago
> ... but also in other languages where it could happen, but you know, doesn't. My rust experience denies this.
GlitchMr 4y ago
Additionally, they do in fact have a timeout for build process in the Go playground. I tried running https://go.dev/play/p/mwyBUjcKvAo https://go.dev/play/p/mwyBUjcKvAo, and it says "timeout running go build".
doctor_eval 4y ago
Frankly, I think that Go compiler speed is a huge feature, and this trade-off seems quite reasonable to me. It’s not that the bug will be undetected, it’s that it will be detected at the point of use rather than the declaration. But still at compile time. So what? It’s going to be pretty obvious what’s wrong.
morelisp 4y ago
> more important that the compiler is fast than that it is able to tell programmers about clear bugs in their code. The actual tradeoff being made here is more like "it's better to give a good error message in 1s than a great error message in 10s." That doesn't seem obviously wrong to me.
dolmen 4y ago
> In go, if you want a macro, you write a separate go program that generates code, write a "//go:generate" comment, and then have no way of knowing when to run it, no way of having incremental compilation with it, and it invariably ends up being slower than the compiler expanding it itself would have been. 1. Macros are just one use case replaced by go:generate step. 2. go:generated files are usually not the most moving parts while developing (except of course while you write the generator or its input. In fact go:embed has allowed to move some generated files to compile time and have not seen the need to replace go:generate with go:embed for those. (Does anyone has a compile benchmark?) 2. The Go compiler already uses caches and we can expect more and more caching will happen in future releases. So I'm really not concerned by the compile time of files produced by go:generate instead of macros.
Merovius 4y ago
> To be clear here, the tradeoff being made is that it is more important that the compiler is fast than that it is able to tell programmers about clear bugs in their code. I'm not sure that's entirely true. Or it seems at least misleading. "Fast" here means "guaranteed to finish before the heat death of the universe", in the worst case. And as I'm trying to explain in the post, it is easier to explain clear bugs to the programmer, if you don't have to rely on solving NP-complete problems. NP-complete problems are notoriously hard to report errors for, which the user understands.

skybrian 4y ago

> In go, if you want a macro, you write a separate go program that generates code, write a "//go:generate" comment, and then have no way of knowing when to run it You run it every time you do a commit (at least), and then you check in the changes. Also, you probably run it during development by using a Makefile that will automatically run it whenever the input changes. This means that downstream packages never have to run it. They don't even have to have the code generation tool installed. This means a package maintainer can use any code generation tool they like. It was probably done that way due to experience with Bazel, where doing a build means you need to build and run code generators written in multiple programming languages. It does indeed slow things down.

jrockway 4y ago
I think the Bazel style is to never check in the generated code, and to generate it at build time (potentially hitting a cache based on the declared inputs, of course). For cases where I use "go generate", I tend to check in the generated file so that you can install the resulting package with "go install". (It's also nice when someone who just wants to work on another part of the code doesn't have to install the generation tool just to run the application/module.) In this case, you know to run "go generate" because the feature you just added doesn't work. (Though I tend to add a CI rule to make sure that the generator in CI produces the file that is checked into the repo. Because, yeah, you can forget, and it's nice for the computer to double check that.)
- tylerhou 4y ago
  > I think the Bazel style is to never check in the generated code Not necessarily; there is plenty of checked in generated Go code at Google (as well as other PLs, but probably not as much), and other projects (e.g. Entgo) seem to encourage checking in generated Go code as well.
- Arbortheus 4y ago
  I don't really understand how this approach would work well with IDEs/autocompletion though. For example, suppose I'm writing some mocked unit tests in Go and want to use mockgen to generate some mocks on an interface... //go:generate mockgen (N.B. This generation is done by Bazel) type Object interface { Thing() string } Then I need to use the generated mocked type in my test: mockThing := NewMockObject(ctrl) mockThing.EXPECT().Thing().Return("Foo") But NewMockObject doesn't exist as far as my IDE knows, because the generated code is only generated at build-time and only exists in the Bazel sandbox. The code will compile and run successfully, but the developer experience is terrible because you lose the ability to use your IDE's code lens, syntax checks, autocompletion. Everything that's generated shows a red squiggly this-doesnt-exist line. Maybe I'm just fundamentally misunderstanding how Bazel is supposed to be used. My setup is pretty typical. I develop in VS Code, and build using Bazel, but I do not allow Bazel to automate code generation because the generated sources do not appear in my local environment - just in the bazel sandbox. Maybe someone can suggest a better approach?

tapirl 4y ago

> ... but also in other languages where it could happen, but you know, doesn't. My rust experience denies this.

GlitchMr 4y ago

Additionally, they do in fact have a timeout for build process in the Go playground. I tried running https://go.dev/play/p/mwyBUjcKvAo https://go.dev/play/p/mwyBUjcKvAo, and it says "timeout running go build".

doctor_eval 4y ago

Frankly, I think that Go compiler speed is a huge feature, and this trade-off seems quite reasonable to me. It’s not that the bug will be undetected, it’s that it will be detected at the point of use rather than the declaration. But still at compile time. So what? It’s going to be pretty obvious what’s wrong.

morelisp 4y ago

> more important that the compiler is fast than that it is able to tell programmers about clear bugs in their code. The actual tradeoff being made here is more like "it's better to give a good error message in 1s than a great error message in 10s." That doesn't seem obviously wrong to me.

dolmen 4y ago

> In go, if you want a macro, you write a separate go program that generates code, write a "//go:generate" comment, and then have no way of knowing when to run it, no way of having incremental compilation with it, and it invariably ends up being slower than the compiler expanding it itself would have been. 1. Macros are just one use case replaced by go:generate step. 2. go:generated files are usually not the most moving parts while developing (except of course while you write the generator or its input. In fact go:embed has allowed to move some generated files to compile time and have not seen the need to replace go:generate with go:embed for those. (Does anyone has a compile benchmark?) 2. The Go compiler already uses caches and we can expect more and more caching will happen in future releases. So I'm really not concerned by the compile time of files produced by go:generate instead of macros.

Merovius 4y ago

> To be clear here, the tradeoff being made is that it is more important that the compiler is fast than that it is able to tell programmers about clear bugs in their code. I'm not sure that's entirely true. Or it seems at least misleading. "Fast" here means "guaranteed to finish before the heat death of the universe", in the worst case. And as I'm trying to explain in the post, it is easier to explain clear bugs to the programmer, if you don't have to rely on solving NP-complete problems. NP-complete problems are notoriously hard to report errors for, which the user understands.