4 ms·

What's so confusing from this post, is the complete lack of motivations for why anyone would want this over a shell script at the root of their project. I'd rat

by codemac 12y ago

What's so confusing from this post, is the complete lack of motivations for why anyone would want this over a shell script at the root of their project. I'd rather it even blindly rerun the command every `go build` than this half hearted attempt. This looks much like my team's `make update` rule (that merely runs ./mk/update.sh). Well, except now the dependencies on external tools are scattered throughout your project. Oh and build failures wouldn't even tell you that you need to run go generate. What a mis-step. Then again, maybe I'm somehow mistaking a build system with a compiler[0]? [0]: https://news.ycombinator.com/item?id=8733493 https://news.ycombinator.com/item?id=8733493

enneff 12y ago
You just want this feature to be more than it is. It's a small, simple thing. Don't overthink it. If you need a more elaborate build system, use one.
- codemac 12y ago
  > If you need a more elaborate build system, use one. I thought I was: https://golang.org/doc/articles/go_command.html https://golang.org/doc/articles/go_command.html > An explicit goal for Go from the beginning was to be able to build Go code using only the information found in the source itself, not needing to write a makefile or one of the many modern replacements for makefiles. If Go needed a configuration file to explain how to build your program, then Go would have failed.
  - enneff 12y ago
    I said "use a more elaborate build system." That means, "more elaborate than the Go tool." So, no, you're not.
- wtetzner 12y ago
  It's a small, simple thing, but I still don't see what value it adds to just using a script.
  - NateDad 12y ago
    It's cross platform compatible with no other dependencies, and the instructions live close to the code that are affected by them.
    - TheDong 12y ago
      "cross platform compatible with no other dependencies". No, not at all. Not even close. Your go generate comment takes the form of //go:generate someCommand arg1 arg2 That's as far from cross-platform as possible. For example, their "yacc" example will not run on a platform without yacc. Any command you put there will have to be present on each platform and any command you put there is a dependency. Worse yet, it's a dependency that you can't explicitly declare and manage. You can't say "runtime dependencies: github.com/tools/godep, generate dependencies: yacc". This is a terrible mess if you want to write a cross-platform generator and it's an even bigger mess if your generator isn't something present on every platform already because otherwise generate will just fail with "command not found".
      - 4ad 12y ago
        Nitpick: Go's yacc is part of the standard Go distribution, every Go installation has it. But it doesn't matter, go generate can indeed use programs a user might not have. But you know what, that doesn't matter either, it's true for shell scripts and it's true for makefiles as well. Go generate changes nothing in that respect, however, it does improve portability significantly in other ways. For example, in the Go tree we replaced shell scripts with go generate and perl scripts with go code called by go generate. This is important because Windows and Plan 9 don't have bourne shells, Plan 9 doesn't have perl at all and in Windows perl has to be installed. Note that there are no makefiles, which don't exist natively on Windows and Plan 9 either.
brandonbloom 12y ago
.PHONY targets and multi-line Make rule bodies are a pet peeve of mine too. I'm not sure why people are so afraid to create extra scripts. When you're in the business of running processes, the programming language is shell and the unit of abstraction is a script file: You shouldn't be afraid to make a .sh file at nearly the level of granularity you'd make a top-level function in your general purpose environment.
- gvalkov 12y ago
  I find that in recent versions of GNU make, the ONESHELL and define directives alleviate some of the pain of multi-line rules. There is also this emacs package [1] I wrote while maintaining an inline-shell heavy makefile framework that had to run on make 3.81 (the version that comes with CentOS 6). I definitely agree with you on extracting rules into separate scripts, but I've yet to see a make-based build system that embraces this idea. [1]: https://github.com/gvalkov/makefile-shell-backslash
- pjc50 12y ago
  Because having one function per file is a huge hassle when editing them. I'm working on a system which is built out of >50 batch files in that manner; I had to write a tool to show me the callgraph in order to properly understand it.
twotwotwo 12y ago
The norm in Go is that project info lives, as much as possible, in the source files themselves--that includes dependencies, architecture-specific build tags, canonical import paths, etc. Also, as many steps as reasonably possible are accessible through the 'go' tool--testing, building, fmt, vet, get/install, etc.--even if the go tool has to invoke other programs on the local machine to do it (like how go get invokes git/hg, or go build can invoke a C compiler for cgo). This applies the "project info in the source" and "make stuff doable through the go tool" ideas to codegen. I don't think it's life-changing (I hope it's not too life-changing, i.e., that people don't go too crazy building hard-to-maintain Go-generating hacks) but it seems consistent.
skybrian 12y ago
Yes it's under-motivated, but you can see it if you read in between the lines. Start out with the goal that authors of open source Go packages should check in generated Go source and nothing more is needed. Even this much support for code generation isn't strictly necessary. The article only gives one reason why generated code should be checked in ("if only for the reason that the program it invokes might not be available on the target machine") but I suspect there might be other unstated reasons: it's useful to have a permanent record of generated code in source control for auditing and debugging, and it ensures that package authors can't make downstream builds slower by choosing a slow build tool. In the open source world, your dependencies are maintained by people in other organizations. Avoiding dependencies on other teams' crufty build files is a good thing.
- MaulingMonkey 12y ago
  > The article only gives one reason why generated code should be checked in ("if only for the reason that the program it invokes might not be available on the target machine") but I suspect there might be other unstated reasons: it's useful to have a permanent record of generated code in source control for auditing and debugging, and it ensures that package authors can't make downstream builds slower by choosing a slow build tool. I've definitely found diffing the generated code useful for confirming refactorings of the generation code don't change anything unexpected - and I imagine those doing my code reviews do as well. I might not make much use of the actual VCS history, but committing to VCS gives me sane diffs to look at for minimal effort. > In the open source world, your dependencies are maintained by people in other organizations. Avoiding dependencies on other teams' crufty build files is a good thing. Agreed (although sometimes the build files are good.) I'm pretty sure that moving crufty build rules into such a primitive build system isn't help on that front however. Even as a Microsoft coolaid drinking, Visual Studio using, C# loving windows dev... I think I'd much rather hack up or rewrite someone's Makefile than go diving through the source to find all the similar-but-not-quite-the-same build rules scattered throughout the codebase.
olalonde 12y ago
> What's so confusing from this post, is the complete lack of motivations for why anyone would want this over a shell script at the root of their project. I'm not very familiar with go but I think the motivation might be inspired by Python's "one way to do it" maxim. https://wiki.python.org/moin/TOOWTDI https://wiki.python.org/moin/TOOWTDI This seems to be the philosophy adopted by Go as well (e.g. the gofmt tool, the integrated build system, only one loop construct, slices, etc.). Personally, I think this philosophy is extremely beneficial in making code more readable and I would say this feature is consistent with it.
- zak_mc_kracken 12y ago
  > I'm not very familiar with go but I think the motivation The motivation is pretty transparent: since the Go team is categorically refusing to implement generics, they are trying to find workarounds to avoid having developers copy/paste code to simulate genericity. Their solution is to have a tool generate that code instead of developers doing the copy/paste.
  - mahmoudimus 12y ago
    That was my initial impressions as well. Seems like a convoluted way to get around Generics. Very interesting stance from the Go team.
DAddYE 12y ago
While I agree with you I think it's just the cross compatibility the main benefit. So our friends on windows don't have to install cygwin to run a Makefile. However I'd admit that I don't know windows and real motivations behind it.
TheDong 12y ago
Worse yet, a shell script would work whereas go generate does not. Go generate is probably the most half-assed thing the go team has done yet. Now, I know the above line is harsh, but I don't say that without proof. Run the following and bear witness to its terrible implementation. curl http://pastebin.com/raw.php?i=fhu14iwk > main.go cat main.go go run main.go go generate -v main.go go generate -help go generate -x -v -run ".*(echo|one).*" Yeah... If you don't feel like looking at the above, the implementation of go generate is a hack of substring matching, not language parsing, and it also doesn't implement, in its production release, the "-run" flag documented in "-help".
- breakingcups 12y ago
  Your concern seems fair and valid. Have you raised this issue on the mailing list?
- bluesign 12y ago
  I don't think this is a valid test case, as far as I understand, //go:generate works like "preprocessor directive". Maybe I can agree that // prefix is a bad choice, as coming from C world I would prefer #
  - TheDong 12y ago
    Here's a corresponding C program in which I use a #define that is not processed by the pre-processor since it's not a dumb string matcher. http://pastebin.com/raw.php?i=dE7QsTgS http://pastebin.com/raw.php?i=dE7QsTgS Define might be. But actual directives are not. In go, the directives are.
Someone 12y ago
Indeed. Why mix two grammars in one file? Now, that "go generate" call has to parse go (at least up to comments; I think it should ignore "//go:generate" inside multi-line strings), and then parse those comments. I would think it would be way simpler to only allow those special comments in the input file for "go generate". If I were snarky, I would add that, if this gets used a lot, it might be handy to add dependency information to that file, so that "go generate" can skip repeated work. Once it does that, people can call it every time they compile their code, gaining convenience without much loss of time. I also wonder what the distribution of code will look like. The goal seems to be to shield ordinary programmers from code generation. That is somewhat laudable (I would like to see a 'layered' language where upper layers are highly similar, but simpler to use and a bit less powerful, but haven't seen a good one yet), but at odds with the GPL. I think the GPL will require you to distribute the master code. When used as an alternative for generics, distributing only the generated code also will lead to diverging of the various generated versions. People will fork and then modify only eight of those ten generated files.

by codemac 12y ago

enneff 12y ago

You just want this feature to be more than it is. It's a small, simple thing. Don't overthink it. If you need a more elaborate build system, use one.

codemac 12y ago
> If you need a more elaborate build system, use one. I thought I was: https://golang.org/doc/articles/go_command.html https://golang.org/doc/articles/go_command.html > An explicit goal for Go from the beginning was to be able to build Go code using only the information found in the source itself, not needing to write a makefile or one of the many modern replacements for makefiles. If Go needed a configuration file to explain how to build your program, then Go would have failed.
- enneff 12y ago
  I said "use a more elaborate build system." That means, "more elaborate than the Go tool." So, no, you're not.
wtetzner 12y ago
It's a small, simple thing, but I still don't see what value it adds to just using a script.
- NateDad 12y ago
  It's cross platform compatible with no other dependencies, and the instructions live close to the code that are affected by them.
  - TheDong 12y ago
    "cross platform compatible with no other dependencies". No, not at all. Not even close. Your go generate comment takes the form of //go:generate someCommand arg1 arg2 That's as far from cross-platform as possible. For example, their "yacc" example will not run on a platform without yacc. Any command you put there will have to be present on each platform and any command you put there is a dependency. Worse yet, it's a dependency that you can't explicitly declare and manage. You can't say "runtime dependencies: github.com/tools/godep, generate dependencies: yacc". This is a terrible mess if you want to write a cross-platform generator and it's an even bigger mess if your generator isn't something present on every platform already because otherwise generate will just fail with "command not found".
    - 4ad 12y ago
      Nitpick: Go's yacc is part of the standard Go distribution, every Go installation has it. But it doesn't matter, go generate can indeed use programs a user might not have. But you know what, that doesn't matter either, it's true for shell scripts and it's true for makefiles as well. Go generate changes nothing in that respect, however, it does improve portability significantly in other ways. For example, in the Go tree we replaced shell scripts with go generate and perl scripts with go code called by go generate. This is important because Windows and Plan 9 don't have bourne shells, Plan 9 doesn't have perl at all and in Windows perl has to be installed. Note that there are no makefiles, which don't exist natively on Windows and Plan 9 either.

brandonbloom 12y ago

.PHONY targets and multi-line Make rule bodies are a pet peeve of mine too. I'm not sure why people are so afraid to create extra scripts. When you're in the business of running processes, the programming language is shell and the unit of abstraction is a script file: You shouldn't be afraid to make a .sh file at nearly the level of granularity you'd make a top-level function in your general purpose environment.

gvalkov 12y ago
I find that in recent versions of GNU make, the ONESHELL and define directives alleviate some of the pain of multi-line rules. There is also this emacs package [1] I wrote while maintaining an inline-shell heavy makefile framework that had to run on make 3.81 (the version that comes with CentOS 6). I definitely agree with you on extracting rules into separate scripts, but I've yet to see a make-based build system that embraces this idea. [1]: https://github.com/gvalkov/makefile-shell-backslash
pjc50 12y ago
Because having one function per file is a huge hassle when editing them. I'm working on a system which is built out of >50 batch files in that manner; I had to write a tool to show me the callgraph in order to properly understand it.

twotwotwo 12y ago

The norm in Go is that project info lives, as much as possible, in the source files themselves--that includes dependencies, architecture-specific build tags, canonical import paths, etc. Also, as many steps as reasonably possible are accessible through the 'go' tool--testing, building, fmt, vet, get/install, etc.--even if the go tool has to invoke other programs on the local machine to do it (like how go get invokes git/hg, or go build can invoke a C compiler for cgo). This applies the "project info in the source" and "make stuff doable through the go tool" ideas to codegen. I don't think it's life-changing (I hope it's not too life-changing, i.e., that people don't go too crazy building hard-to-maintain Go-generating hacks) but it seems consistent.

skybrian 12y ago

Yes it's under-motivated, but you can see it if you read in between the lines. Start out with the goal that authors of open source Go packages should check in generated Go source and nothing more is needed. Even this much support for code generation isn't strictly necessary. The article only gives one reason why generated code should be checked in ("if only for the reason that the program it invokes might not be available on the target machine") but I suspect there might be other unstated reasons: it's useful to have a permanent record of generated code in source control for auditing and debugging, and it ensures that package authors can't make downstream builds slower by choosing a slow build tool. In the open source world, your dependencies are maintained by people in other organizations. Avoiding dependencies on other teams' crufty build files is a good thing.

MaulingMonkey 12y ago
> The article only gives one reason why generated code should be checked in ("if only for the reason that the program it invokes might not be available on the target machine") but I suspect there might be other unstated reasons: it's useful to have a permanent record of generated code in source control for auditing and debugging, and it ensures that package authors can't make downstream builds slower by choosing a slow build tool. I've definitely found diffing the generated code useful for confirming refactorings of the generation code don't change anything unexpected - and I imagine those doing my code reviews do as well. I might not make much use of the actual VCS history, but committing to VCS gives me sane diffs to look at for minimal effort. > In the open source world, your dependencies are maintained by people in other organizations. Avoiding dependencies on other teams' crufty build files is a good thing. Agreed (although sometimes the build files are good.) I'm pretty sure that moving crufty build rules into such a primitive build system isn't help on that front however. Even as a Microsoft coolaid drinking, Visual Studio using, C# loving windows dev... I think I'd much rather hack up or rewrite someone's Makefile than go diving through the source to find all the similar-but-not-quite-the-same build rules scattered throughout the codebase.

olalonde 12y ago

> What's so confusing from this post, is the complete lack of motivations for why anyone would want this over a shell script at the root of their project. I'm not very familiar with go but I think the motivation might be inspired by Python's "one way to do it" maxim. https://wiki.python.org/moin/TOOWTDI https://wiki.python.org/moin/TOOWTDI This seems to be the philosophy adopted by Go as well (e.g. the gofmt tool, the integrated build system, only one loop construct, slices, etc.). Personally, I think this philosophy is extremely beneficial in making code more readable and I would say this feature is consistent with it.

zak_mc_kracken 12y ago
> I'm not very familiar with go but I think the motivation The motivation is pretty transparent: since the Go team is categorically refusing to implement generics, they are trying to find workarounds to avoid having developers copy/paste code to simulate genericity. Their solution is to have a tool generate that code instead of developers doing the copy/paste.
- mahmoudimus 12y ago
  That was my initial impressions as well. Seems like a convoluted way to get around Generics. Very interesting stance from the Go team.

DAddYE 12y ago

While I agree with you I think it's just the cross compatibility the main benefit. So our friends on windows don't have to install cygwin to run a Makefile. However I'd admit that I don't know windows and real motivations behind it.

TheDong 12y ago

Worse yet, a shell script would work whereas go generate does not. Go generate is probably the most half-assed thing the go team has done yet. Now, I know the above line is harsh, but I don't say that without proof. Run the following and bear witness to its terrible implementation. curl http://pastebin.com/raw.php?i=fhu14iwk > main.go cat main.go go run main.go go generate -v main.go go generate -help go generate -x -v -run ".*(echo|one).*" Yeah... If you don't feel like looking at the above, the implementation of go generate is a hack of substring matching, not language parsing, and it also doesn't implement, in its production release, the "-run" flag documented in "-help".

breakingcups 12y ago
Your concern seems fair and valid. Have you raised this issue on the mailing list?
bluesign 12y ago
I don't think this is a valid test case, as far as I understand, //go:generate works like "preprocessor directive". Maybe I can agree that // prefix is a bad choice, as coming from C world I would prefer #
- TheDong 12y ago
  Here's a corresponding C program in which I use a #define that is not processed by the pre-processor since it's not a dumb string matcher. http://pastebin.com/raw.php?i=dE7QsTgS http://pastebin.com/raw.php?i=dE7QsTgS Define might be. But actual directives are not. In go, the directives are.

Someone 12y ago

Indeed. Why mix two grammars in one file? Now, that "go generate" call has to parse go (at least up to comments; I think it should ignore "//go:generate" inside multi-line strings), and then parse those comments. I would think it would be way simpler to only allow those special comments in the input file for "go generate". If I were snarky, I would add that, if this gets used a lot, it might be handy to add dependency information to that file, so that "go generate" can skip repeated work. Once it does that, people can call it every time they compile their code, gaining convenience without much loss of time. I also wonder what the distribution of code will look like. The goal seems to be to shield ordinary programmers from code generation. That is somewhat laudable (I would like to see a 'layered' language where upper layers are highly similar, but simpler to use and a bit less powerful, but haven't seen a good one yet), but at odds with the GPL. I think the GPL will require you to distribute the master code. When used as an alternative for generics, distributing only the generated code also will lead to diverging of the various generated versions. People will fork and then modify only eight of those ten generated files.