6 ms·
> Other than Eli’s post, there are no documentation or articles on Go compiler internals. How does someone get started working on them? How do they navigate and
by johnisgood 5y ago
> Other than Eli’s post, there are no documentation or articles on Go compiler internals. How does someone get started working on them? How do they navigate and find all these intricacies without spending hours? Maybe Google has some internal documentation on the Go compiler.
It would be nice to have more information out there on the internals of the Go compiler. Perhaps there is.
I found stuff like:
- https://github.com/emluque/golang-internals-resources https://github.com/emluque/golang-internals-resources
- https://www.altoros.com/blog/golang-internals-part-1-main-concepts-and-project-structure/ https://www.altoros.com/blog/golang-internals-part-1-main-co...
- https://github.com/teh-cmc/go-internals https://github.com/teh-cmc/go-internals
But yeah, Eli's articles[1] are pretty good.
[1] https://eli.thegreenplace.net/2019/go-compiler-internals-adding-a-new-statement-to-go-part-1/ https://eli.thegreenplace.net/2019/go-compiler-internals-add...
- _wldu 5y agoThis is a great demo. The old C compiler backdoor, but in Go: https://github.com/yeokm1/reflections-on-trusting-trust-go https://github.com/yeokm1/reflections-on-trusting-trust-go The Gopher Con Singapore (2018) video is a really great summary (20 mins). He modifies the compiler and inserts the backdoor during the presentation: https://www.youtube.com/watch?v=T82JttlJf60&list=PLq2Nv-Sh8EbbIjQgDzapOFeVfv5bGOoPE&index=14 https://www.youtube.com/watch?v=T82JttlJf60&list=PLq2Nv-Sh8E...
- melony 5y agoThe Go compiler is about as straightforward as it can get. Just read the source: https://github.com/golang/go https://github.com/golang/go
- avinassh 5y agoI don't think I would have figured out how one adds a new token if not for Eli's post. This comment [0] perfectly explains the quirks I ran into. As an exercise, can you help me figure out how to add a token just from the source and discover these quirks? On second thought, reading from the source and figuring it out could have been possible if you spent hours. But don't you think it should also have some comments to navigate? [0] - https://news.ycombinator.com/item?id=29489113 https://news.ycombinator.com/item?id=29489113
- londons_explore 5y agoGoogle has an amazing code search tool. You can try it out here[1]. That generally makes browsing source code much quicker and easier, which in turn makes understanding the structure of huge codebases much easier. With that tool, I prefer to just dive into the source in most cases rather than read documentation, especially when there is a good chance the documentation is wrong/outdated. [1]: https://source.chromium.org https://source.chromium.org
- preseinger 5y agoCode explains what and how, but not why. Why is necessary for building a robust mental model of any system, and can only be provided by documentation (or other humans).
- londons_explore 5y agoGoogles source code files frequently have 50+ lines of comments at the top of the file to explain the why...
- preseinger 5y agoRight, so, documentation... ?
- johnisgood 5y ago> https://source.chromium.org https://source.chromium.org Wow, it is pretty cool! Is there an open source software that is similar to this? It reminds me of https://elixir.bootlin.com/ https://elixir.bootlin.com/. I really want something like these two. A cross-referencer, but seems like the Chromium one does more (editing, for one). The Chromium one seems to do more git-related stuff as well. Currently checking out https://github.com/bootlin/elixir https://github.com/bootlin/elixir. Edit: Oh Wow, GitHub just improved their code search. Pretty cool. Seems like it can do cross-referencing (even between different repositories!).
- ferdowsi 5y agoI clicked into the Go Github repo and found documentation pretty easily. The compiler code itself is well documented, and Go's code navigation tooling itself helps learning. https://github.com/golang/go/tree/master/src/cmd/compile https://github.com/golang/go/tree/master/src/cmd/compile
- avinassh 5y agoI did run into this. The page linked to is a high level documentation with very few details which are specific to the codebase. Take the example of adding a new token. You have to run go generate to generate token strings. But nowhere in the docs or in the code it is mentioned what exactly is the ‘stringer’ and how to install it.
- avinassh 5y agoThank you for these links! I think I have seen the first two at some point, but they weren't helpful. The `go-internals` looks great, and I will check them out. I am also curious about the daily development cycle by a regular Go contributor. How do they make changes, how do they do quick tests before running the whole test suite etc