6 ms·
Since the Go compiler is so primitive you can strongly influence the size of the program by just restructuring it. An example: if you have a function that conve
by jeffbee 6y ago
Since the Go compiler is so primitive you can strongly influence the size of the program by just restructuring it. An example: if you have a function that converts integer return codes to errors, and you have another function that calls it a lot:
func codeToError(i int) error {
switch i {
case ...
...
}
}
func stuff() error {
if rv := callJunk(); rv < 0 {
return codeToError(rv)
}
if rv := moreJunk(); rv < 0 {
return codeToError(rv)
}
...
return nil
}
You're going to get multiple whole copies of the inner function inlined into the outer one. This can add up, if the outer function is long and/or if the inner switch statement is huge.
You can save a lot of code by jumping through an adapter function instead, like:
func callWithErrorsConverted(f func() int, g func(int) error) error {
return g(f())
}
... because now the compiler doesn't get to inline g into f. These things make a big difference in whole-program size. I wrote about it in the context of tailscale and protobufs here:
https://paper.dropbox.com/doc/Bloaty-Puffs-and-the-Go-Compiler--A4NFFR1SRcfOaz0kMw0ITuQ0AQ-YiblPOe6AD2tKfEj5RzFT https://paper.dropbox.com/doc/Bloaty-Puffs-and-the-Go-Compil...
- pantalaimon 6y agoI’m pretty happy I don’t have to worry about such details in C and can rely on the compiler to do the easy optimizations for me.
- ascar 6y agoWait, isn't that trick basically about avoiding optimization (inlining the function to avoid the context switch) to reduce code size? The C compiler would do something similar here and also increase code size for performance unless you explicitly use `-Os`
- monocasa 6y agoThe better C compilers use better heuristics in these cases, knowing that binary size can mean I$ pressure and therefore be a detriment to performance.
- a_e_k 6y agoThey also tend to have ways to override those heuristics (e.g., `__attribute__((noinline))` in GCC).
- singron 6y agogo has //go:noinline for the same effect. I don't think you need these weird trampoline calls and they are probably somewhat fragile since a future compiler might figure out how to inline through them.
- jeffbee 6y agoNobody needs to bounce through trampoline calls like this, but for people willing to chase a modest code size decrease like the 6% in the article, it's worth knowing that how your code is arranged may have a larger effect on program size than you're used to from other compiled languages.
- millstone 6y agoCache effects mean this is not a pure tradeoff. Any inlining routine needs a cost model. From the article it seems like Go compiler's cost model is not yet well tuned.
- jeffbee 6y agoIt's not really a language problem, it's a compiler problem, so at least in the future we might get a compiler that recognizes error-handling idioms. Or we just get llvm? The language problems, like the performance-hostile calling convention, are harder to fix.
- AmericanChopper 6y ago> It's not really a language problem, it's a compiler problem This has never seemed like such a clear distinction to me. A language is essentially just an instruction set for a compiler.
- NullPrefix 6y agoWell, there is a disctinction. It being a compiler problem, you can just accept the performance/size penalty and leave your code as is, but hope that in the future it will eventually change and the eventually really smart compiler will accept your current code as is and produce output without the penalty. Language problem would means you have to rewrite it differently.
- AmericanChopper 6y agoA language is both the abstractions that it implements, and the way it implements them. Any problem in the code you write could be solved by changing the compiler as much as it could be solved by changing the code. The two parts of the system are inseparable. I’d say a language can only be as good as its best compiler.
- somethingwitty1 6y agoBut they are quite different. There are languages that are tightly coupled, but not true of all (like Go). You could write all your applications in the Go Language, but use a transpiler. And as other comments call out, it is important to not conflate the two. A language issue is one that can't be easily fixed without breaking backwards compatibility. Compiler optimizations can be made (and is done all the time) without breaking the language.
- gameswithgo 6y agoohhh you think weird stuff doesn’t happen in C compilers? heh
- rowanG077 6y agoPretty insane that the compiler for such a popular language is so basic.
- gameswithgo 6y agoits also one of the fastest these things were a choice
- rowanG077 6y agoA feature flag doesn't have to impact compiler performance unless enabled.
- PudgePacket 6y agoI don't know if it would be in the spirit of Go. (I'm not for/against the philosophy of the decision making, just pointing it out).
- gameswithgo 6y agoit does, indirectly in a few ways. code size affects performance, human effort spread over more code paths, more branches. you could have two totally separate compilers, which go does, there is the gcc variant. i don’t know if it tends to produce faster binaries though.
- lostmyoldone 6y agoIt's even faster (at least on OSX) if you can invoke it correctly, which at least 2 years ago was almost impossible. If you manage to supply exactly the packages (transitively), and not a wildcard more, it would avoid recompiling the system libraries on each invocation. One would think a compiler where so much effort was put into making it fast would make it easy to have it run fast? The primitives were sort of there, but they didn't really fit together, and I actually had to write quite a bit of script/code to make it work. Got very fast though, thought I broke it the first time I got it to work. <return> and instant prompt. Toke me a while to convince myself it was actually compiling anything at all. Cut the recompile time in a VM to less than a tenth. Still slowed down by some pointless file IO (don't remember exactly what) but that was less of an issue. Vanity package URL's though was what broke this camels back, after GOPATH and some other miscellanea had strained it. When I ... probably needed to quickly fork+fix an external package, and suddenly had this additional obstacle, I was so ... unhappy. Already too much work, and then you get more work for exactly no discernible reason at all. It's even called a vanity url officially.
- badsectoracula 6y ago> Since the Go compiler is so primitive... That doesn't sound like Go's compiler is primitive, a primitive compiler wouldn't attempt the inlining at all. This sounds like a performance optimization which isn't something that a primitive compiler would attempt.
- lostmyoldone 6y agoThe compiler is not doing any sophisticated analysis at all, it can inline since the language is designed to be easy and fast to compile, but in my opinion at the expense of a little too much. I went through most of the compiler about two years ago to see ,amongst other, how much of its time was spent during IO since both compile time and correctness of dependency handling varied wildly in virtualized environments. Which is where I learned a lot, as for example that the correct way to invoke the compiler is essentially impossible to do with the supplied tools, but if you do it, it was actually possible to not recompile system libraries each time you build, that refactoring code where module scope is used for a most things is incredibly tedious. To compile to/from memory, as a service I had to change approximately 10 000 lines of code without being entirely done, as everything was tied into the module namespace and a little bit too concrete vfs like file access mechanism. Thus both the lifetimes of data, and where IO went were impossible to change without changing almost every type declaration site, hence often declaration, and attribute reference cite. Anything module based needed to be threaded through in appropriate places. Yes, and because some package names are hardcoded to be forbidden by parts of the toolchain unless it knows it is compiling itself, you have to patch the compiler (iirc) and the build tool if you want it to compile itself to a different name, which you kind of have to when you are doing something incredibly experimental. Otherwise it'd just overwrite itself because of gopath. Because of how the compiler/build works, it becomes contagious, so you'll need different names for everything. Well, you could probably get away with a little less renaming if you compiled the compiler with make, but at the point I realized that, I was already too far in for it to be any point in backtracking. Hardest issue was when it deadlocked on a mutex for a while, turns out one the main compiler go routines concurrency were not implemented correctly, so without the implicit serialization of the disk IO, it deadlocked almost every compile. Apologies for the digression, go was kind of forced on me, and some of the most touted "opinionated" parts lost me countless of hours of grief.