6 ms·
Padding is hard
- zeveb 11y agoGiven how much Go already does for one—and given the static nature of Go binaries—is there any reason why the compiler shouldn't reorder structs so that they take the least space possible? I can see that that would prevent one from reliably dumping memory to disk and then re-reading it, but It's hardly reliable to do that anyway. One should be using an encoding anyway…
- gtfierro 11y agoGoing beyond this being a minor optimization that isn't going to matter for most people, my guess is that they probably don't reorder structs because it may not be what the programmer intended. When you start introducing mechanisms for altering how the compiler sees a certain piece of code (like a struct definition), then you also need to give the programmer a way of expressing which thing they want the compiler to do, like with __attribute__ in C. Adding that feature in a straightforward way would probably involve introducing more syntax, which Go wants to avoid.
- masklinn 11y agoAt the same time, getting either source-layout guarantees or a way to explicitly specify layout is necessary for some task types, to avoid false sharing for instance. Go currently does neither.
- coldtea 11y ago>Given how much Go already does for one How much is that? As far as modern (2000 and onwards) languages Go, Golang and the compiler are pretty barebones. The compiler especially is as basic as it gets...
- kibwen 11y agoIf you're handing off a struct to C code, the order of your fields is going to be significant. If the Go compiler began reordering struct fields as an optimization, it would need to provide some way to disable that optimization on a per-struct basis (and unless Go could automatically detect which structs are crossing an FFI boundary, this optimization would need to be opt-in rather than opt-out in order to avoid breaking existing code). As a precedent, Rust deliberately does what you describe and reserves the right to reorder struct fields (though AFAIK it's not actually exercising this right yet), and in order to manually enforce a struct's layout you must tag it with the #[repr(C)] attribute.
- masklinn 11y agoGo struct memory layout is currently undefined[0], so you're already not supposed to hand structs off to C, or to memcpy it to or from the wire. Unless you're using cgo maybe, I don't know if (or that) cgo provides any guarantee. [0] https://github.com/golang/go/issues/10014 https://github.com/golang/go/issues/10014
- kibwen 11y agoIs this mentioned in the spec anywhere? That piece of information was precisely what I was looking for, but wasn't able to find.
- masklinn 11y agoThe spec doesn't define field ordering anywhere, which is why it's undefined. Russ Cox commented on the bug I linked essentially confirming it: > It seems fine to me for the spec not to guarantee anything about struct field order in memory. The spec doesn't operate at that level. And while he adds > That said, no Go compiler should probably ever reorder struct fields. the final result is struct memory layout is unspecified and implementation-dependent (essentially identical to repr(rust), as I understand it)
- kibwen 11y ago
- adonovan 11y agoThe most compact representation is not always the most efficient. If two fields in the same struct variable are accessed by different goroutines, placing them far apart so that they reside in different cache lines may avoid a great deal of contention.
- masklinn 11y agoExcept technically struct memory layout is currently undefined in Go so you're relying on an implementation detail.
- adonovan 11y agoThis kind of optimized placement could in principle be done by a sufficiently smart compiler, but no such compiler exists. That's why existing compilers allow their users to control layout by, as you say, relying on an implementation detail.
- masklinn 11y agoNo, the language can specify source—order struct layout, either by default or opt-in.
- Someone 11y agoI would guess that their stance is "If you care about that, do it yourself" IMO, that is the right attitude, given that moving fields around may interfere with how parts of large structures map to cache lines, and that may have huge performance impacts, if some parts of a structure are rarely accessed. Yes, the compiler could try to optimize that, too, but saying that is a hard problem is an understatement, and it needs statistics that the compiler doesn't have available. So, it definitely is unsuitable for go's goal of fast compilation.
- masklinn 11y ago> I would guess that their stance is "If you care about that, do it yourself" No, struct layout is currently undefined[0] so you can't do that yourself. Any manual packing you defined today may or may not hold across platforms or future go revisions. [0] https://github.com/golang/go/issues/10014 https://github.com/golang/go/issues/10014
- neilc 11y ago> why the compiler shouldn't reorder structs so that they take the least space possible I think you'd get most of the benefit by writing a static analysis tool (a la "go vet") that suggests when reordering a struct's fields would reduce memory consumption.
- masklinn 11y agoExcept Go doesn't guarantee source-order struct layout, so that'd be a suggestion relying on implementation details (see also: relying on scheduling order of goroutines).
- hoprocker 11y agoSide question: who is in that animated gif about a quarter of the way down the page (fingers to temples, starbursts in background)? I see that around on dev blogs quite a bit.
- deleted 11y ago[deleted]
- atria 11y agoSo there isn't any equivalent to #pragma pack in Go? I'll stick with C/C++, thanks.
- ajkjk 11y agoReally, just like that? Seems irrational to discount every other argument involved in such a decision.
- ajkjk 11y agoIf you're like me and had no idea what _ struct{} // to prevent unkeyed literals means, some explanation turns up here: https://groups.google.com/forum/#!topic/golang-nuts/NSjVW82i0mY https://groups.google.com/forum/#!topic/golang-nuts/NSjVW82i...
- nkurz 11y agoThe article mentions that the "Go spec says the address of a struct’s fields must be naturally aligned", and seems to take for granted that is necessary for speed. A separate question would be whether it actually helps performance. For modern x64/x86 the answer seems be to "no": http://lemire.me/blog/2012/05/31/data-alignment-for-speed-myth-or-reality/ http://lemire.me/blog/2012/05/31/data-alignment-for-speed-my... Would it make sense for the Go spec to drop the alignment requirement? I'd think that a smaller memory footprint would be the better choice, and I'd think that interoperability would be better served by allowing arbitrary alignments.