5 ms·

> How often is "reduce cache misses" that different from "use as little space as possible"? I thought the same exact thing. If I can reduce my struct sizes I c

> How often is "reduce cache misses" that different from "use as little space as possible"? I thought the same exact thing. If I can reduce my struct sizes I can pack N more structs into my cache line. That's almost certainly going to be the best cache-based win.

deleted 5y ago
[deleted]
tedunangst 5y ago
If your struct is large enough that you care about shaving padding, you probably have hot fields and cold fields and the best cached based win will be arranging them contiguously.
- morelisp 5y ago
  If I have a [10000]T I need to shave very little padding before I see some impact, even though both the padded and unpadded T might be relatively small. Specifically in Go, a smaller size can get even lone values into a smaller size class. Saving one byte may save you 384 if it's from 2305 to 2304.
- fredophile 5y ago
  The example in the link had 3 fields and padding made the structure 50% bigger than it needed to be. That probably likely big enough to have hot and cold fields but clearly they cared about shaving padding.
- anonymoushn 5y ago
  Most structs that care about shaving padding will be very small.
- mcherm 5y ago
  Actually, you care about shaving padding when the FIELDS are small compared to the alignment, not the structs. This frequently is a concern in Go.
- pornel 5y ago
  If you have a large-enough group of distinctly cold fields, a better option may be to move them to another struct, possibly allocated out of line.
- usrusr 5y ago
  Can go structs be nested by value? (As opposed to by reference) I'd imagine that this could be a very practical way to to do a manual hot/cold grouping in an environment that prioritizes packing over keeping order. (if the cold ones are particularly cold you might of course prefer them to be in a by reference nested struct anyways, so the hot subset packs better with their peers in an array)
- jeltz 5y ago
  That does not match my real world experience. If you have an array of small structs then the padding matters. And if some fields are hot while others are cold you usually want to split the array of structs into two arrays of structs, one with the cold part and one with the hot. The use case Go optimizes for is not common in the fields I have been working in. I think the default should be to minimize size and that it should be possible to opt out for the rare case where exact order matters.
- scottlamb 5y ago
  Counterexample: the struct in the article. It's "large enough that you care about shaving padding" in aggregate. The 2 optimal arrangements eliminate padding so it uses 1/16th of an x86-64 cache line. This guarantees the whole struct is on one cache line and uses the least total cache, so the 2 arrangements with padding are strictly worse, regardless of which fields are hot or cold. A struct of arrays form may be better still, but that's more than rearranging fields and beyond the compiler to produce. More generally, there are surely cases where it's better to arrange all the hot fields together at the expense of increasing the struct size, but I'm unsure it's the common case, and my bullshit alarm fires when people say so without evidence. (Even more so if they're also saying programmers are commonly arranging the fields optimally by hand.)

deleted 5y ago

[deleted]

tedunangst 5y ago

If your struct is large enough that you care about shaving padding, you probably have hot fields and cold fields and the best cached based win will be arranging them contiguously.

morelisp 5y ago
If I have a [10000]T I need to shave very little padding before I see some impact, even though both the padded and unpadded T might be relatively small. Specifically in Go, a smaller size can get even lone values into a smaller size class. Saving one byte may save you 384 if it's from 2305 to 2304.
fredophile 5y ago
The example in the link had 3 fields and padding made the structure 50% bigger than it needed to be. That probably likely big enough to have hot and cold fields but clearly they cared about shaving padding.
anonymoushn 5y ago
Most structs that care about shaving padding will be very small.
mcherm 5y ago
Actually, you care about shaving padding when the FIELDS are small compared to the alignment, not the structs. This frequently is a concern in Go.
pornel 5y ago
If you have a large-enough group of distinctly cold fields, a better option may be to move them to another struct, possibly allocated out of line.
usrusr 5y ago
Can go structs be nested by value? (As opposed to by reference) I'd imagine that this could be a very practical way to to do a manual hot/cold grouping in an environment that prioritizes packing over keeping order. (if the cold ones are particularly cold you might of course prefer them to be in a by reference nested struct anyways, so the hot subset packs better with their peers in an array)
jeltz 5y ago
That does not match my real world experience. If you have an array of small structs then the padding matters. And if some fields are hot while others are cold you usually want to split the array of structs into two arrays of structs, one with the cold part and one with the hot. The use case Go optimizes for is not common in the fields I have been working in. I think the default should be to minimize size and that it should be possible to opt out for the rare case where exact order matters.
scottlamb 5y ago
Counterexample: the struct in the article. It's "large enough that you care about shaving padding" in aggregate. The 2 optimal arrangements eliminate padding so it uses 1/16th of an x86-64 cache line. This guarantees the whole struct is on one cache line and uses the least total cache, so the 2 arrangements with padding are strictly worse, regardless of which fields are hot or cold. A struct of arrays form may be better still, but that's more than rearranging fields and beyond the compiler to produce. More generally, there are surely cases where it's better to arrange all the hot fields together at the expense of increasing the struct size, but I'm unsure it's the common case, and my bullshit alarm fires when people say so without evidence. (Even more so if they're also saying programmers are commonly arranging the fields optimally by hand.)