3 ms·

> I'd think that a fundamental problem is memory I/O bandwidth, and (depending on cache policy) poor cache hit ratios. You seem to be assuming that a vectorize

> I'd think that a fundamental problem is memory I/O bandwidth, and (depending on cache policy) poor cache hit ratios. You seem to be assuming that a vectorized loop over an array of structs only ever uses one element from the struct and ignores the rest. In that case, then yes, you will get some bandwidth problems, but modern hardware prefetchers are so good that a vectorized loop that only looks at each Nth byte or whatever is actually much faster than one might think. I'll also note that the serial (non-SIMD) case also suffers here, but often not quite so badly as the SIMD case suffers. However, there's lots of cases where you want the whole struct. E.g. complex numbers are typically a struct of two real numbers, and SIMD arithmetic on vectors of structs of complex numbers is able to work just fine with the full theoretical SIMD performance improvement.

CoastalCoder 2y ago
> You seem to be assuming that a vectorized loop over an array of structs only ever uses one element from the struct and ignores the rest. That is the worst case scenario of my concern with memory bandwidth. More generally, I'd imagine that any good roofline analysis would need to consider this data-layout issue. > but modern hardware prefetchers are so good that a vectorized loop that only looks at each Nth byte or whatever is actually much faster than one might think. I believe you. I wasn't really thinking about how efficiently the processor can pull the correct bytes into the register lanes. I was just focusing on the fundamental memory <--> CPU xfer part of the process. TL;DR: I'm trying to get better at using roofline analysis in my optimization efforts.