4 ms·
The article describes three optimizations: - Changing the memory layout to be column-oriented (i.e. for greater cache friendliness, as the sibling comment ment
by vitus 7y ago
The article describes three optimizations:
- Changing the memory layout to be column-oriented (i.e. for greater cache friendliness, as the sibling comment mentions).
- Iterating over batches (analogous, possibly equivalent to loop unrolling).
- Vectorizing.
Iterating over batches (especially going from batches of 1 -> 1024) could very well explain a large portion of the "interpretation overhead" that was removed, since it'd basically just be checking loop conditions and jumping back to the beginning of the loop; in that case, it can be a significant portion of the overall execution, and batching in this way reduces that overhead by 1000x.
And sure, vectorizing might only add up to a 4x speedup, but taken with these other changes (especially don't discount cache coherence), I could totally see where the 10x comes from. So the submission title is somewhat misleading to attribute all of it to vectorization (the article title also cites the community), but I could see it being valid, since these were steps needed to take full advantage of vectorization.
I do object to the measurement of execution in instructions, since in x86 not every instruction is created equal (multiplications are the poster child of "instructions that take a long time", and certain AVX instructions can cause throttling in Intel CPUs). That said, the article does cite a nearly 9x speedup, which is solid.