2 ms·
I assume that he referred to the fact that the code does not need an array in memory, with constant values that must be accessed with indexed loads. Using such
by adrian_b 5y ago
I assume that he referred to the fact that the code does not need an array in memory, with constant values that must be accessed with indexed loads. Using such a table in memory can be relatively slow, as proven by the benchmarks for this method.
There are constants, but they are used directly as arguments for the AVX-512 intrinsics. They must still be loaded into the AVX-512 registers from memory, but they are loaded from locations already known at compile-time and the loads can be scheduled optimally by the compiler, because they are not dependent on any other instructions.
For a table stored in an array in memory, the loads can be done only after the indices are computed and the loads may need to be done in a random order, not in the sequential order of the memory locations. When the latencies of the loads cannot be hidden, they can slow a lot any algorithm.
- orlp 5y agoIf your program is sufficiently large and diverse the constants embedded in the instructions are no better than a table load/lookup. The jump to the conversion routine is unpredictable, and the routine will not be in the instruction cache, causing the CPU to stall while waiting for the instructions to load from RAM. This will never show up in a microbenchmark where the function's instructions are always hot. In fact, a lot of microbenchmarking software "warms up" the code by calling it a bunch of times before starting to time it, to maximize the chances of them being able to ignore this reality.
- dzaima 5y agoThe AVX-512 constants aren't embedded in the instructions, but the address to load being static means the CPU can start to cache them when the decoder gets to them, possibly even before it knows that the input to the function is. In contrast, for the scalar code, the CPU must complete the divisions (which'll become multiplications, but even those still have relatively high latency) before it can even begin to look for what to cache.
- nitwit005 5y agoThe code itself sits in memory, which means the "table" is still in memory.