4 ms·

> about 3x the overhead to convert from int to float. That 3x overhead has nothing to do with converting types. A CPU can't compute x += y before it has the o

> about 3x the overhead to convert from int to float. That 3x overhead has nothing to do with converting types. A CPU can't compute x += y before it has the old value of the x. This is called "data dependency chain". Modern AMD64 CPUs have 1 cycle of latency to add integers, 3 cycles of latency to add FP32 or FP64 numbers. This means no matter what else is in the loop, a loop which increments a single float or double variable can't possibly be faster than 3 cycles / iteration. As demonstrated by the OP, Apple M1 is very similar in that regard.

vikingerik 4y ago
Parallelism can break that speed limit, at least in theory. You could use four different registers to hold x+y, x+2y, x+3y, x+4y and run four instances of the loop in parallel, if there are no other data dependencies. I'm not sure if hardware microcode can be smart enough to detect that and do it with register renaming and such, but a sufficiently smart compiler could. (Or realistically it would be optimized into SIMD instructions; what I said in the first paragraph is really an ad-hoc implementation of SIMD.)
- Const-me 4y ago
  > run four instances of the loop in parallel Indeed, here’s an example which uses even higher latency FMA instructions to accumulate: https://stackoverflow.com/a/59495197/126995 https://stackoverflow.com/a/59495197/126995 The example uses 4 independent accumulators to saturate the throughput, instead of stalling on the latency. BTW, on most modern CPUs that code doesn’t saturate the compute throughput, the bottleneck is loads. > a sufficiently smart compiler could They tend to avoid doing that for floats, for accuracy reasons: float addition ain’t associative.
  - makapuf 4y ago
    > They tend to avoid doing that for floats, for accuracy reasons: float addition ain’t associative. Indeed, unless you use -ffast-math (also known as -fidontcareaboutthe results). See https://stackoverflow.com/questions/7420665/what-does-gccs-ffast-math-actually-do#22135559 https://stackoverflow.com/questions/7420665/what-does-gccs-f...
    - erik_seaberg 4y ago
      To be fair, lots of devs haven’t studied IEEE 754 error analysis, and don’t know exactly how much precision they can rely on either way. I know I don’t, I just sort of assume the worst.
jeffbee 4y ago
Isn't a cast from int to float going to generate CVTSQ2SD or similar instruction, that are slow and high latency? On Ice Lake Xeon these are faster but as recently as Skylake they had reciprocal throughput of 2.
- Const-me 4y ago
  Yeah, cvtsi2sd has 3-9 cycles or latency on Zen3, but throughput ain’t too bad. The table on uops.info says on Zen3, cvtsi2sd has 1 cycle throughput, meaning a core can complete these instructions every cycle. There’s no other dependencies so the pipeline will deal with the latency just fine, by dispatching instructions for multiple loop iterations. It can’t deal with the latency of ARM64 equivalent of addss/addsd due to the dependency chain. I think that’s the bottleneck of the OP’s microbenchmark.

vikingerik 4y ago

Parallelism can break that speed limit, at least in theory. You could use four different registers to hold x+y, x+2y, x+3y, x+4y and run four instances of the loop in parallel, if there are no other data dependencies. I'm not sure if hardware microcode can be smart enough to detect that and do it with register renaming and such, but a sufficiently smart compiler could. (Or realistically it would be optimized into SIMD instructions; what I said in the first paragraph is really an ad-hoc implementation of SIMD.)

Const-me 4y ago
> run four instances of the loop in parallel Indeed, here’s an example which uses even higher latency FMA instructions to accumulate: https://stackoverflow.com/a/59495197/126995 https://stackoverflow.com/a/59495197/126995 The example uses 4 independent accumulators to saturate the throughput, instead of stalling on the latency. BTW, on most modern CPUs that code doesn’t saturate the compute throughput, the bottleneck is loads. > a sufficiently smart compiler could They tend to avoid doing that for floats, for accuracy reasons: float addition ain’t associative.
- makapuf 4y ago
  > They tend to avoid doing that for floats, for accuracy reasons: float addition ain’t associative. Indeed, unless you use -ffast-math (also known as -fidontcareaboutthe results). See https://stackoverflow.com/questions/7420665/what-does-gccs-ffast-math-actually-do#22135559 https://stackoverflow.com/questions/7420665/what-does-gccs-f...
  - erik_seaberg 4y ago
    To be fair, lots of devs haven’t studied IEEE 754 error analysis, and don’t know exactly how much precision they can rely on either way. I know I don’t, I just sort of assume the worst.

jeffbee 4y ago

Isn't a cast from int to float going to generate CVTSQ2SD or similar instruction, that are slow and high latency? On Ice Lake Xeon these are faster but as recently as Skylake they had reciprocal throughput of 2.

Const-me 4y ago
Yeah, cvtsi2sd has 3-9 cycles or latency on Zen3, but throughput ain’t too bad. The table on uops.info says on Zen3, cvtsi2sd has 1 cycle throughput, meaning a core can complete these instructions every cycle. There’s no other dependencies so the pipeline will deal with the latency just fine, by dispatching instructions for multiple loop iterations. It can’t deal with the latency of ARM64 equivalent of addss/addsd due to the dependency chain. I think that’s the bottleneck of the OP’s microbenchmark.